Game-Theory Based V2G Coordination Strategy for Providing Ramping Flexibility in Power Systems

: Large-scale integration of renewable generation into power systems invariably a ﬀ ects the system ramping capability. However, the vehicle-to-grid (V2G) concept that allows for using electric vehicles (EVs) as energy storages with the capability of bidirectional energy transfer between the EVs and the grid, can be employed to mitigate the above adverse e ﬀ ect. This paper proposes a game-theory-based V2G coordination strategy that uses EV clusters to improve ramping ﬂexibility in power systems. In the proposed strategy, the V2G concept, representing the interactions between the distribution system operator (DSO) and EV clusters, is formulated as a Stackelberg game. The DSO acts as a leader who decides the charging prices for the buses to which the EV clusters are connected, while the EV clusters simply serve as followers, scheduling their own charging and discharging. This bi-level model is further reduced to a single-level, mixed-integer second-order cone programming (MISOCP) problem based on the Karush-Kuhn-Tucker (KKT) conditions, the strong duality theorem and second-order cone (SOC) relaxation. The performance of the proposed V2G coordination strategy on a modiﬁed IEEE 33-bus system connecting EV clusters and PV generations is investigated through simulations, and the results demonstrate that the largest ramp of the system can be reduced by up to 39% when EV clusters are providing ﬂexibility, while the EV clusters can also have greatly reduced charging costs. and L.C.; methodology, Z.J. and L.C.; software, Z.J.; validation, L.C., and formal analysis, J.Z. and L.C.; investigation, J.Z. and L.C.; J.Z. and data curation, preparation, J.Z. and J.Z., project acquisition,


Introduction
Power systems have been experiencing a fast increase in renewable energy integration all over the world. It is estimated that by 2050, solar PV power generation will account for 16% of the total electricity in the world and 43% of the added electricity in the United States [1]. However, the increasing penetration of renewable generation in power systems will impose significant challenges to the system operation [2][3][4]. A main concern is that the intermittent nature of renewables will cause fast ramping and change in the system net load over a short period of time. Reference [5] depicts California's daily net load curves, also known as "duck curves", from 2012 to 2020. Such duck curves highlight the rapid ramping of net demand under high penetration of renewables and illustrates that the system, with increasing integration of renewables, confronts the problem of a shortage of ramping capability, threatening the stable operation of power systems.
A number of researches have investigated strategies to flatten the duck curve, which can be categorized into four types. The first one is to apply renewable generation curtailment as reported in [6][7][8]. This strategy is easy to implement with small adjustments to the controllers. However, this led to a high curtailment ratio of renewable energy. The second type of strategy, proposed by 1.
We propose to use the EV clusters as a flexible resource to improve system ramping capability while fully considering the power flow congestion in the power network.

2.
In order to study V2G coordination, we propose a dynamic pricing mechanism, which models the DSO as a leader deciding the electricity trading prices at the EV-cluster-connected buses, and models the EVs as followers responding to the prices by scheduling their V2G charging and discharging.
The Stackelberg-game-based bi-level model that we propose is reformulated as a single-level mixed-integer second-order cone programming (MISOCP) problem by using Karush-Kuhn-Tucker (KKT) conditions, the strong duality theorem and second-order cone (SOC) relaxation, and thus can be efficiently solved by commercial solvers.

Active Distribution System
In this paper, we consider an active distribution system connecting multiple EV clusters as shown in Figure 1

Active Distribution System
In this paper, we consider an active distribution system connecting multiple EV clusters as shown in Figure 1 In practical system operations, the direct participation of the EVs into the demand-side management program is generally not allowed due to the low energy capacities of individual EVs. For example, in ISO-NE, the minimum capacity for providing regulation service is 1 MW, but the capacity of a single EV is generally less than 100 kW [27]. Therefore, there is a need for an agent to coordinate a cluster of EVs to meet the minimum capacity requirement as depicted in Figure 1. The agent managing an EV cluster is a third-party entity, such as a charging station owner, who has a competitive relationship with the DSO in the deregulated electricity market. In the system, V2G technology and the two-way communication infrastructure allow the bidirectional flows of power and information between the distribution system and the EV clusters.

Proposed Stackelberg-Game-Based Coordination Strategy
Before presenting the proposed strategy, it is necessary to analyze the interactions between the DSO and the EV clusters. Apparently, the EVs' charging and discharging behaviors are mainly driven by economic factors as each EV is essentially a rational economic entity. The DSO needs the EVs to provide ramp flexibilities. To this end, the DSO needs to offer economic incentives to the EV owners, who will consider the degradation of their batteries. Such an interaction between the DSO and the EV clusters constitutes a Stackelberg game, in which the DSO sets the transaction price to guide the EVs' charging and discharging, i.e., the provision of flexibility. The game is illustrated in Figure 2, and the detailed descriptions of each participant in the game are given as follows: DSO: The DSO acts as a leader to encourage the EVs to provide ramping flexibility. Based on the forecasted energy prices and ramping request in the next operation horizon, the DSO determines a In practical system operations, the direct participation of the EVs into the demand-side management program is generally not allowed due to the low energy capacities of individual EVs. For example, in ISO-NE, the minimum capacity for providing regulation service is 1 MW, but the capacity of a single EV is generally less than 100 kW [27]. Therefore, there is a need for an agent to coordinate a cluster of EVs to meet the minimum capacity requirement as depicted in Figure 1. The agent managing an EV cluster is a third-party entity, such as a charging station owner, who has a competitive relationship with the DSO in the deregulated electricity market. In the system, V2G technology and the two-way communication infrastructure allow the bidirectional flows of power and information between the distribution system and the EV clusters.

Proposed Stackelberg-Game-Based Coordination Strategy
Before presenting the proposed strategy, it is necessary to analyze the interactions between the DSO and the EV clusters. Apparently, the EVs' charging and discharging behaviors are mainly driven by economic factors as each EV is essentially a rational economic entity. The DSO needs the EVs to provide ramp flexibilities. To this end, the DSO needs to offer economic incentives to the EV owners, who will consider the degradation of their batteries. Such an interaction between the DSO and the EV clusters constitutes a Stackelberg game, in which the DSO sets the transaction price to guide the EVs' charging and discharging, i.e., the provision of flexibility. The game is illustrated in Figure 2, and the detailed descriptions of each participant in the game are given as follows: information, the cluster agent will play the game with the DSO. In the game, the cluster agent acts as a follower who minimizes the total charging costs of its clustered (managed) EVs by scheduling their charging and discharging, based on the transaction prices given by the DSO.
The Stackelberg game discussed above forms the basis for the proposed coordination strategy in this paper.  Figure 2. Stackelberg game between the distribution system operator (DSO) and the EV clusters.

Mathematical Formulation
As discussed in the previous section, the proposed coordination strategy is built on a Stackelberg game among the DSO and multiple EV clusters. In this section, we provide the mathematical model of this game. In the following, the game is formulated as a bilevel model, and is then reformulated as a single-level MISOCP problem that can be efficiently solved using commercial optimization solvers.

Model of DSO
Let re t P and load t P be the total renewable generation and the total load in the distribution grid at time slot t. So, the system net load (without considering the EV clusters) at time slot t is (3) Figure 2. Stackelberg game between the distribution system operator (DSO) and the EV clusters.
DSO: The DSO acts as a leader to encourage the EVs to provide ramping flexibility. Based on the forecasted energy prices and ramping request in the next operation horizon, the DSO determines a set of electricity prices as incentives to the EV clusters. Considering the locations of the clusters in the distribution grid, the DSO sets dynamic transaction prices: π n,min t ≤ π n t ≤ π n,max t ∀n, ∀t, where π n denotes the charging and discharging price designed by DSO for the EV cluster agent n in the operational time horizon, π n t represents the transaction prices in time slot t, and π n,min t and π n,max t denote the lower and upper bounds of the transaction prices, respectively. EV cluster agent: The EV cluster agent is responsible for coordinating the EVs in the game. The agent is equipped with a database for storing its EVs' data, including arrival and departure times, the initial and desired energy levels, etc., which could be uploaded by the EV owners upon arrival at the charging station. In fact, the EV owners may have privacy considerations when sharing such data with the agent, but the privacy issue is out of the scope of this paper and is left for future research. The ramping discussed in this paper is provided in the day-ahead power market. In the market, a cluster agent will forecast the number and the operational data of the EVs during the next operating time period based on the historical information in its database, and by using the forecasted information, the cluster agent will play the game with the DSO. In the game, the cluster agent acts as a follower who minimizes the total charging costs of its clustered (managed) EVs by scheduling their charging and discharging, based on the transaction prices given by the DSO.
The Stackelberg game discussed above forms the basis for the proposed coordination strategy in this paper.

Mathematical Formulation
As discussed in the previous section, the proposed coordination strategy is built on a Stackelberg game among the DSO and multiple EV clusters. In this section, we provide the mathematical model of this game. In the following, the game is formulated as a bilevel model, and is then reformulated as a single-level MISOCP problem that can be efficiently solved using commercial optimization solvers.

Model of DSO
Let P re t and P load t be the total renewable generation and the total load in the distribution grid at time slot t. So, the system net load (without considering the EV clusters) at time slot t is When multiple EV clusters are participating in providing the ramping flexibility, the ramping of the net load between two consecutive time slots and the ramp vector during the entire time horizon are given in (4) and (5), respectively: where E m,n t and e m,n t respectively denote the charging and discharging powers of EV m coordinated by cluster agent n.
Therefore, the largest ramp of the system net load during the time horizon is where · ∞ represents the infinity-norm.
In the Stackelberg game shown in Figure 2, the DSO aims at minimizing the cost of obtaining the ramp flexibility from all participating EV clusters. So, according to (1)-(6), the DSO's objective function is: where the first term denotes the cost of the ramp with a quadratic function according to [28], a and b are the positive cost coefficients, and the second term is the revenue obtained from the transaction with the EVs. Since π n t , E m,n t and e m,n t are decision variables, the objective function (7) is neither convex nor linear. From (7), it can be seen that the appropriate transaction prices π n t set by the DSO will address the ramp shortage issue due to the fact that it will influence the participating EVs' charging and discharging behaviors.
Next, we present how the power flow congestion in the distribution grid is modeled in the game. A distribution grid is typically a radial power network, as shown in Figure 1. The power flows of a radial network can be solved by the distflow branch model [29], as given below: Energies 2020, 13, 5008 where ( j) is the set of bus indices that are connected to bus j; R ij and X ij are branch ij's resistance and reactance, respectively; P ij t and Q ij t respectively denote branch ij's active and reactive power flows; P

Model of an EV Cluster
As a follower in the Stackelberg game, each EV cluster agent aims at minimizing the total charging and discharging cost of its clustered EVs. Therefore, the objective function of an EV cluster n is: In terms of the EVs' charging powers, the EV cluster agent should consider the following constraints for its clustered EVs: Complementary slackness conditions: 0 ≤ e n,m t ⊥µ n,m 3,t ≥ 0 t ∈ t n,m in , t n,m out , ∀n, ∀m 0 ≤ e n,m,max where µ n,m 1,t and µ n,m 2,t are the dual variables of constraint (15), µ n,m 3,t and µ n,m 4,t are the dual variables of constraint (16), η n,m 1,t and η n,m 2,t are the dual variables of constraint (17), ζ n,m is the dual variable of constraint (18), µ n,m 5,t and µ n,m 6,t are the dual variables of constraint (19), and the symbol ⊥ denotes that the inner product of the two vectors equals zero.
Due to the existence of non-linearities in the complementary slackness conditions of (22)-(27), they are linearized by using the Big-M method as follows: 0 ≤ e n,m,max − e n,m where b n,m 1,t → b n,m 6,t are binary variables, and M is a large-enough constant. According to (20) and (21), one has: By subtracting (35) from (34) and using (22) According to (26) and (27), one has: Based on (37) and (38), accumulating the equation (36) over the time horizon, one can obtain: According to the strong duality theorem, the optimal solutions to the primal and the associated dual problem will yield the same objective value [30]. Therefore, substituting (18) Therefore, the upper-level objective function can be reformulated as:

Single-Level MISOCP Model
After the linearization of the bilinear terms in the objective function (7) and those in the SOCP of the distflow branch model, the original game problem is reformulated as a compact form given as follows: (52) The model in (52) is a single-level MISOCP problem, which can be efficiently solved by commercial solvers, such as CPLEX and GUROBI.
The model in (52) is a single-level MISOCP problem, which can be efficiently solved by commercial solvers, such as CPLEX and GUROBI.

Simulation Parameters
In this section, the effectiveness and performance of the proposed game-based V2G coordination strategy is verified based on numerical simulations on a modified IEEE 33-bus distribution network with connected EV clusters, as shown in Figure 3.

Simulation Parameters
In this section, the effectiveness and performance of the proposed game-based V2G coordination strategy is verified based on numerical simulations on a modified IEEE 33-bus distribution network with connected EV clusters, as shown in Figure 3.     As depicted in Figure 3, buses 8, 24 and 30 individually connect a set of EV clusters; PV generation units are located at buses 10, 17, 21 and 33; and SVCs are connected at buses 14 and 22. The base capacity and voltage of the distribution system are set to 10 MVA and 12.66 kV, respectively. The normal operational range of the bus voltages is 0.95~1.05 p.u. Each SVC has a capacity of ±500 kVar, and each PV unit has a capacity of 1.1 MVA. For each bus, the load data from [29] is used as the peak load at 21:00, while the load data at other hours are obtained proportionally based on the load curve in Figure 4. The forecasted PV generation at each bus is depicted in Figure 5. The detailed data in

Simulation Parameters
In this section, the effectiveness and performance of the proposed game-based V2G coordination strategy is verified based on numerical simulations on a modified IEEE 33-bus distribution network with connected EV clusters, as shown in Figure 3.

Simulation Parameters
In this section, the effectiveness and performance of the proposed game-based V2G coordination strategy is verified based on numerical simulations on a modified IEEE 33-bus distribution network with connected EV clusters, as shown in Figure 3.    Each EV cluster coordinates 20 EVs. Each EV has the maximum charging/discharging power of 10 kW and a battery capacity of 50 kWh. An EV's arrival and departure times follow a Gaussian distribution [32]. In the simulation, those two times follow N~(7, 0.5) and N~(20, 0.5) in EV Cluster 1, they follow N~(6, 0.5) and N~(19, 0.5) in EV Cluster 2, and they follow N~(8, 0.5) and N~(22, 0.5) in EV Cluster 3, respectively. The arrival and departure times of EVs can be obtained by Monte Carlo simulation. The values of the desired energy levels S of the EVs in Clusters 1, 2 and 3 are randomly generated from the ranges [10,20] kWh, [15,30] kWh and [20,40] kWh, respectively. It is assumed that each EV will not depart from its charging station until being fully charged. The DSO's retail price in the market is set to be 1.5 times of the wholesale price in [33], considering the DSO's construction and maintenance cost of the distribution network. The ramp cost coefficients are obtained from [28]. In this paper, all simulations are performed in MATLAB R2016A with the optimization solver of Gurobi 9.0.2. Figure 6 shows the transaction prices determined by the DSO in the Stackelberg game. Considering the fact that the DSO needs to offer an attractive transaction price to the EVs to encourage them to join the program, the upper and lower bounds of transaction prices are set to be 1.3 and 0.4 times of the wholesale price, respectively, which are lower than the retail electricity prices. In Figure 6, it can be seen that the transaction prices reach the upper bound before 10:00 and are between the upper and lower boundaries after 10:00. This indicates that the DSO discourages charging/discharging the EVs before 10:00 and encourages them to join the program after 10:00. From Figure 6, it is also observed that the transaction price determined by the DSO for the clusters are different after 10:00. This is due to the fact that the EV clusters, being connected at different buses in the distribution network, have different impacts on the power flows and thus will cause the DSO to generate different prices.

Simulation Results
EV Cluster 3, respectively. The arrival and departure times of EVs can be obtained by Monte Carlo simulation. The values of the desired energy levels S of the EVs in Clusters 1, 2 and 3 are randomly generated from the ranges [10,20] kWh, [15,30] kWh and [20,40] kWh, respectively. It is assumed that each EV will not depart from its charging station until being fully charged. The DSO's retail price in the market is set to be 1.5 times of the wholesale price in [33], considering the DSO's construction and maintenance cost of the distribution network. The ramp cost coefficients are obtained from [28]. In this paper, all simulations are performed in MATLAB R2016A with the optimization solver of Gurobi 9.0.2. Figure 6 shows the transaction prices determined by the DSO in the Stackelberg game. Considering the fact that the DSO needs to offer an attractive transaction price to the EVs to encourage them to join the program, the upper and lower bounds of transaction prices are set to be 1.3 and 0.4 times of the wholesale price, respectively, which are lower than the retail electricity prices. In Figure  6, it can be seen that the transaction prices reach the upper bound before 10:00 and are between the upper and lower boundaries after 10:00. This indicates that the DSO discourages charging/discharging the EVs before 10:00 and encourages them to join the program after 10:00. From Figure 6, it is also observed that the transaction price determined by the DSO for the clusters are different after 10:00. This is due to the fact that the EV clusters, being connected at different buses in the distribution network, have different impacts on the power flows and thus will cause the DSO to generate different prices. Next, the impact of a cluster's coordination of the EVs' charging and discharging behaviors is analyzed. Figure 7a depicts the ramp (the change between hours) of the system net load without the EV clusters being connected, and Figure 7b-d depicts the charging and discharging powers of the EVs in different clusters. Next, the impact of a cluster's coordination of the EVs' charging and discharging behaviors is analyzed. Figure 7a depicts the ramp (the change between hours) of the system net load without the EV clusters being connected, and Figure 7b-d depicts the charging and discharging powers of the EVs in different clusters.

Simulation Results
In Figure 7a, the system net load has a negative ramp around 12:00 and a positive one around 17:00. The negative ramp is caused by the injection of a large amount of PV power and the reduction of load between 11:00 and 12:00 as shown in Figures 4 and 5, respectively. Similarly, the positive ramp is due to the PV power reduction and the load increase between 16:00 and 17:00. During the time periods of negative and positive ramps, the system needs to ramp down and up the generations, respectively, to maintain the power balance. The system may lack such a fast ramp or may need a high cost to ramp the generators. To address this problem, the proposed coordination strategy regulates the transaction prices ( Figure 6) and uses them as incentives to encourage EV clusters' charging and discharging to contribute to the system's ramping flexibility. In Figure 7a, the system net load has a negative ramp around 12:00 and a positive one around 17:00. The negative ramp is caused by the injection of a large amount of PV power and the reduction of load between 11:00 and 12:00 as shown in Figures 4 and 5, respectively. Similarly, the positive ramp is due to the PV power reduction and the load increase between 16:00 and 17:00. During the time periods of negative and positive ramps, the system needs to ramp down and up the generations, respectively, to maintain the power balance. The system may lack such a fast ramp or may need a high cost to ramp the generators. To address this problem, the proposed coordination strategy regulates the transaction prices ( Figure 6) and uses them as incentives to encourage EV clusters' charging and discharging to contribute to the system's ramping flexibility.
Although the transaction prices before 10:00 hit the upper bound (see Figure 6), they are still less than the prices at other time slots. Therefore, it can be seen from Figure 7b-d that all the EVs choose to charge before 10:00 once connected to the grid. In addition, all the EV clusters discharge at 17:00 when the system encounters the largest positive ramp. Such discharging power can effectively alleviate the system's ramp burden caused by the impact of a PV power reduction and a load increase. In other words, the proposed coordination strategy enhances the system ramping flexibility especially at the hours with the largest net-load ramp. Similarly, Figure 7b-d also demonstrate that some EVs choose to charge at 12:00 when the system experiences a negative ramp.
To further study the impact of the proposed strategy on the EVs' charging costs, Table 1 compares the charging costs of the EV clusters before and after joining the program. It is obvious that the charging cost of all the EV clusters decreases when using the proposed strategy. This demonstrates that the proposed strategy is also economically sound from the perspective of EVs and therefore can attract EV owners to join the program. Although the transaction prices before 10:00 hit the upper bound (see Figure 6), they are still less than the prices at other time slots. Therefore, it can be seen from Figure 7b-d that all the EVs choose to charge before 10:00 once connected to the grid. In addition, all the EV clusters discharge at 17:00 when the system encounters the largest positive ramp. Such discharging power can effectively alleviate the system's ramp burden caused by the impact of a PV power reduction and a load increase. In other words, the proposed coordination strategy enhances the system ramping flexibility especially at the hours with the largest net-load ramp. Similarly, Figure 7b-d also demonstrate that some EVs choose to charge at 12:00 when the system experiences a negative ramp.
To further study the impact of the proposed strategy on the EVs' charging costs, Table 1 compares the charging costs of the EV clusters before and after joining the program. It is obvious that the charging cost of all the EV clusters decreases when using the proposed strategy. This demonstrates that the proposed strategy is also economically sound from the perspective of EVs and therefore can attract EV owners to join the program. Next, we studied the performance of the DSO in the coordination. Figure 8 shows the bus voltages during the operational time period. Figure 9 compares the ramp of net load at each time slot, with and without the proposed coordination. It can be seen from Figure 8 that the bus voltages are within the given range, which is the premise for the economic dispatch of the system. In Figure 9, when the proposed coordination is applied, the largest positive ramp decreases from 1.54 MW to 0.94 MW with a reduction rate of 39% and the largest negative ramp also decreases from 1.28 MW to 0.90 MW, which effectively contributes ramp flexibility to the DSO. Moreover, Figure 9 shows that the net load has a relatively slower ramp before 10:00, which results from the DSO's strategy on regulating the electricity price, as analyzed in Figure 6.
Next, we studied the performance of the DSO in the coordination. Figure 8 shows the bus voltages during the operational time period. Figure 9 compares the ramp of net load at each time slot, with and without the proposed coordination. It can be seen from Figure 8 that the bus voltages are within the given range, which is the premise for the economic dispatch of the system. In Figure 9, when the proposed coordination is applied, the largest positive ramp decreases from 1.54 MW to 0.94 MW with a reduction rate of 39% and the largest negative ramp also decreases from 1.28 MW to 0.90 MW, which effectively contributes ramp flexibility to the DSO. Moreover, Figure 9 shows that the net load has a relatively slower ramp before 10:00, which results from the DSO's strategy on regulating the electricity price, as analyzed in Figure 6.

Conclusions
In this paper, a game-theory-based V2G EV cluster charging coordination strategy has been proposed to provide ramp flexibility for power systems that are integrated with large-scale variable Next, we studied the performance of the DSO in the coordination. Figure 8 shows the bus voltages during the operational time period. Figure 9 compares the ramp of net load at each time slot, with and without the proposed coordination. It can be seen from Figure 8 that the bus voltages are within the given range, which is the premise for the economic dispatch of the system. In Figure 9, when the proposed coordination is applied, the largest positive ramp decreases from 1.54 MW to 0.94 MW with a reduction rate of 39% and the largest negative ramp also decreases from 1.28 MW to 0.90 MW, which effectively contributes ramp flexibility to the DSO. Moreover, Figure 9 shows that the net load has a relatively slower ramp before 10:00, which results from the DSO's strategy on regulating the electricity price, as analyzed in Figure 6.

Conclusions
In this paper, a game-theory-based V2G EV cluster charging coordination strategy has been proposed to provide ramp flexibility for power systems that are integrated with large-scale variable

Conclusions
In this paper, a game-theory-based V2G EV cluster charging coordination strategy has been proposed to provide ramp flexibility for power systems that are integrated with large-scale variable renewable generations. The proposed strategy essentially couples EVs as flexibility providers for power systems that need ramp flexibility under high penetration levels of renewables. The strategy first models the interactions between the DSO and the EV clusters as a Stackelberg game, in which the Energies 2020, 13, 5008 14 of 17 DSO offers dynamic transaction prices to the EV clusters, while considering the network constraints, and the EVs reschedule their charging/discharging as a response to the dynamic prices. Next, for computational efficiency, the established bilevel model is reformulated to a single-level MISOCP problem by using KKT conditions, the strong duality theorem and SOC relaxation. Simulated results on the modified IEEE 33-bus system convincingly indicate that the proposed strategy improves the ramping flexibility of the system while reducing the charging costs of all EVs at the same time.
In this paper, the model is solved in a centralized way and the privacy issue when sharing the EVs' data is not considered. With the increasing scale of grid-connected EVs, the centralized calculation may fall into the disaster of dimensionality. Therefore, future work will be to study the privacy issue and the optimal scheduling of EVs using a distributed algorithm.