Optimal Bi-Level Scheduling Method of Vehicle-to-Grid and Ancillary Services of Aggregators with Conditional Value-at-Risk

: With the global net-zero strategy implementation, decarbonisation of transport by massive deployment of electric vehicles (EVs) has been considered to be an essential solution. However, charging EVs and integration into electricity grids is going to be a fundamental challenge to future electricity systems. Hence, in this situation, how to effectively deploy massive numbers of EVs, and in the meantime what can be developed to deliver vehicle-to-grid (V2G) services, become a fundamental yet interesting tech-economical issues. Furthermore, uncertainty in lack of vehicle availability and EV battery degradation could lead to revenue loss when using EVs as ancillary services aggregators. With such considerations, this paper presents a new optimised V2G aggregator scheduling service that has taken into consideration of a number of risks, including EV availability and battery degradation through conditional value-at-risk. The proposed method for V2G scheduling service, as an independent aggregator, is formulated as a bi-level optimisation problem. The performance of the proposed method is to be evaluated through case studies on the Birmingham International Airport parking lot with onsite renewable generation. Uncertainties of EVs and the differences in weekdays and weekends are also compared.


Introduction
With the rapid development of electric vehicles (EVs), the role of EVs in the energy market has received ever increasing attention. The increase in the penetration rate of electric vehicles has led to greater challenges in EV charging management and scheduling, which will also affect the reliability of the grid system.
An EV can be seen as a stationary battery when it is parked or being charged. In addition to coordinated charging, there have been a number of emerging research works on vehicle-to-grid (V2G) in recent years. The optimisation principle of V2G is to maximise the benefit of grid connected vehicles through various ancillary services [1,2]. In addition, small-scale EV charging stations can be aggregated to participate into ancillary market for large EV fleets [3].
The economic advantages of V2G are eye-catching. It can provide additional income for car owners to offset the cost of running EVs and can further justify the economic benefits of replacing conventional combustion engines with EVs. From the perspective of the grid and energy suppliers, EV and V2G can help balancing the grid demand and supply, as well as providing reactive power compensation, which can improve the stability and reliability of the grid [4][5][6]. In the meantime, in [7,8], the authors have presented an integration of V2G and wind/solar energy.
However, the uncertainty of V2G may have a huge impact on the system performance that cannot be ignored, including unpredictable EV charging behaviors and the uncertainty (1) The optimisation method has taken into account the risk of loss of revenue due to lack of vehicle charging capacity to provide service and EV battery degradation, and the CVaR was used to mitigate the uncertainties. (CVaR, also known as expected shortfall, was originally used to evaluate the market and credit risk of investment portfolios [22,23]). (2) A preferred operating point will be suggested within the ancillary capacity, with consideration of the onsite renewable generation and the above risk.
This paper presents a bi-level scheduling model to maximise the profit by considering uncertainties in participated vehicles through CVaR in Section 2 and explain the details of the bi-level problem in Section 3. Section 4 combines the bi-level model with conditional value-at-risk, the impact of risk aversion parameters is also evaluated. Table 1 is the explanation of the abbreviations in the equation

Bi-Level Scheduling Method for Vehicle-to-Grid and Ancillary Services
A bi-level scheduling method is divided into two problems. The upper problem is calculated by subtracting the cost of the aggregator from revenue. The lower problem aims to maximise the profits of electric car owners. In addition to introducing and analyzing how to use the bi-level to optimise the design of charging stations, this method also considers uncertainties study methodology, risk management and the system adjustment signal, so that the accuracy of the results can be further improved.

Upper-Level Problem
In the upper-level model, when a renewable energy charging station charges an electric vehicle, it can provide services that trade with the grid. In Equation (1), F CS is the upper-level profit of aggregators. B and C are the upper-level income and the upper-level cost, respectively.
The upper-level cost (C) is composed of three parts, namely charging cost, distributed power generation cost and energy storage cost, as shown in Equation (2). C = k c f c c f P Nc f + k rdg c rdg P Nrdg +k es c esp P Nes + k es c ese E Nes (2) where c c f , c rdg , c esp and c ese stand for capital cost of charging facility, renewable distributed generation, energy storage in power and the energy storage in energy rating, respectively. P Nc f , P Nrdg , P Nes , and E Nes stands for installed capacity of charging of charging facility, charging of renewable distributed generation, energy storage power capacity and EVs. and energy rating capacity. The upper-level income (B) is the sum of the charging power price and the power exchanged with the grid, including the maintenance costs of charging facilities, renewable distributed power generation and energy storage.
where δ ch t and δ int t are charging price and price of exchanged power between grid. P ch t and P int t stands total charging power of EVs and exchanged power between grid. c c f m , c rdgm and c esm are the annual maintenance cost of charging facility, renewable distributed generation, and energy storage. Preferred operating point at time t (POP i (t)) consists of two parts: Positive preferred operating point (POP Pos,i (t)) and negative preferred operating point (POP Neg,i (t)) at time t.
The Equation (5) shows the cost of battery degradation (Deg i (t)), which is figure out by multiply the battery cost per kilowatt with all energy deliver from vehicle to grid. BatC is battery replacement cost. MinP i (t) and RSRP i (t) stands for minimum additional power draw and responsive reserve power draw.
The decision variables are POP Pos,i , POP Neg,i , MaxP i , MinP i and RSRP i of per EV per hour in this model. The constrains of this model illustrated by Equations (6)- (15). Firstly, Equations (6) and (7) limit the battery maximum charge during the parking time by the battery capacity, while it must be positive: There is an assumption that make sure the state of charge should be reach to 80% of total capacity when the EV departure. The constraint is illustrated as Equation (8): where FPD i is final power draw at time t with considering the regulation and reserve service. E f i is the electric vehicle battery charger efficiency. M C,i is maximum battery capacity of the electric vehicle.
Beyond that, the variables of the first hour during a day show as inequalities (9)-(11): The positive preferred operating point and a negative operating point each hour are limited by the maximum charge and discharge rate as in the following equation. These constraints also consider the regulation by additional power draw and response reserve of the system: The upper-level model consists of the main aim function which is limited in Equations (16)- (30). The main solution is to define the optimal renewable energy source charging station configuration design and pricing strategy according to the renewable energy source charging station aggregators' perspective.
The equations of the upper-level model correspond to the maximum profit of the renewable energy charging station aggregator. The variables of the objective function can be divided into decision variables and operation variables. Decision variables include energy storage, energy storage power capacity, installed capacity of charging facility, installed capacity of renewable distributed generation, and charging tariff upper limit. Operation variables include total power output of renewable distributed generation, energy storage charging, energy discharging power, exchanged power with the power grid, and total power charging of EVs.
The constraints (16)- (19) set the maximum and minimum limits of charging facility, renewable distributed generation and energy storage charging capacity in an indepen-Energies 2021, 14, 7015 5 of 16 dent renewable energy source charging station, where P Nc f , P Nrdg , P Nes , and E Nes stands for installed capacity of charging of charging facility, charging of renewable distributed generation, energy storage power capacity and EVs. and energy rating capacity.
In order to maintain the stability of the electric vehicle charging market, constraint (20) sets the charging price δ ch limit provided by the renewable energy charging station aggregator within a predetermined range. A benchmark can be added to determine price variability: Constraint (21) restricts the exchange of energy between the grid (P int t ) and the electric vehicle charging station and cannot exceed the maximum capacity of the substation transformer: Constraint (22) set the limit of the maximum power that renewable charge stations can use from renewable distributed generation. Where P rdg and P Nrdg stands for total power output of renewable distributed generation and installed capacity of charging of renewable distributed generation.
The above constraints (23)-(25) limit the charging and discharging of energy through charging facilities and energy storage. P ch , P esc and P esd stands for total charging power of EVs, energy storage charging power and energy storage discharging power.
Constraints (26) and (27) illustrate the limits of energy storage through the state of charge when participating in auxiliary services. E es is the energy of the storage, SOC es is energy storage state of charge. η esc is efficient of energy storage charging.
Constraints (28) ensure that the energy at the end of the case must be equal to the energy at the beginning of the case to ensure the sustainability of renewable charging stations.
The balance of the charging and the discharging system is restricted by constraints (29), where P rdg and P int t stands for total power output of renewable distributed generation and exchange of energy between the grid Equation (30) links the upper model with the lower model by making the total energy demand of the EV owner equal to the energy provided to the EV.

Lower-Level Problem
In the lower-level model, the profit of electric vehicle owners can be maximised through the following objective function: where F EV t is profit of electric vehicle owners; U EV t is the marginal utility and C EV t is the cost of electric vehicle users.
Taking into account the price provided by renewable charging stations, the behavior of electric vehicles participating in auxiliary services. In this paper, the profit of electric vehicle owners is calculated by subtracting the cost of charging the electric vehicle from the marginal utility. Reference [24] explained that the cost of EV users to charge their cars decreases with the increase of energy from renewable charging stations. The marginal utility (U EV t ) and cost of electric vehicle users C EV t are shown in Equations (32) and (33) where d ev t is total charging of the EV. δ EV t is the charging price.
The equations above are subject to: Constraints (34) and (35) set the upper and lower limits of the total charging of the EV.
Constraints (36) and (37) limit the charging of each EV block. Since there are lowerlevel discrete variables, this is a non-convex problem. Therefore, the proposed method in this paper applied Karush-Kush-Tucker (KKT) to convert the bi-level program into a single linear program.

Uncertainties Study Methodology
Assume that the behavior of electric vehicles obeys a truncated Gaussian distribution. Where SOC I,i is state of charge at the initial moment of the ith electric vehicle.
In this model, the time of EV arrival and departure from the charging point follows a truncated Gaussian distribution.
where t arr i and t dep i is EV arrival and departure time. According to Equation (42), the EV parked in the parking lot at time t (i EV t ) is calculated by the number of EVs at time t − 1 and the EV arriving/departing at time t.

Profit Risk Management of Electric Vehicle (EV) Charging Stations
Different types of EV and charging strategies cause the profit of charging stations to be a random variable. Therefore, the authors introduced a risk-neutral equation to avoid the bad profit distribution of EV charging stations. According to references [26,27], several methods have been widely used to solve the high profit variability and negative profit problems of charging stations, such as shortage probability, variance, expected shortage, etc. CVaR was chosen to mitigate the risk in engineered investments in the proposed method [21][22][23]. The mathematical expression of CVaR is as follows: where ξ is the VaR, confidence level has been presented as α, ρ s and η s present probability and the difference between profit and the VaR non-negative variable. When the profit smaller than VaR, the η s present difference between them, otherwise, η s is zero. The objective function of upper level can be expressed as: subject to: β is the risk-aversion parameter. With the increase of β, the objective of aggregators become more risk averse system. When β equals to zero, the objective of aggregators become risk neutral system.
Considering the objectives and constraints of conditional risk management respectively, it is a mixed-integer linear program that requires the application of high-performance solvers.

The System Adjustment Signal of Aggregator
The EV charging station is a special aggregator. The system adjustment signal is the key to this aggregator system [24]. EVs will respond to the signal by adjusting the behavior of charge/discharge [28]. Aggregators receive the signals from the energy market and analyse them by integrated computing, then send the charge/discharge signal to EVs in the parking lot [29]. Figure 1a,b below shows the regulatory scheduling is independent of responsive reserve scheduling. The results of two dispatches can be combined finally to obtain the behaviour of each kind of EV.

Solution to the Bi-Level Problem
The previous part introduced the method used to solve the bi-level problem in this research. First, the KKT condition can convert a lower-level model into a series of constraints of a higher-level model by adding a Lagrange multiplier variable. After transmission through KKT, the lower level becomes a non-linear constraint set. Therefore, the "linprog" that solves the single-stage problem is no longer applicable to the bi-level model [24]. The following sections explain the details of the function.
Since the Lagrangian function with Lagrangian multiplier is a non-linear problem, (34)-(37) can be expressed as:

Solution to the Bi-Level Problem
The previous part introduced the method used to solve the bi-level problem in this research. First, the KKT condition can convert a lower-level model into a series of constraints of a higher-level model by adding a Lagrange multiplier variable. After transmission through KKT, the lower level becomes a non-linear constraint set. Therefore, the "linprog" that solves the single-stage problem is no longer applicable to the bi-level model [24]. The following sections explain the details of the function.
Since the Lagrangian function with Lagrangian multiplier is a non-linear problem, (34)-(37) can be expressed as: Besides the non-linear constraints, there are nonlinear elements in the objective function such as δ ch P ch .
Objectives with non-linear elements have been transferred to the standard mixedinteger linear program as constraints (55)-(67).

Case Study
In this paper, the case study is included using Birmingham International Airport as an example for a duration of 24 h. In this case, it is assumed there are 10,000 EVs. Suppose there are three types of electric vehicle in the parking lot: 30% Nissan Leaf (2011 model), 50% Tesla Model 3 (long range) and 20% Audi e-tron. The assumption is made against market share of battery electric vehicle in the UK by aggregating similar body type, engine and battery sizes [30]. A charge-discharge rate of 90% was selected.

Electricity Spot Price Data
The electricity spot price is from the European Power Exchange (EPEX SPOT) on 11 February 2020 as shown in Figure 2 below. This is a representative day of an average market day and the weather was mild. The currency of the real-time electricity price is converted to U.S. dollars ($/MW) based on real-time exchange rates. Simulation and optimisation are completed by MATLAB. Figure 2 above shows that at 4 a.m., the electricity price reached a trough value, which was 14.73 $/MW. and there were two peak electricity prices at 3.00 p.m. and 8.00 p.m. In the following analysis, the fluctuation of electricity prices will be combined to explain the charging behavior of EV users and the impact on the profit of aggregators.

Application of a BASIC Bi-Level Service Scheduling Method
Based on the electricity market price shown in Figure 2, this paper conducts a bi-level model case study on three different brands of cars, in which the profit of electric vehicle owners needs to be considered. The result of the preferred operating point is shown in Figure 3.  Figure 2 above shows that at 4 a.m., the electricity price reached a trough value, which was 14.73 $/MW. and there were two peak electricity prices at 3.00 p.m. and 8.00 p.m. In the following analysis, the fluctuation of electricity prices will be combined to explain the charging behavior of EV users and the impact on the profit of aggregators.

Application of a BASIC Bi-Level Service Scheduling Method
Based on the electricity market price shown in Figure 2, this paper conducts a bi-level model case study on three different brands of cars, in which the profit of electric vehicle owners needs to be considered. The result of the preferred operating point is shown in Figure 3.   Figure 2 above shows that at 4 a.m., the electricity price reached a trough value which was 14.73 $/MW. and there were two peak electricity prices at 3.00 p.m. and 8.0 p.m. In the following analysis, the fluctuation of electricity prices will be combined to ex plain the charging behavior of EV users and the impact on the profit of aggregators.

Application of a BASIC Bi-Level Service Scheduling Method
Based on the electricity market price shown in Figure 2, this paper conducts a bi-leve model case study on three different brands of cars, in which the profit of electric vehicl owners needs to be considered. The result of the preferred operating point is shown i Figure 3. In the first and second valleys of market prices, electric vehicles will be limited by battery capacity and maximum charge/discharge rates. According to the simulation results shown in Figure 3, if a bi-level model is adopted, the profit of electric vehicle owners must be considered. From midnight to 7 a.m., the load generated by the three types of car charging is similar, which means that the charging behavior of the three different types of cars during this period is similar. However, as time goes by, a large number of EVs migrate from residential areas to workplaces when they arrive at work. From 9.00 a.m. to 11.00 a.m., these three types of cars all participated in the V2G mode, which means that the EV is used as an energy storage device to reverse charge. This eased the grid load during this period. The changes in electricity market prices lead to changes in charging behavior, which resulting in different power draws. These changes are gradually reflected after 12.00 p.m. in Figure 4. 11.00 a.m., these three types of cars all participated in the V2G mode, which means that the EV is used as an energy storage device to reverse charge. This eased the grid load during this period. The changes in electricity market prices lead to changes in charging behavior, which resulting in different power draws. These changes are gradually reflected after 12.00 p.m. in Figure 4.

Uncertainties of EV Charging Behavior
The time when an electric vehicle is connected to the power grid and when it leaves the grid is random [9]. Due to uncertainty in the driving habits of EV owners, many studies have adopted truncated Gaussian distributions as input. The following equation describes the probability of occurrence of each EV behavior. Table 1 lists the probability distribution of electric vehicle behavior. Electric car owners tend to charge during the low market price, the charging behavior of EV owners to Nissan Leaf (2011 model), Tesla Model 3 (long range) and Audi e-tron are similar according to the results, so a case will be presented on the Audi e-tron model only.
It can be seen from Figure 5 that the time for the EV to leave the original location and arrive at the second location is around 9 a.m., and the market price reaches the first peak at 10.30 a.m. within a day. As market prices rise, aggregators will decide to start V2G and

Uncertainties of EV Charging Behavior
The time when an electric vehicle is connected to the power grid and when it leaves the grid is random [9]. Due to uncertainty in the driving habits of EV owners, many studies have adopted truncated Gaussian distributions as input. The following equation describes the probability of occurrence of each EV behavior. Table 1 lists the probability distribution of electric vehicle behavior.
Electric car owners tend to charge during the low market price, the charging behavior of EV owners to Nissan Leaf (2011 model), Tesla Model 3 (long range) and Audi e-tron are similar according to the results, so a case will be presented on the Audi e-tron model only.
It can be seen from Figure 5 that the time for the EV to leave the original location and arrive at the second location is around 9 a.m., and the market price reaches the first peak at 10.30 a.m. within a day. As market prices rise, aggregators will decide to start V2G and sell energy at high prices. By contrast, around 8 p.m., although the market price reached the second peak on the day, the electric cars are usually still in a charging state.
The figure above shows the results of regulation up, regulation down and responsive reserve power scheduling. Since only a few electric vehicles choose to park overnight, the sensitivity is high at night. The overall trend is a negative correlation between market prices and liquidation behavior. By contrast, the relationship between charging behavior and market price is positively correlated.
Energies 2021, 14, x FOR PEER REVIEW 12 of sell energy at high prices. By contrast, around 8 p.m., although the market price reache the second peak on the day, the electric cars are usually still in a charging state. The figure above shows the results of regulation up, regulation down and responsiv reserve power scheduling. Since only a few electric vehicles choose to park overnight, th sensitivity is high at night. The overall trend is a negative correlation between mark prices and liquidation behavior. By contrast, the relationship between charging behavi and market price is positively correlated.

The Different Result of Peak Time and Workdays
The market price of Figure 6 comes from EPEX SPOT. This paper uses data from 1 February 2020 and 4 April 2020 (Easter). The average market price is greater than th working day. However, most families prefer to travel during weekends, which mean more electric cars will be parked in the international airport parking lot. Therefore, th aggregator has more dispatchable electric vehicles during the holidays or weekends.

The Different Result of Peak Time and Workdays
The market price of Figure 6 comes from EPEX SPOT. This paper uses data from 11 February 2020 and 4 April 2020 (Easter). The average market price is greater than the working day. However, most families prefer to travel during weekends, which means more electric cars will be parked in the international airport parking lot. Therefore, the aggregator has more dispatchable electric vehicles during the holidays or weekends.      At the same time, Figure 8 shows the proportion of regulation down, regulation u and responsive reserve on weekdays and weekends. Compared to working days, the pro portion of response reserves has increased by approximately 5%. The basic trend of elec tric vehicle behavior is the same as before. However, as passenger traffic increases durin the weekends, aggregators can arrange more dispatchable electric vehicles. Therefore, th average power consumption is greater than the working day. When the number of electri At the same time, Figure 8 shows the proportion of regulation down, regulation up and responsive reserve on weekdays and weekends. Compared to working days, the proportion of response reserves has increased by approximately 5%. The basic trend of electric vehicle behavior is the same as before. However, as passenger traffic increases during the weekends, aggregators can arrange more dispatchable electric vehicles. Therefore, the average power consumption is greater than the working day. When the number of electric vehicles reaches a certain level, the percentage of battery deregulation costs in total profits will decrease. In other words, aggregators will earn more profits during holidays than workdays.
Energies 2021, 14, x FOR PEER REVIEW 14 vehicles reaches a certain level, the percentage of battery deregulation costs in total p will decrease. In other words, aggregators will earn more profits during holidays workdays.  Figure 9 shows the influence of risk aversion parameters on the optimal pr Change the risk aversion parameter from 0.1 to 2.0 in the objective function of the u level model.    Figure 9 shows the influence of risk aversion parameters on the optimal project. Change the risk aversion parameter from 0.1 to 2.0 in the objective function of the upperlevel model. The daily profit of the parking lot is $7329 when the risk aversion parameter is equal to 0.1. As the risk aversion parameters decrease, the profit of the aggregator will decrease. The daily profit of the parking lot is only about $1536 when the risk aversion parameter is equal to 2.0. Aggregators can obtain more profits through lower risk aversion parameters. Therefore, it is recommended that the aggregator operate under lower risk aversion.

Conclusions
This paper addresses the challenges in risk mitigation of EV charging available and battery degradation via CVaR. EV users have different behaviors of using electric vehicles on weekdays and weekends, which leads to uncertainty in the charging behavior of EVs. The proposed model has considered the risk of loss of revenue due to lack of vehicle The daily profit of the parking lot is $7329 when the risk aversion parameter is equal to 0.1. As the risk aversion parameters decrease, the profit of the aggregator will decrease. The daily profit of the parking lot is only about $1536 when the risk aversion parameter is equal to 2.0. Aggregators can obtain more profits through lower risk aversion parameters. Therefore, it is recommended that the aggregator operate under lower risk aversion.

Conclusions
This paper addresses the challenges in risk mitigation of EV charging available and battery degradation via CVaR. EV users have different behaviors of using electric vehicles on weekdays and weekends, which leads to uncertainty in the charging behavior of EVs. The proposed model has considered the risk of loss of revenue due to lack of vehicle charging capacity to provide service and EV battery degradation. A bi-level optimisation was formulated to determine a preferred operating point of charging for an individual EV with consideration of the uncertainties. The KKT method was used to decompose the problem and the impact of risk aversion was also analyzed. The performance of the proposed method has been evaluated through a case study on the Birmingham International Airport parking lot with onsite renewable generation.
From the results of the case study, the profit of the aggregator on weekends is 5% greater than that on weekdays. According to different charging behaviors on weekdays and weekends, the aggregator can provide different modes of operation, such as providing incentives to encourage all electric cars parked in the parking lot and participate in ancillary services. Uncertainty research is an extension of the basic model that can make research more precise. As the risk aversion parameters decrease, the profit of the aggregator will decrease, which means that the profit of the aggregator is proportional to the risk coefficient. Therefore, finding a suitable risk coefficient for any aggregator can balance the profit and risk.