Participation of an EV Aggregator in the Reserve Market through Chance-Constrained Optimization

: The adoption of Electric Vehicles (EVs) will revolutionize the storage capacity in the power system and, therefore, will contribute to mitigate the uncertainty of renewable generation. In addition, EVs have fast response capabilities and are suitable for frequency regulation, which is essential for the proliferation of intermittent renewable sources. To this end, EV aggregators will arise as a market representative party on behalf of EVs. Thus, this player will be responsible for supplying the power needed to charge EVs, as well as offering their ﬂexibility to support the system. The main goal of EV aggregators is to manage the potential participation of EVs in the reserve market, accounting for their charging and travel needs. This work follows this trend by conceiving a chance-constrained model able to optimize EVs participation in the reserve market, taking into account the uncertain behavior of EVs and their charging needs. The proposed model, includes penalties in the event of a failure in the provision of upward or downward reserve. Therefore, stochastic and chance-constrained programming are used to handle the uncertainty of a small ﬂeet of EVs and the risk proﬁle of the EV aggregator. Two different relaxation approaches, i.e., Big-M and McCormick, of the chance-constrained model are tested and validated for different number of scenarios and risk levels, based on an actual test case in Denmark with actual driving patterns. As a ﬁnal remark, the McCormick relaxation presents better performance when the uncertainty budget increases, which is appropriated for large-scale problems.


Introduction
Electrification of transport, alongside renewable generation, are key aspects for carbon-free energy systems. The literature points out that EVs will be key resources for mitigating renewable generation effects in the power system operation [1,2]. To this end, EVs will be able to participate in energy and reserve markets through EV aggregators [3]. More precisely, the EV aggregator is the entity that collects the EVs' power flexibility to participate in the energy and reserve markets, as EVs are not individually eligible to participate in these markets because they do not meet the minimum power requirements. Within this, the authors in [4,5] proposed methodologies for the strategic participation of EV aggregators in energy and reserve markets.
The work in [5] implemented a stochastic approach that improves the EV aggregator's participation, assuming a price-taker behavior. This approach relies on the rules of the Nordic power system, namely in Eastern Denmark. It uses stochastic optimization to handle the uncertainty related to EVs driving patterns and the availability of providing the service.
The risk of failing to provide the service is one of the main concerns when EVs are proposed to participate in reserve markets, since these markets are essential to ensure power system reliability. The EV aggregator can partially mitigate this risk by managing a portfolio of many EVs, which reduces the non-compliance of the service delivery [6]. Thus, one can suggest that risk-analysis techniques are important for EV aggregator participation in reserve markets, and therefore, improve their decision-making.
In this context, this work proposes an enhanced decision-making process for EV aggregators by incorporating a risk-analysis technique. This technique will handle the risk associated with the provision of frequency services. In this work, chance-constrained programming is applied as a risk-analysis technique over the two-stage stochastic programming modeled in [5]. Chance-constrained programming can define the risk that the EV aggregator is willing to take and optimize the best strategy to participate in the reserve market in order to maximize profit. Note that for high-risk scenarios, it is assumed that the system operator will have enough liquidity, thereby covering the scenarios neglected by the chance-constrained approach. Thus, this work contributes to the development of a tool that optimizes the exclusion of problematic scenarios from the final offer of the EV aggregator. To our knowledge, this is the first chance-constrained model applied to EV aggregator participation in reserve markets. The main contributions of the study are threefold:

•
To improve the strategic offering model developed in [5], by incorporating chance-constrained stochastic programming as a risk-analysis technique; • To model the chance-constrained stochastic programming model, through the deterministic equivalent way, recurring to the Big-M and McCormick relaxation methods; • To explore the trade-off between reliability and expected profit, assessing the level of risk assumed the aggregator.
In addition to this introduction section, Section 2 provides a comprehensive review of the EV aggregator problem. Section 3 details the mathematical formulation of the addressed problem through the equivalent deterministic approach of chance-constrained programming. Section 4 provides the main results of the methodology applied to the case study presented in [5]. Finally, Section 5 gathers the main conclusions.

Related Work
In this section, different models concerning EV aggregator integration and modeling are exposed. Different approaches are considered by several authors [3,7,8], namely the participation in reserve markets. The main objective is to increase the aggregator's profit from participating in frequency regulation.
Rücker et al. studied the provision of automatic frequency restoration reserve by EVs [9]. The results proved to be positive and profitable, however, are still not enough to cover the initial investment costs. Furthermore, in [10] the benefits EVs can bring to the power system operator are analyzed. The authors developed a new algorithm intending to control the active and reactive power flow, giving to the EV owner the possibility to reduce his electricity bill, as well as the possibility for the system operator to control the voltage profiles. Jabalameli et al. also studies the possibility to control the charging/discharging process of EVs in order to mitigate the voltage magnitude. By using the EVs as an active and reactive power source, they engage in energy markets without further battery degradation [11]. In [12] is proposed an optimal charging model to participate in the reserve market as a dynamic programming problem. In this way, optimal charging is achieved for all EVs, focusing on EVs' standpoint. A methodology is proposed in [13] to assess the value of the contract price and the charging and discharging processes regarding the EV aggregator. It also seeks the fair retail price for the EV owner in the day-ahead market. The authors model their problem as a multi-objective problem, considering goal programming. The main results indicate that it is possible to achieve balanced prices that satisfy the EV aggregator and the EV owner, highlighting that the EVs availability is a key factor in this process. In [14], the author presents a mathematical model for optimal charging and discharging for the participation in the spinning reserve market. The authors state that EV owners should respect the EV aggregator and not disconnect it earlier than planned. To solve the mixed-integer non-linear programming problem, a simulated annealing algorithm is used in order to increase aggregator's profit. The study concludes that low spinning reserves and high market prices reduce aggregator's profit. In [15], the EV unidirectional and bidirectional frequency services are compared. The results pointed to a slightly better performance of the bidirectional frequency service, however, this also implies higher costs and consequently higher risk.
The aforementioned work in [5] implemented an EV aggregator participation, modeling it as a two-stage stochastic programming model. The methodology studies both deterministic and stochastic approaches and the results show better performance when the stochastic methods are considered. In [16] is proposed a stochastic risk-averse optimal bidding strategy for EVs fleet and energy storage management. Risk is assessed through conditional value-at-risk method, so-called CVaR. This model aims to maximize aggregator's profit while accounting for battery degradation. Furthermore, in [17], the authors carry out a risk-averse analysis considering the CVaR method. A stochastic bi-level decision-making methodology is proposed aiming to maximize the expected profit, both for the EV aggregator and for the EV owner to participate in the day-ahead and reserve markets. The results prove that the more risk-averse the aggregator is, the lower the profits achieved. Considering the same risk assessment approach, the authors in [18] conduct a two-stage stochastic and dynamically updated multi-period mixed-integer linear program for the strategic participation of EVs in day-ahead, intra-day and real-time markets. This model also addresses the cost of battery degradation. According to the results, EVs tend to offer discharging bids in day-ahead, intra-day and real-time markets, in addition to offering down-regulation in real-time.
In [19], the flexibility of an EV aggregator is studied to avoid forecasting errors, enabling its participation in the reserve market. The model minimizes the total cost of the system, showing that the participation of the aggregator in the reserve market may benefit from reduced wholesale costs. The authors suggest that this reduced cost may help to attract new EV owners or increase the profit from the retail activity. Similarly, the authors in [4] proposed an optimal bidding strategy of an EV aggregator in the day-ahead and reserve market considering the perfect forecast information of day-ahead energy and regulation prices. It concluded that the EV charging costs can turn into profits when the EV aggregator gets involved in the reserve market. Goebel et al. investigated the design and performance of a system that would enable EV aggregator to participate in wholesale electricity markets including intraday and pay-as-bid reserve markets, which require exact reserve delivery and have long operating intervals [20]. The main goal is to maximize the reserve provision, ensuring sufficient energy for travel. It brings forward new ideas for achieving control of flexible loads with uncertain behavior subject to restrictive rules, concluding that the more flexible markets became, the more profit EV aggregators can get through EV charging.
Chen et al. developed a business model for EV aggregators, where the charging process is defined according to the market conditions [21]. The problem is formulated as a two-stage stochastic optimization and a Lagrangian relaxation is used in the charging demand constraint. The main findings demonstrate that more EVs owners can be attracted when they make a greater profit by adjusting the charging process. Other authors have focused their effort on developing models for the EV integration in systems with thermal plants [22] or renewable energy sources, such as wind turbines [23]. Likewise, the EVs integration in micro-grids also have been studied [24,25]. In [26,27], charging methodologies are proposed based on EV users' preferences.

Chance-Constrained Participation Model
This section explains the chance-constrained formulation applied to the participation of an EV aggregator in the reserve market. Nordpool market rules for frequency regulation are assumed, focusing on the frequency-controlled normal operation reserve (FCN-R) in East Denmark [28].
In addition, this section explains the relaxation techniques applied to the chance-constrained formulation, in order to make a tractable optimization problem.

Problem Description
The participation of an EV aggregator in the FCN-R must consider symmetric offering for upward and downward reserve. This work considers the uncertainty related to the EVs' availability to provide the service and the energy trip, which impacts the state-of-charge (SOC) of the EV. The goal is to maximize the aggregator's profit by participating in the FCN-R service at the day-ahead stage, accounting for energy not supplied penalties at the real-time stage. Thus, the problem is modeled as a two-stage stochastic chance-constrained problem, in which the trade-off between security of provision and revenue is considered. The objective function (1) combines the revenue of offering in the market at the day-ahead stage (rev DA ), accounting for the activation of the reserve and related imbalance costs The day-ahead stage is given by the capacity payment λ of upward and downward reserve offered by the aggregator, in each time step t, given by: Real-time stage remuneration considers revenues for reserve activation and penalties for non-provision of the reserve, both upward and downward, as can be seen in (3). In addition, RLXU and RLXD variables are included to represent the amount of power that was not provided in the scenario ω of the set Ω, but was offered in the day-ahead. Note that the discharge in the energy service is ignored because this study does not consider the participation in the energy service.
The objective function is subject to several constraints concerning technical limitations from the first and second stages, as presented in [5]. From this model, chance-constrained programming has been applied to two different equations, namely: where Pr represents the probability of the constraint (4), which is the balance between the system's needs and the availability of the aggregator to supply the contracted upward and downward reserve. The value of the system needs can be positive or negative ∆SI depending on the aggregator's willingness to provide the service at each time. The RLXU and RLXD variables represent the relaxation of the balancing equation, in cases where the power available in the EVs is insufficient to provide what the aggregator has promised at the day-ahead stage. The probability of ignoring this constraint is considered through the chance-constrained, allowing the aggregator to neglect some scenarios that significantly deviate from the aggregator commitment. The other equation in which chance-constrained is applied is the SOC balance Equation (5), given by: Pr(SOC (ev,t,ω) = SOC (ev,t−1,ω) − E Trip(ev,t,ω) + ∆tη Ch(ev) P E Ch(ev,t,ω) − P UP Ch(ev,t,ω) + P DW Ch(ev,t,ω) − ∆t η Dch(ev) (P E Dch(ev,t,ω) − P UP Dch(ev,t,ω) + P DW Dch(ev,t,ω) )) ≤ 1 − , ∀t ∈ T; ∀ω ∈ Ω; ∀ev ∈ EV where the SOC of each EV is determined. In this equation the previous SOC, the energy of each trip, and its participation in the reserve market are considered for each scenario ω and each time step t, accounting for the charging and discharging efficiency. The chance-constraint considers the possibility of neglecting some scenarios that make it difficult to meet the EVs' energy needs.

Bilinear Reformulation of the Chance-Constrained Problem
One way of solving a chance-constrained problem is to obtain the deterministic equivalent problem, which can be performed through the Big-M method or by bilinear reformulation [29]. The big-M method can thus reformulate constraint (4) as: where z represents the binary variable. When z = 0, constraint (4) assumes its original form. Otherwise, when z = 1, the constraint is ignored, meaning that can be violated. For a good performance of this relaxation method, the Big-M parameter (M) should be large enough [29]. Equation (7) constrains the number of scenarios in which z will take the value 1, based on the control parameter defined by the decision-maker. The use of the Big-M method is usually associated with issues such as determining the proper value of the M-parameter, without compromising the convergence of the problem. Usually, it is also associated with a decrease in computational efficiency, being linked to long times of computational effort. This method was also applied to constraint (5).

Case Study
In this section, the chance-constrained approach is applied to a realistic case study in East Denmark. The results of the Big-M and McCormick relaxation techniques are compared. First, the main characteristics of this test case are introduced, then the results analysis and discussion are presented.

Case Characterization
This study is based on a real case, comprising the fleet of EVs from the Frederiksberg Forsyning utility company within the scope of the PARKER project [31]. Ten EVs were considered, namely Nissan e-NV200 vans. 10 ENEL V2G charging units were used to charge and discharge the EVs. Each ENEL charging unit has a capacity of 10 kW, with a total sum of 100 kW. All the input data such as market prices, EVs, and charger characteristics can be accessed in [5]. The simulation of the proposed model was performed with MATLAB and GAMS.

Big-M vs. McCormick
Firstly, it is meant to analyze the expected profit under the Big-M method applied to two different chance-constraints, namely the TSO balance (4) and SOC (5) constraints. These constraints refer to technical aspects, so they can be assessed by the decision-maker on the risk they are willing to address. The results presented are for one-day simulation.
In general, one can see that the Big-M method applied to the balance between system needs and the availability of the aggregator to provide the contracted reserve ( Figure 1) has higher expected profits than when applied to the SOC equation (Figure 2). The difference between these two chance-constraints tends to increase in proportion to the predefined risk and the number of scenarios. For the same number of scenarios, considering the risk level equal to zero ( = 0%), the objective function for each chance-constraint simulation takes the same value. This means that the constraint cannot be violated in any case, so the problem is purely stochastic with no risk consideration. On the other hand, for risk levels higher than 0 ( > 0%), the objective function increases as long as the risk increases. That is, while the risk increases, scenarios are being neglected, and therefore the constraints can be violated. The worst cases scenarios are the first to be neglected, which ends with higher participation of the aggregator in the market and consequently higher expected revenue. Still, there are cases where the risk level ( ) is not large enough to remove scenarios, as happens under five scenarios where only with 20% of risk level ( = 20%) the first scenario is violated. Except in these cases, the chance-constraint applied to the TSO balance constraint always presents the best expected results for the aggregator.  McCormick method was also applied to the TSO balance constraint. In Table 1 are compared the results between this method and the Big-M method.
The results obtained are very similar; however, the McCormick method presents a better performance, since there are more cases in which it presents a higher expected profit. In cases where this happen, this increase is much higher than in cases where the Big-M method performs better. One can conclude that the results are similar for low levels of risk and a low number of scenarios. For higher risk levels, Big-M shows better results than McCormick. However, when the number of scenarios increases significantly, the performance of the methods is reversed. Thus, the McCormick method performs better than the Big-M method. This may indicate that the McCormick method can obtain better solutions than the Big-M for real dimension problems.

Computational Effort
An important issue for the aggregator is the timeline for submitting offers. Thus, the tools should provide solutions within this timeline. More precisely, the aggregator must submit an offer, even if it is not the optimal one within the time limit to ensure its participation in the market. To this end, computational runtime boundaries were set, in order to ensure that problem solutions are obtained within the market operation timeline. The Big-M method was limited to two hours, while the McCormick method was limited to four hours. The McCormick method requires more computational processing time, since it involves a linearization of bilinear terms and, therefore, a higher number of variables are modeled. Table 2 shows the Big-M method runtime and it is possible to conclude that only for 5 scenarios the algorithm can find the optimal solution, while for all other scenario numbers the time limit is reached, meaning that the solver was not able to find the optimal solution, but a feasible one. Similar behavior was obtained under the McCormick method. Still, note that the Big-M method is often faster than the McCormick method, and therefore, the most appropriate method for limited timeline rules of the market. In general, one can choose the McCormick approach, if there is a significant amount of scenarios and sufficient computational capabilities to handle the tight timeline of the market. Otherwise, the Big-M method should be chosen, providing a faster and more reliable solution.

Risk Assessment and Related Neglected Costs
The main objective of the aggregator is to maximize the expected profit. However, the objective function is related to costs, and a reduction of this value can lead to an increase in expected revenue. Thus, the aggregator needs to quantify the existing risk and then be able to minimize or manage it. The aggregator has to take the responsibility that certain decision comes with associated risk.
Therefore, the cost that the neglected scenarios represent for the aggregator (i.e., when the binary variable z (t,ω) is active) were determined for 50 scenarios and all risk levels in a one-week simulation, as presented in Figure 3. Note that these neglected scenarios are the ones that most decrease the value of the objective function. Therefore, it is crucial to provide tools to the cost trend of these neglected costs, in order to better support the EVs aggregator's strategic decision. In this way, the EV aggregator can quantify the benefits/losses that are likely to occur for a certain risk level. Thus, for the different risk levels, the costs associated with the neglected scenarios were calculated and compared with the expected revenue.
Analyzing Figure 3, one can see that the costs of neglecting scenarios increase along with the expected revenue. As expected, revenue increases with predefined risk, as predicted, these costs are increasing steadily. In the case of risk-averse behavior (i.e., when = 0%), the methodology considers all the scenarios representing the uncertainty set. However, when the risk starts to be higher ( = 15%), one can see that the curve in Figure 3 becomes even steeper, highlighting the possibility of extra costs due to neglected scenarios. The outcome gives to the EV aggregator wherever to adopt a risk-averse or risk-neutral behavior. The more scenarios the aggregator neglects, the higher the expected revenue will be. However, if these neglected scenarios occur, they entail higher costs. Therefore, the decision-maker must consider this trade-off between expected revenue and the potential costs of neglecting the worst-case scenarios.

Conclusions
EVs will change the current paradigm of the electricity markets, as they will play a key role, especially integrating more renewable energy sources. The ideal characteristics of EVs for supporting frequency regulation combined with EV aggregator's ability to participate in the ancillary services market are the key points for this paradigm shift. In this scope, this work modeled the strategic offering of an EV aggregator, aiming to maximize the aggregator's profit through a two-stage stochastic model with chance-constrained programming. The McCormick and Big-M relaxation methods have been implemented, and their performance was compared.
The results show that the Big-M and McCormick approaches behave similarly, with small but important deviations. More precisely, for a low number of scenarios, the results are similar in both methods with differences in expected profit below 0.001%. However, when the number of scenarios increases, the differences become more pronounced, allowing to conclude that for a high number of scenarios, the McCormick relaxation method presents a better solution for the aggregator, regarding the expected profit. Still, this method presents a larger computational effort, being its main drawback when compared to the Big-M approach. In addition, the aggregator's response in the hour-ahead to out-of-sample validation suggests that neglecting the worst-cases can be very expensive. That is, the worst-cases (when occurring) are usually the cases that may result into high losses to the aggregator's revenue. In fact, as the risk increases, the expected profit tends to decrease, since a significant percentage of scenarios that may occur are neglected.
Future work will focus on two distinct directions to improve the proposed model. On the one hand, the computational efficiency of the model will be pursued, using decomposition techniques to handle the complexity of the two-stage model. On the other hand, the upward and downward reserve is not symmetrical in other electricity markets to which the model may be extended. Funding: This work is supported by the ERDF-European Regional Development Fund through the Operational Programme for Competitiveness and Internationalisation-COMPETE 2020 Programme, and by National Funds through the Portuguese funding agency, FCT-Fundação para a Ciência e a Tecnologia, within project ESGRIDS-Desenvolvimento Sustentável da Rede Elétrica Inteligente/SAICTPAC/0004/2015-POCI-01-0145-FEDER-016434.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: