Robust Allocation of Reserve Policies for a Multiple-Cell Based Power System

Junjie Hu 1 ID , Tian Lan 2,*, Kai Heussen 3 ID , Mattia Marinelli 3 ID , Alexander Prostejovsky 3 ID and Xianzhang Lei 2 1 State Key Laboratory of Alternate Electrical Power Systems with Renewable Energy Sources, North China Electric Power University, Beijing 102206, China; junjiehu@ncepu.edu.cn 2 Global Energy Interconnection Research Institute Europe GmbH, 10117 Berlin, Germany; xianzhang-lei@sgcc.com.cn 3 Center for Electrical Power and Energy, DK2800 Lyngby, Denmark; kh@elektro.dtu.dk (K.H.); matm@elektro.dtu.dk (M.M.); alepros@elektro.dtu.dk (A.P.) * Correspondence: lantian@geiri.sgcc.com.cn; Tel.: +49-176-9570-4134


Introduction
With increasing concerns regarding global warming and environment pollution, there has been a worldwide movement in the promotion of renewable technologies for electricity generation and for reducing the greenhouse-gas emissions.Many distributed generation units, including wind turbines [1], photovoltaic generators [2,3], fuel cells and fuel cell/gas/steam powered combined heat and power systems [4], are being used and connected to the power systems.However, a large penetration of intermittent and variable generation introduces operational challenges to the power system.To accommodate the variability of the renewables, it is important to harness the flexibility of the newly introduced units, such as batteries and electric vehicles, especially in the distribution network level [5][6][7].Enabling this flexibility comes at the cost of an increasingly complex control system, characterized by many state and decision variables [8].
The problems encountered in the power systems have received much attention, and various efforts have been made to address the problems.These range from developing proper control schemes for individual component operation such as hierarchical aggregation method [9,10] to radical rethinking of system operations [11][12][13][14].Traditionally, the system is kept secure by distinguishing the role between transmission system operators (TSO) and distribution system operators (DSO).For example, the TSO centrally controls a few big power plants through a supervisory control and data acquisition system.The DSO centrally manages the status of key devices, such as breakers, reference setting points of on/off load tap changers, capacity banks, etc.However, it is impossible for TSO to control the large number of distributed energy resources found today, as the grid control systems are centralized by design, and do not yet actively integrate distributed energy resources into the operation on a meaningful scale.
To address the aforementioned problem, the European FP7 project ELECTRA IRP proposes and develops a Web-of-Cells (WoC) architecture for operating the future power system [11][12][13][14].In this approach, the power systems operation is divided into connected cells, each responsible for their own balancing and voltage control, thus establishing a robust, decentralized horizontal decomposition as opposed to the conventional centralized and vertical system operation.The WoC concept reformulates the control architecture of electric power systems to accommodate the challenges of fully distributed generation, reduced inertia, storage integration and flexible demand.Cells, which are defined as non-overlapping topological subsets of a power system, are associated with a scale-independent operational responsibility to contribute to system operation and stability.The operating state, including power exchanges and reserve parameters, can then be continuously optimized by coordination across cells.
In this paper, we study how joint energy and reserve scheduling, which are provided by flexible load units such as storage units and electric vehicles (EVs) that have limited power, energy and specific use patterns, could be operated more efficiently in the WoC architecture based power system.To do so, a robust power system reserve allocation approach [15,16] combining a predictive dispatch with optimal control policies in the form of linear decision rules (LDR) [17] is proposed.LDR concept is used in operations research field, where current states, past data or future predictions are combined linearly to make an operational decision.In [15], LDR-based reserve policies consisting of a nominal power schedule with a series of planned linear modification is proposed to accommodate fluctuating renewable energy resources.These policies are time-coupled, which exploits the temporal correlation of these prediction errors.The study showed that LDR-based reserve policies can reduce reserve operation cost compared to existing standard reserve operation method.In [16], the authors proposed an adjustable robust optimization approach to account for the uncertainty of renewable energy sources in optimal power flow.The optimized solution has two part: (1) the base-point generation is calculated to serve the forecast load which is not balanced by RES; and (2) the generation control using participation factors ensures a feasible solution for all realizations of RES output within a prescribed uncertainty set.However, both papers only applied the LDR method for one power system.Compared to stochastic programming, robust optimization method only requires knowledge of the range of variation of the uncertain parameters as opposed to an accurate specification of the uncertain parameter in stochastic programming.Therefore, robust optimization has been gaining popularity for decision making under uncertainty.In [18], the authors applied LDR-based reserve policies to the Web-of-Cells [19].Firstly, the study shows that the method works fine for a single cell operation, i.e., the power and energy curve of batteries are within the capacity for any realized RES output.Then, three ad hoc cooperation strategies of web-of-cells are studied and compared using the LDR method.The three cooperation strategies include: (a) no cooperation between cells; (b) full cooperation between all cells; and (c) in between these two extreme cases.The study shows that Strategy (a) has a clear disadvantage over other two cooperation strategies.
Building forth on the previously developed work [18], this paper has two advancements: (1) It develops a model that adapts the application of the LDR-based reserve policies to multiple-cell based power systems rather than single-cell based power systems.Based on the proposed model, cross-cell reserve allocation, indicating the cooperation scheme among cells, can be determined.The results show that the involvement of the cross-cell reserve depends on the availability of reserve resources in the local cell.(2) To facilitate the real time operation in the real system, the effects of different time intervals on the LDR control policies are investigated in this paper.The investigation of the effects is made considering energy curves, power curves, and cost index of each discussed case.The remainder of the paper is organized as follows.In Section 2, the optimization formulation for one and multiple cell based power system is proposed, given the basic power system model.Comprehensive case studies are performed in Section 3. Conclusions are drawn in Section 4.

Methodology
The methodology used for solving the robust optimal reserve allocation problem is introduced in this section.The methodology is based on LDR, and can be applied to the operation of power systems, which could be single-cell based or multiple-cell based.A single-cell based power system can be considered as an isolated system, which could be an autonomous microgrid or a regional network.A multiple-cell based power system consists of two or more cells linked via interconnectors between each other.In the following subsections, the LDR based methodology is firstly proposed.Then, the optimization problem regarding single-cell and multiple-cell based power systems are, respectively, formulated using proposed methodology.

Basic Power System Model
A power system with various participants connected to a power grid is considered.The participants of a power system could be production units, loads, or storage units, which either inject power into or extract power from a node in the network.They are categorized into two types in terms of power injection: participants with inelastic power injection and those with elastic power injection.The inelastic power flows indicate the power flows of the participants cannot be regulated by control signals.These participants could be certain loads and renewable generators, such as wind turbines and PV panels.Regarding a participant i of this kind, the power injection into or extraction from a network y i can be modeled as: where r i ∈ R T indicates a nominal prediction of the power injection or extraction; T is the divided discrete time steps of a planning time horizon, over which electricity can be traded on intra-day markets [20]; δ ∈ R N δ T is the random forecast error vector; G i is a linear function used for mapping the uncertainty δ to power flows; and N δ is the number of elements in the uncertainty vector at a given time.If the power flows of the participants can be perfectly predicted, the prediction error δ will be zero.The elastic power injection indicates that the power flows of the participants can be influenced by control signals.In other words, the flexibility of these participants can be exploited and used to mitigate the disturbance in the network.This can be achieved using the control signals determined by the results of proper optimization.According to [15], the elastic power injections can be modeled as C j x j , where x j ∈ R n j T is a vector of future states of participant j, n j is the state dimension, and C j ∈ R T×n j T is the stacked output matrix used for selecting the needed element of the state vector x j .The future state vector x j is given as: where x j 0 is the current state of the participant j, u j ∈ R T is the control input to the elastic power participant j for balancing the system, and A j and B j are the corresponding stacked state transition matrices.

Constraints for Production Units
Different participants need to be limited by corresponding constraints.Regarding a production unit at period t, which could be an inelastic or elastic power unit, the upper and lower bounds of the power are imposed by the following constraints:

Constraints for Storage Units
Regarding a storage unit j, which could be a battery, an electric vehicle, etc., whose power injection is elastic, the storage unit's power and energy constraints need to be imposed: As all participants are connected to a network, the sum of all the inelastic power injection y i and elastic power injection C j x j has to be zero at all times t = 1, ..., T. This can be achieved using an equality constraint: where N inelas is the number of the inelastic power participants in a network, and N elas is the number of the elastic power participants.

Linear Decision Rule Based Robust Optimization of Reserve Allocation
Inserting Equations ( 1) and ( 2) into Equation ( 7), the following equation can be obtained: As mentioned, to keep power balance in a network, the power injection or extraction of the inelastic power units, ∑ N inelas i=1 (r i + G i δ), must be balanced by the power contribution of the elastic power units, ∑ N elas j=1 C j (A j x j 0 + B j u j ), at any point in time.Regarding elastic power units, u j is the control input that can regulate the power flow of the elastic power participant j.According to the LDR method, control input signal u j can be expressed to policies of the affine form: where u j is described by a nominal schedule e j ∈ R T plus a linear variation D j ∈ R T×T , the nominal schedule e is mainly used for balancing the nominal prediction of the inelastic power flows r, and D is the dynamic response to the prediction errors δ.The matrix D defines a map from the uncertainty into the realization of the power contribution of the elastic power units.In order for the use of future disturbances to be causal, D j takes the lower-triangular form.

One Cell-Based Reserve Allocation Model
In a one-cell based power system, the reserve allocation optimization problem is formulated as a cost function as follows: min E{ ∑ j∈φ (α j P j (δ) T P j (δ) + β j P j (δ) + γ j Ī)} (10) where j ∈ φ indicates the elastic power participant; P i is the power contribution of the participant; α j , β j , and γ j are stacked vectors of quadratic, linear, and constant coefficients of the cost function of the power contribution, respectively; and Ī is a vector of all of them.This optimization problem is subject to system constraints, power constraints, and energy constraints, as mentioned in the previous section.The power contribution of the elastic power participant can be given as: with the help of Equation ( 2), P j (δ) can be further written as: According to policies of the affine form, as given in Equation ( 9), the entire optimization problem in Equation (10) becomes tractable due to the restricted variety of candidate u j .
Substituting Equation ( 12) into the objective function, i.e.Equation ( 10), the following form can be obtained: where u j can be further expressed as policies of the affine form, as shown in Equation ( 9); together with an assumption that E[δ] = 0, the optimization problem can be written as: with b j = C j B j (16) where X, Y is the trace of product X Y; regarding the reformulation of the equality and inequality constraints, the approach is similar to the one presented in [15,18].

Multiple Cells-Based Reserve Allocation Model
Compared to the single cell-based power system, the robust optimization of reserve allocation using LDR in multiple cells-based is more complicated.An example of a three-cell power system is shown in Figure 1.Each cell has its respective generations, loads, and storage units.Three cell are connected with tie lines, as depicted in the Figure 1.Regarding the resources with flexibilities, such as electric vehicles and batteries, it is assumed that a portfolio of all flexible energy-constrained resources in one cell can be represented by an aggregator as a single unit.In the WoC concept, local imbalances within a cell are supposed to be solved locally.Cells are not necessarily supposed to be self-sufficient, but are required to have the balancing capability provided by elastic power units for mitigating deviations from a given schedule.The basic concept of reserve allocation in multi-cell-based power systems is that the local reserve provided by local elastic power units is prioritized for handling of local imbalances, which is reflected in the cost function of each available power unit.As long as local reserves can handle the local imbalances, no reserve is needed from other cells.Cross-cell reserve allocation happens when the local resource cannot handle the local imbalance.
Cells are managed by so-called Cell System Operators (CSOs), whose roles incorporate the tasks of traditional Distribution and Transmission System Operators (DSOs and TSOs, respectively) [21].A dedicated Cell Controller (CC) performs monitoring and automated balancing tasks.Each cell is assigned one CC, which is managed by one CSO using bi-directional communication, as shown in Figure 2. A CSO, on the other hand, may be responsible for more than one CC to allow for flexible adaption to present-day grid partitioning and management schemes.
Among other tasks, CSOs procure reserves and send the schedules to the CCs, which automatically carry out balancing tasks around the given setpoints and trajectories.As indicated in Figure 2, information and measurements from each cell controller, including the prediction of the generation of the renewables, availability of the elastic power units as well as their power and energy constraints, information of loads, etc., are transferred to the cell operator, based on which the cell operator can utilize the proposed optimization to distribute the control signals, i.e., the D and e presented in Equation ( 9), for each cell controller to allocate the reserve in each cell.The reserve optimization problem for multiple-cell based power system can be generally formulated as: where l and m indicate the cell indices; j is the participant involved in the reserve allocation; φ cell and φ l are the sets of cell and participants in cell l, respectively; P l,l is the allocated resource in cell l that is reserved for the imbalance in cell l; and P l,m is resource that is reserved from cell l to cell m, also called the cross-cell reserve service.Finally, v j l,m is a binary decision variable used for determining the involvement of the cross-cell reserve allocation: v j l,m = 0 P j l,l (δ max ) >= P j max 1 P j l,l (δ max ) < P j max (18) Similar to Equation ( 12), P l,l and P l,m can be expressed as: According to the LDR method, u j l,l and u j l,m can be expressed as: where participant j's power schedule in local cell u j l,l is determined by a nominal schedule e j l,l , a linear variation D j l,l and predication errors δ.Similarly, expression of u j l,m can be obtained.Together with the assumption that E[δ] = 0, the optimization problem can be written as: Equation ( 23) establishes an overall optimization for a multiple-cell based power system.To solve the one-cell and three-cell optimization problems, YALMIP, a toolbox in MATLAB, is used [22,23].

Case Studies
In this section, several case studies are carried out.The setup is based on a configuration of SYSLAB, which is a laboratory testing facility for Smart Grid concepts located at the Risø campus of the Technical University of Denmark.

SYSLAB System
Figure 3 shows the SYSLAB facility [24,25], which is a 400 V three-phase grid designed for studying advanced grid control and communication concepts.The facility has 16 busbars and 116 automated coupling points.A wide range of distributed energy resources, such as wind turbines, solar panels, electric vehicles, etc., can be remotely operated via a distributed monitoring and control platform.Because of the very flexible connection interface of the busbars, various topologies of the system can be configured and operated.Division of the cells in the system can also be flexible and different from the one shown in Figure 3.

One Cell-Based Simulation
Figure 4 shows one-cell based isolated system, which comprises a battery, a solar PV, an EV, and a mobile load.The mobile load is used for imitating a typical residential electric load profile.The imbalance in the system is mainly caused by the difference between the fluctuated PV generation and the electric load demand.In this system, the inelastic power unit is the PV panel, whose power generation cannot be regulated by control signals.The EV and the battery act as elastic power units, which can provide reserve service due to their controllability of the power flow.The simulation is carried out for 30 min from 11:30 to 12:00.The time interval is chosen to be 3 min, and the horizon length therefore is 10.From the simulation, the policies for the reserve allocation, i.e., the matrix D introduced in Section 2, of the battery and the EV are presented in Figure 5.As described in Section 2.2, D is a lower-triangular matrix that defines a map from the uncertainty to the realization of power contribution of a reserve participant.The matrix is visualized in Figure 5 for both the battery and the EV.According to Equation ( 9), the dynamic response to the prediction errors at time instant t is determined by [D j ] t,0 , [D j ] t,1 , ..., [D j ] t,t , which are the elements in row t in the figure.The battery is allocated with more reserve than the reserve allocation contributed by the EV.This is because the battery has a wider power range and larger capacity.Furthermore, the reserve cost of the policy based scheme over the simulation period is compared to that of a flexible-rate scheme.Flexible rate scheme is commonly used for reserve optimization [15,26].In this paper, the results given by flexible rate scheme act as a reference case.The differences between the two schemes are summarized as follows:

One-cell based system
1.
Flexible-rate reserves [15]: D j is a diagonal matrix.This indicates the best possible response to uncertainty without time coupling.The previous uncertainty therefore has no impact on the present operation because the causality of the uncertainty is omitted.The optimization is over the elements of e j and the diagonal parts of D j .2.
Policy-based reserves: Compared to the above scheme, this scheme considers the time coupling by taking D j as the lower-triangular form.It allows full exploitation of the information that will be available at each time step when the reserve is deployed.The comparison results regarding the two schemes are given in Table 1.It is shown that the cost index given by the policy-based reserve is smaller than that given by the flexible-rate reserve.This is due to the full exploitation of the covariance matrix using the lower-triangular form of D. From the viewpoint of optimization, the feasibility region of D is larger for policy-based reserves, which gives the optimization more opportunity to find better results.

Three Cells-Based Simulation
A three-cell based isolated system is presented in Figure 6.Cell 1 has a vanadium battery described in Table 2 and an Aircon wind turbine that has a maximum power generation of 9.8 kW.Cell 2 comprises an EV and a solar panel.Cell 3 only has a mobile load in it.The basic idea of the co-operation is that the local imbalance has priority to be handled by using local reserve, while the cross-cell reserve is allocated only when there is a need.To demonstrate the co-operation of multiple cells, two scenarios are developed by setting the maximum power availability of EV.A three-hour simulation is carried out.The horizon length is set to 12, and the time interval is 15 min.In this scenario, P min and P max of the EV are set to −15 kW and 15 kW, respectively.With this power availability, the imbalance in Cell 2 caused by the fluctuated power generation of the PV panel can be handled locally.Similarly, Cell 1 can also handle its local imbalance due to the large power capacity of the vanadium battery.Therefore, it is expected that there will be no cross-cell reserve allocation between Cell 1 and Cell 2. Cell 3, however, has only an inelastic power unit, which is the mobile load that represents the residential electric load demand.As it is assumed that the load profile can be precisely predicted, there is no reserve allocation between Cell 3 and other cells, but only nominal schedule e 13 and e 23 exist for supporting the residential load.

Scenario 2
To create the need for cross-cell reserve allocation, in Scenario 2, the P min and P max of the EV in Cell 2 are set to −5 kW and 5 kW, respectively.In this case, the local reserve in Cell 2 cannot handle the fluctuated power generation of the PV.Therefore, reserve allocation from Cell 1 to Cell 2 is necessary.In other words, the elastic resource in Cell 1 needs to be utilized to support the compensation of the predication errors in Cell 2. For that purpose, the vanadium battery in Cell 1 is divided virtually into two parts.

Impact of Time Interval
According to Equations ( 14) and ( 23), the optimization formulations depend on the covariances of the historic imbalance signals E[δδ T ].The covariances of the historic imbalance further depend on the time interval that is used in the simulation and even real time operation.Three cases are carried out based on the one-cell system shown in Figure 4.The configurations of the three cases are given in Table 1.The simulation duration is set to 30 min for all three cases.A simulation with time interval of 3 min is introduced in Section 3.2, while the policy results of a simulation with time intervals of 2 min and 5 min are given in Table 1.Then, a comparison is made based on the time intervals that are less and more than 3 min.Combining the results presented in Section 3.2, the cost indices of the three cases are presented in Table 1.It is noted that the cost index decreases with the increased time interval.This is because the calculation of the covariance of historic imbalance is determined by the time interval.Furthermore, the effects of the time interval on power and energy curves are depicted in Figure 9 based on the renewable inputs and load profile in a specific day.Only the battery curves are presented as an example.It is shown that the power and energy curves contain more information when the time interval is smaller.A longer time interval will smooth the prediction errors of the historic data.This will finally reduce the needed reserve and cost.Longer time intervals can reduce the computation time.However, due to the average operation of the prediction errors, time intervals that are too long might result in inaccurate optimization results, i.e., insufficient reserves.On the other hand, shorter time intervals can make the results more reliable, but the computation time will increase accordingly.In real time operation, it is important to choose a proper time interval to make a trade off.
Table 1 presents the comparison of the cost index given by two schemes.It is shown that the cost index of flexible-rate reserve is higher than that of policy-based reserve for all three cases.The cost saving is important because the presented results are obtained from the 30 min simulation.The cost saving of the policy-based reserve will further increase in accordance with the increasing scale and the operation duration of the system.

Discussion and Conclusions
Linear decision rule-based control policy for coordinating reserve allocation to the ELECTRA Web-of-Cells system architecture has been developed, implemented, demonstrated, and applied to a three-cell based power system.It has been demonstrated that control policies based on linear decision rules are well suited for integration of distributed energy resources which have flexibility in balancing control, and the robust allocation method is applicable to realistic imbalance signals.Furthermore, it is concluded that a proper time interval selection is important in the linear decision rules-based application.We note that this method relies on several inputs such as: (a) the covariance information of historical data; and (b) the predicted base power production and the prediction error bound of the inelastic power injection, which needs to be improved in the near future.In addition, when adapting the method into real cases, the cost function of elastic power units that represents the balancing service provision should be carefully designed.

Figure 1 .
Figure 1.An example of a three-cell small scale power system.

3 Figure 3 .
Figure 3. SYSLAB layouts for the two test cases.Devices and lines used in the one-cell case are marked in red.Additional components for the three-cell case are indicated in orange.

Figure 5 .
Figure 5. Policy of reserve allocation for one-cell system with time interval of 3 min.

Figure 7
shows the simulation results of Scenario 1.Because of the sufficient flexibility in each cell, the cross-cell reserve allocation does not exist, which leads to D 12 = 0 and D 21 = 0, as depicted in the figure.The imbalances inside Cell 1 and Cell 2 are all handled in their own cell; it can be seen that D 11 = I and D 22 = I, indicating that the predication errors in the two cells are fully compensated using the vanadium battery and the EV in their respective cells.

Figure 7 .
Figure 7. Policy of reserve allocation for three-cell power system in Scenario 1.

Figure 8 .
Figure 8. Policy of reserve allocation for three-cell power system in Scenario 2.

Figure 9 .
Figure 9. Energy and power curves of battery.

Table 1 .
Comparison of cost index regarding different time intervals.

Table 2 .
Properties of devices used in the one-cell system.