Distributed Capacity Allocation of Shared Energy Storage Using Online Convex Optimization

This paper studies capacity allocation of an energy storage (ES) device which is shared by multiple homes in smart grid. Given a time-of-use (TOU) tariff, homes use the ES to shift loads from peak periods to off-peak periods, reducing electricity bills. In the proposed ES sharing model, the ES capacity has to be allocated to homes before the homes’ load data is completely known. To this end, an online ES capacity allocation algorithm is developed based on the online convex optimization framework. Under the online algorithm, the complex allocation problem can be solved round by round: at each round, the algorithm observes current system states and predicts a decision for the next round. The proposed algorithm is able to minimize homes’ costs by learning from home load data in a serial fashion. It is proven that the online algorithm can ensure zero average regret and long-term budget balance of homes. Further, a distributed implementation of the online algorithm is proposed based on alternating direction method of multipliers framework. In the distributed implementation, the one-round system problem is decomposed into multiple subproblems that can be solved by homes locally, so that an individual home does not need to send its private load data to any other. In simulation, actual home load data and a TOU tariff of the United States are used. Results show that the proposed online approach leads to the lowest home costs, compared to other benchmark approaches.


Introduction
In the vision of smart grid, distributed generation units and renewable energy resources will be increasingly deployed on the demand side [1,2].End users will become both energy producers and consumers.Together with high uncertainty of user load and renewable generation, demand side energy management will be greatly challenging.Under this circumstance, the role of energy storage (ES) becomes particularly important as ES is able to provide valuable services, such as load shifting [3,4], demand/supply balancing [5,6], and renewable output smoothing [7,8].ES can significantly enhance flexibility and controllability of demand side energy management.For example, residential energy users can use ES to store rooftop solar power in daylight and release the energy when it is needed.ES enables the controllability of load, reducing home electricity bills.However, the one-time investment expense of ES is relatively high so that the initial cost can be only recouped after a long time.This discourages end users to buy ES especially when their future loads are uncertain.
To deal with this issue, the concept of ES sharing [9,10] has been proposed.In ES sharing, a user does not have to buy a complete ES device but accesses the shared ES based on its need.
Numbers of studies have investigated the benefits brought by ES sharing.It is shown in [10,11] that ES sharing allows users to sufficiently exploit idle ES resources, reducing users' costs and improving ES utilization.A common scenario of ES sharing is that multiple users share an ES device.Aiming at such a scenario, the existing research can be classified into two types: energy management [11][12][13] and capacity allocation [14][15][16].The first type mainly studies energy flow management among the shared ES and users.Users generally do not occupy a certain part of the ES's capacity.Users' charging and discharging powers would be optimized as a whole from a system perspective.The second type concentrates on allocating the ES's capacity to users, which is also the focus of this paper.The general goal would be to select a solution of cost-saving capacity allocation.Individual users can determine detailed charging/discharging operation by themselves after capacity allocation.
There are not many studies working on capacity allocation of a shared ES.In [14], a two-stage model is formulated to describe the interaction between a shared ES and users, in which the ES adjusts the capacity price to maximize its profit, and users adjust the amounts of purchased capacity to minimize their electricity bills.In [15], it is considered that users initially own a portion of a shared ES, and a distributed negotiation scheme is designed to allow users to trade their capacities.In [16], multiple users jointly invest an ES device; the resultant ES cost is shared among the users; the cost sharing problem is solved using cooperative game theory.However, the capacity allocation approaches in [14][15][16] are offline approaches: [14] and [15] rely on predictive user load profiles, and [16] requires attaining cumulative distribution functions of peak loads from historical user load data.In fact, users' loads would be highly uncertain and difficult to predict accurately.Offline approaches lack learning ability.If predictive data is inaccurate and historical data is too old, the resulting offline solutions would be unreliable in practice.
To address the above issue, this paper studies ES capacity allocation using online convex optimization [17,18].The proposed online capacity allocation algorithm can minimize users' costs by learning from user load data in a serial manner.Specifically, we consider an ES device shared by multiple homes.Homes (also called users) consume energy from the grid which adopts time-of-use (TOU) pricing.Users exploit the shared ES to reduce their electricity bills by charging the ES when grid prices are low and discharging the ES when grid prices are high.Subsequently, a user has to decide how much ES capacity to purchase without knowing its load profile of the next day.To this end, we develop an online algorithm based on online convex optimization framework to allocate ES capacity to users.We also introduce two metrics to evaluate the algorithm's performances, which are regret and constraint violation.Next, a distributed implementation of the online algorithm is developed using alternating direction method of multipliers (ADMM) framework [19].In the distributed implementation, users perform local computation and communicate with neighbors in an arbitrary connected peer-to-peer (P2P) network.In simulation, we use the United States' home load data from Dataport database [20].The results show that the proposed online approach has the lowest system costs, compared with other benchmark approaches.The contributions of this paper are as follows:

•
The ES capacity allocation problem is formulated to be solvable in online convex optimization framework, in which capacity has to be allocated before user load profiles are disclosed.

•
An online capacity allocation algorithm is developed, which does not require day-ahead load forecasting or estimating probability distribution functions of loads.

•
A distributed implementation of the online algorithm is proposed, in which users do not need to send private load data to any other.
The rest of this paper is organized as follows: The ES sharing system model and problem formulation are presented in Section 2. The online algorithm for ES capacity allocation is described in Section 3. The distributed implementation of the online algorithm is provided in Section 4.
The simulation results are demonstrated and discussed in Section 5. Finally, the paper is concluded in Section 6. Time-of-Use Tariff : It is considered that users pay for electricity from the grid according to a TOU tariff [10,16].Figure 2 shows a TOU tariff for residential consumers, which is provided by PG&E [21], an energy company in California.The TOU tariff divides a day into three periods: off-peak period (21:00-10:00), peak period 1 (10:00-13:00 and 19:00-21:00), and peak period 2 (13:00-19:00).The corresponding electricity prices are denoted by P off , P pk 1 , and P pk 2 , respectively.It is considered that the time periods and corresponding prices are unchanged every day.Given a TOU tariff, users can charge the ES in off-peak periods and discharge the ES to serve loads in peak periods.This prevents from buying grid energy at high prices, reducing electricity bills.For a TOU tariff, each peak period is indexed by j ∈ J = {1, . . ., J}.In the case of Figure 2, we have J = 2. Load Shifting Cost: To lower electricity bills, users desire to buy ES capacity to shift loads to off-peak periods.Users can change their capacity purchase decisions every round, and each round contains 24 h, as shown in Figure 2. To allow users to have sufficient time to charge the ES, we set that a round starts at the beginning of an off-peak period.Each round is indexed by t ∈ T = {1, . . ., T}.At the beginning of round t, user i makes a decision c i,t = [c i,1,t , . . ., c i,J,t ] where c i,j,t denotes the amount of ES capacity (kWh) that user i buys for shifting the load of peak period j.Let D i,j,t stand for the original load (kWh) in peak period j, with D min i,j ≤ D i,j,t ≤ D max i,j .The cost of user i during peak period j at round t is expressed by

System Model and Problem Statement
It is shown that if c i,j,t ≥ D i,j,t , user i can use discharging energy to fully meet D i,j,t and then has zero cost in peak period j.If c i,j,t < D i,j,t , user i still needs to buy D i,j,t − c i,j,t units of energy from the grid at price P pk j .Let d i,j,t denote the amount of energy that is shifted from peak period j to the off-peak period.In other words, to serve the load in peak period j, user i buys d i,j,t units of energy from the grid in the off-peak period and stores the energy in the ES.Due to energy loss of ES charging and discharging, d i,j,t is given by Thus, the cost of user i in the off-peak period at round t is described by where B i,t denote the original load in the off-peak period and R i,t is the energy left in the ES at the beginning of round t.As these two values do not relate to decision c i,t , they are excluded in the following formulation.Let P es denote the price of ES capacity.The cost of user i for shifting the load of peak period j at round t is given by which includes three parts: the ES capacity purchase cost, the cost in peak period j, and the corresponding cost in the off-peak period.l i,j,t (•) is a piecewise-linear function, and it can also be written as It is shown that We consider that this inequality holds for the proposed ES sharing system.Intuitively, if ES price P es is higher than arbitrage price P pk j − P off /(η c η d ), there is no incentive to encourage users to use the ES.To study ES sharing problems, this paper considers that the ES price is relatively low, i.e., the convexity of ( 5) holds.This facilitates the later cost minimization problem solving, and this is also the basic setting of other related works [10,16].
Satisfaction Degree: As shown in ( 5), the slopes of l i,j,t (•) are the same for all users, which means that given one unit of ES capacity, the amounts of cost reduction for all users may be the same.In fact, users who have high D i,j,t prefer the ES more than those with low D i,j,t .To characterize the preference of user i for using ES, we introduce the following function to describe the satisfaction of a user getting c i,j,t units of ES capacity: where ω is a parameter adjusting the importance of satisfaction.The reason for using (6) is that it can describe two features of the satisfaction of a user.First, the growth of satisfaction normally gets slow as the obtained ES capacity goes up, i.e., the marginal satisfaction decreases.Second, c i,j,t = D i,j,t is the best capacity allocation for each user, so we set that all users have the same satisfaction if c i,j,t = D i,j,t , i.e., u 1,j,t (D 1,j,t ) = . . .= u V,j,t (D V,j,t ). Figure 3 illustrates the two features of (6).Logarithmic satisfaction functions are widely used in related literature [22,23].We can use other forms of satisfaction functions (e.g., quadratic functions [8,24]) as long as they have the two aforementioned properties.Capacity Constraint: Because of the loss of charging/discharging, the available capacity of the shared ES is Long-Term Budget Balance: Let U i denote the average budget that user i plans to spend on using the ES per round.Budget balance means that the actual expense is no more than the budget.Users do not need to achieve budget balance at each round, but they desire to achieve it in the long term.The long-term budget balance constraint for user i is given by

System Cost Minimization
The total cost of user i for peak period j at round t is load shifting cost (5) minus satisfaction degree (6), denoted as f i,j,t (c i,j,t ) = l i,j,t (c i,j,t ) − u i,j,t (c i,j,t ).Define the total cost of user i at round t as f i,t (c i,t ) = ∑ j∈J f i,j,t (c i,j,t ).The ES sharing system minimizes the total costs of all users i ∈ V over T rounds.The system cost minimization problem is given by P1: min where c t = [c 1,t , . . ., c V,t ] is the capacity allocation decision for round t.All vectors defined in this paper are column vectors.( 7) and ( 8) are written as (9b) and (9d), respectively, using c i,t = [c i,1,t , . . ., c i,J,t ] and 1 = [1, . . ., 1] ∈ R J .The challenge of solving P1 is twofold: 1) Peak load data {D i,j,t } i∈V,j∈J is fully disclosed at the end of round t.The load data relates to the user cost functions.In other words, the optimal allocation sequence {c * 1 , . . ., c * T } is attainable only at the end of the Tth round, at which { f i,t (•)} i∈V,t∈T is completely known.Thus, only offline approaches can optimally solve P1, using historical or predictive load data.2) The objective function of P1 is the sum of multiple piecewise functions.If the number of rounds T and the number of users V are very large, the number of pieces of P1 objective function is huge.In this case, it is impractical to get the exact expression of P1 objective function even though we assume that { f i,t (•)} i∈V,t∈T are known in advance.

Online Convex Optimization
Considering the above challenges, we solve P1 using online convex optimization, which is specified to multi-round online learning problems with convex (or concave) objective functions [17,18].Specifically, we define f t (c) = ∑ i∈V f i,t (c i ) and C = {c ∈ R JV : ∑ i∈V 1 c i ≤ C es , c i,j ≥ 0, ∀i ∈ V, j ∈ J }.At each round of online convex optimization, a learner selects decision c t from convex set C at the beginning of round t.At the end of round t, peak load data {D i,j,t } i∈V,j∈J is revealed, and the learner observes loss f t (c t ).By employing online convex optimization, day-ahead load forecasting for each individual user is not required.In addition, P1 is decomposed into a series of one-round problems, reducing computational complexity.

Performance Metrics
We desire an online learning algorithm that produces a capacity allocation decision based on load data from previous rounds.To evaluate performance of the algorithm, two metrics are introduced: regret and constraint violation [18].The regret after T rounds is defined as where c * is a fixed optimal solution to the following problem: Regret R(T) gives the difference between the cumulative loss of online decisions {c t } t∈T and that of the best fixed decision in hindsight.Note that computing c * requires complete load data of T rounds.In addition, c * is only attainable in theory.In the case of a very large T, it is impractical to optimally solve problem (11) as there are too many pieces of the objective function.An online algorithm aims at producing a sequence of decisions such that R(T) grows sub-linearly with respect to T, leading to zero average regret, i.e., lim T→∞ R(T)/T = 0.In online convex optimization, capacity allocation is chosen from a fixed set C, which does not involve time-coupling constraint (8).Thus, we need another metric to measure the constraint violation of budget balance.The constraint violation of user i after T rounds is defined as We also desire that an online algorithm can ensure a sub-linear W i (T) so that average constraint violation goes to zero, i.e., lim T→∞ W i (T)/T = 0

Online Algorithm
Based on [18], we develop an online algorithm to solve P1 round by round.First, a virtual queue Q i is used to measure constraint violation of user i, which is defined as where β > 0 is a predefined parameter, and h i (c i,t ) is the constraint violation of user i at t, as shown in (12).h i (c i,t ) ≤ 0 implies that user i achieves budget balance at t. h i (c i,t ) > 0 means that the budget-balance constraint is violated, increasing the virtual queue backlog.This would give a signal to the algorithm for adjusting allocation strategies.Then, we define q i,t = [Q i,t , . . ., Q i,t ] ∈ R J and q t = [q 1,t , . . ., q V,t ].Given q t and {D i,j,t } i∈V,j∈J , the ES capacity allocation for round t + 1 is determined by where α > 0 is a parameter and d t = βP es q t + ∇ f t (c t ).Here, ∇ f t (c t ) = [. . .∇ f i,j,t (c i,j,t ), . ..] denotes the gradient of f t (•) at c t .Note that at the end of round t, both f t (•) and c t are known.Thus, we directly have ∇ f i,j,t (c i,j,t ) = P es − P pk j + P off /(η c η d ) − ω(c i,j,t + D i,j,t ) −1 if c i,j,t < D i,j,t and∇ f i,j,t (c i,j,t ) = P es − ω(c i,j,t + D i,j,t ) −1 if c i,j,t ≥ D i,j,t .Therefore, P1 has been decomposed into a series of one-round quadratic programming problems ( 14), avoiding dealing with the piecewise issue of P1.
The proposed online capacity allocation algorithm is summarized in Algorithm 1.At the end of round t, we observe peak loads {D i,j,t } i∈V,j∈J , based on which we predict capacity allocation c t+1 without knowing load data of round t + 1.According to [18], parameter α is used for scaling norm c − c t 2 2 , and parameter β is used for scaling constraint violation h i (c i,t ).α and β relate to the performance of the online algorithm.The following proposition shows that by setting α and β appropriately, the resulting online decisions can ensure a O( √ T) regret bound and infinite constraint violation.
Algorithm 1: Online algorithm for solving P1 Observes {D i,j,t } j∈J at the end of round t.

5
Update Q i,t+1 according to (13). where Proof of Proposition 1.The proof follows from the analysis in [18], where the following assumptions are made.
In the assumptions, E, H, G, K, and ε are constants.According to Theorems 1 and 2 in [18], if the above five assumptions hold, we have √ T. In the following, we prove that the proposed ES sharing system satisfies the above assumptions.Using the following results, ( 15) and ( 16) can be obtained.

1.
If c i,j,t < D i,j,t , then (∇ f i,j,t (c i,j,t )) 2 ≤ {[P es − P pk Combining these two cases, we have According to the definition of h(c), we can derive that h(c t ) where the inequality is obtained from Cauchy-Schwarz inequality.3.
i , where the second inequality is also derived from Cauchy-Schwarz inequality.4.

Distributed Implementation
If Algorithm 1 is implemented in a center, the center has to aggregate all load data from all users.However, due to privacy issues, users may not be willing to submit their load data to others.Therefore, this section designs a distributed implementation of Algorithm 1.In the distributed implementation, each user carries out local computation based on its load data and communicates with its neighbors via a P2P network without a center.

Network Problem
P2P Network Model: Consider that each user corresponds to a node in a P2P network.The P2P network can be abstracted as an undirected graph G = {V, E }, where E is the set of edges.Each edge, indexed by (i, ) ∈ E , represents a bidirectional communication link between user i and user .The set of neighbors of user i is denoted by N i = { : (i, ) ∈ E }. Users only communicate with their neighbors.The topology of the P2P network can be arbitrary as long as it is connected.Compared to star networks where a center directly communicates with all users [25], P2P networks are more scalable and free from single-point failure [26].Smart grid communication technologies [2,27,28] can implement secure information exchange between home energy users.
Problem at Round t: The problem in ( 14) can be written as Note that at the end of round t, user i knows v i,t .In addition, attaining v i,t only requires local data of user i. (17b) is the global constraint coupling all users.In the following, we focus on the case that (17b) is binding.If (17b) is unbinding, P2 would be inherently separable.
P2P Network Problem: Next, we transform P2 into a network problem solvable in a P2P network.Let y be the Lagrange multiplier of (17b).The Lagrangian dual problem of P2 can be equivalently described as where Since P2 is a convex minimization with an affine constraint, the duality gap is zero [29,30].This means that solving (18) is equivalent to solving P2.Let each user i manage a copy of y, denoted as y i [31].Define y = {y i } i∈V and µ = {{µ i } ∈N i } i∈V .Problem (18) can be transformed to the following P2P network problem: where {µ i } ∈N i are slack variables managed by user i. (20b) are consensus constraints, ensuring y 1 = y 2 = . . .= y V .Thus, P3 is equivalent to problem (18).

ADMM Algorithm
In the following, we employ ADMM to solve P3 in a distributed manner.Let λ i and λ i be the Lagrange multipliers of (20b).Define λ = {{λ i , λ i } ∈N i } i∈V .The augmented Lagrangian function of P3 is given by where ρ is a penalty parameter.Use k to denote the number of iterations.Iterative steps of standard ADMM [19] for solving P3 are as follows: Equations ( 22)-( 25) can be further simplified using the properties of P3.From the definition of µ and (23), we can derive µ k i = µ k i .Besides, according to the analysis of [32], we have λ i , ∀k if we initialize λ 0 = 0.Then, by defining γ i = 2 ∑ ∈N i λ ,k i , ( 22)-( 25) are equivalent to the following [32]: which show that P3 is split into multiple subproblems that can be solved by individual users i ∈ N .Due to (19), y-update (27) involves a min-max problem.According to minimax theorem [33], the min-max problem of ( 27) is equivalent to the following max-min problem: ⇔ max From ( 29), we have the closed-form solution of y i given by The outer maximizer of ( 29) can be obtained by where c i is bounded by c i,j ≥ 0, ∀j ∈ J .Therefore, the ADMM-based distributed algorithm for solving P3 (and P2) is described by Algorithm 2. At each iteration, each user exchanges y i with its neighbors and updates three variables.The c-update (31) involves a quadratic programming problem, which can be solved efficiently.Algorithm 2 falls under dual consensus ADMM, and related convergence analysis can be found in [34].Algorithm 2 is used to compute (14) at each round.

Simulation Setup
In the simulation, the residential TOU tariff shown in Figure 2 is used, in which P off = 17.918 cent/kWh, P pk 1 = 25.596cent/kWh, and P pk 2 = 37.123 cent/kWh [21].We set the TOU tariff unchanged every day.The home load data is taken from Dataport database [20], which includes power data of Texas, Colorado, and California.Figure 4 shows a home's load profiles of peak period 1 and peak period 2 in 2017.The peak load data is used to set D i,j,t in ( 5) and (6).We evaluate the proposed online approach in two systems: 10-home and 100-home systems.In the first system, 10 homes share an ES that has the following parameters: C max = 50 kWh, C min = 1 10 C max , and η d = η c = 0.95.In the second system, 100 homes share an ES with C max = 200 kWh (other parameters are the same as the first system).The price of ES capacity is set to be P es ≤ P pk 1 − P off /(η c η d ).The satisfaction parameter ω adjusts the trade-off between load shifting cost and satisfaction of a home, and we set ω = 30.A home's budget is set to be uniformly distributed in [0.1, 0.5] $.The proposed online approach is tested for T = 365 days (i.e., rounds).The communication network graph is generated following the method in [34].Consider that nodes are randomly located in a 100m×100m area.Let r denote the communication radius of each node.If the geographical distance between two homes is no more than r, a communication link is established between the two homes.The proposed online approach is compared with the following benchmarks: • No ES: In the case of no shared ES, users have to buy electricity from the grid to meet their immediate demands.

•
Budget-based: This is a fixed ES capacity allocation approach based on users' budgets.The amount of capacity allocated to user i for peak period j is given by where L j is the length of peak period j.Note that for each user i, we have P es ∑ j∈J c i,j ≤ U i which indicates that each user is budget-balanced.

•
Moving Average: This is a time-variant ES capacity allocation approach based on moving average [35].This approach is parameterized by a time window w.The allocation decision for round t will be determined based on the average peak loads from round t − w to round t − 1. Define Di,j,t = The amount of capacity allocated to user i for peak period j for round t is given by Note that users' budgets are not considered in this approach, so budget balance may not be guaranteed.
an online capacity allocation algorithm is developed, which is able to minimize homes' costs via learning from home load data.Thirdly, a distributed implementation of the online algorithm is proposed using ADMM framework.Finally, real-world home load data and a practical TOU tariff are used in the simulation.The results show that compared to other benchmark approaches, the proposed online approach has the lowest system cost while maintaining homes' budget balance.

Figure 3 .
Figure 3. Examples of satisfaction functions with D i,j,t = 10 kWh and D i,j,t = 50 kWh.

Figure 4 .
Figure 4. Peak load data of a home during the year of 1 January-31 December 2017 [20].