Practical Operation Strategies for Energy Storage System under Uncertainty

Recent advances in battery technologies have reduced the financial burden of using the energy storage system (ESS) for customers. Peak cut, one of the benefits of using ESS, can be achieved through proper charging/discharging scheduling of ESS. However, peak cut is sensitive to load-forecasting error, and even a small forecasting error may result in the failure of peak cut. In this paper, we propose a two-phase approach of day-ahead optimization and real-time control for minimizing the total cost that comes from time-of-use (TOU), peak load, and battery degradation. In day-ahead optimization, we propose to use an internalized pricing to manage peak load in addition to the cost from TOU. The proposed method can be implemented by using dynamic programming, which also has an advantage of accommodating the state-dependent battery degradation cost. Then in real-time control, we propose a concept of marginal power to alleviate the performance loss incurred from load-forecasting error and mimic the offline optimal battery scheduling by learning from load-forecasting error. By exploiting the marginal power, real-time ESS charging/discharging power gets close to the offline optimal battery scheduling. Case studies show that under load-forecasting uncertainty, the peak power using the proposed method is only 22.4% higher than the offline optimal peak power, while the day-ahead optimization has 76.8% higher peak power than the offline optimal power. In terms of profit, the proposed method achieves 77.0% of the offline optimal profit while the day-ahead method only earns 19.6% of the offline optimal profit, which shows the substantial improvement of the proposed method.


Introduction
Recent developments of battery technologies have made battery prices drop sharply, and thus individual customers actively deploy the energy storage system (ESS) to minimize electricity cost considering time-varying electricity price and renewable generation such as solar and wind power.ESS is also beneficial from a power system point of view, since it contributes to stabilizing the power grid.Thus, there have been many studies to minimize total electricity cost using ESS [1][2][3][4][5][6][7][8].
In minimizing total cost using ESS, energy management system needs to consider several factors such as load profiles, peak power, country-specific tariff, and time-of-use (TOU) pricing.In the literature, the ESS operation problem is formulated to have a linear objective function, and the solution can be obtainable using linear programming [2].Please note that peak cut requires min-max operation, and linear programming can be used for this purpose [3].In case operational constraints include binary decision variables, mixed-integer linear programming was used [4].Recently, to extend battery lifetime, the authors in [7] considered the state-dependent battery degradation cost and applied dynamic programming because linear programming is not effective for developing state-dependent policies.
The works in [2,4,7], however, assumed that short-term load-forecasting is error-free.Indeed, peak cut using ESS is challenging because of the uncertainty of load profile, and it is known that peak cut may be vulnerable to load-forecasting error [3,9]; even if there is small load-forecasting error, the peak power originated both from the load and ESS operation can significantly vary.This is because battery scheduling for peak cut depends on the forecasted load profile [5], and small forecasting error may cause another unwanted peak power.Strategies for managing peak power may differ from countries to countries; for example, under Korea commercial and industrial (KCI) tariff, the peak power measured for 15 min determines the monthly base cost, and it may affect up to the next 12 months [3].Thus, managing peak power is of prime importance under KCI tariff.Since the charging and discharging scheduling of ESS is typically done day-ahead, under the presence of forecasting error, day-ahead scheduling can be problematic, specifically for managing peak load.To mitigate the adverse impact of forecasting error on peak cut, the authors in [3] use robust optimization and propose to reserve some portion of ESS.Robust optimization, however, can be too conservative because it is mainly focused on handling the worst case.In addition, it is not clear how much to reserve for robust operation [3].Chance constrained programming (CCP) alleviates the conservativeness of the robust optimization [10,11].CCP uses the uncertainty with certain probability as constraints.Obtaining the probability, however, is challenging in real situations.The work in [8] generates hourly forecasted load by probabilistic load distribution to alleviate the uncertainty.However, the authors in [8] assume that the probabilistic load distribution follows the normal distribution, which may not necessarily hold in general.Scenario-based approach generates the scenarios of the uncertainty from historical measurements [12][13][14].However, this approach requires extra task such as clustering the generated scenarios.
Another challenge of ESS operation is to minimize battery degradation.Although battery price is falling, battery degradation cost is still one of the main concerns in ESS operation.Battery degradation mainly comes from charging and discharging during operation, and degradation models are based on empirical data such as depth of discharge (DoD) vs. cycle life.Rain-flow counting method was proposed to approximate total amount of battery usage in terms of DoD [15] but may not suffice because it cannot capture the state-of-charge (SoC)-dependent battery degradation.The authors in [16] proposed the concept of degradation density function to capture the battery degradation cost at each SoC, and the degradation density function is further elaborated in [17] for better function approximation.
In this paper, we propose a practical framework to manage peak load under load-forecasting error.The proposed framework has two parts: day-ahead optimization and real-time control.First, for day-ahead optimization, our method exploits dynamic programming to capture the state-dependent battery degradation.However, dynamic programming may not be as effective as linear programming in peak cut and requires an additional process to determine peak cut threshold [7].Please note that determining the proper peak cut threshold is delicate under load uncertainty.In this regard we propose a concept of internalized pricing and show that the proposed method effectively cuts peak load without relying on additional process.Second, to combat against load-forecasting error, we propose a real-time control mechanism on top of the day-ahead optimization.In doing this, we investigate a concept of marginal power, which is defined as the additionally required power to follow the optimal offline scheduling, which is assumed to know the future load profile perfectly.Indeed, offline optimization cannot be done until the scheduling time horizon is over.Hence, we propose a learning method to estimate the marginal power from the observed forecasting errors from historical dataset.In this way we can schedule the ESS in a nearly optimal way to reduce peak power even in the presence of forecasting error while, at the same time, minimizing battery degradation and electricity cost.Finally, we verify the proposed framework by case studies.We leverage the deep learning-based short-term load-forecasting using ResNet and long short-term memory (LSTM) [18] and exploit the sub-sampled electricity consumption data at every minute instead of the typical time interval of 15 min.Our experimental results verify that the proposed method reduces peak load by about 30% and improves the profit by almost four times compared to the case of using only day-ahead optimization.
The rest of this paper is organized as follows.In Section 2, we introduce the overall framework of the proposed system.In Section 3, we describe the internalized pricing-based day-ahead optimization.Real-time control mechanism to overcome uncertainty due to forecasting error is proposed in Section 4. Case studies are provided in Section 5, and we conclude the paper in Section 6.

System Overview
The proposed system has two components as shown in Figure 1.Day-ahead optimization block is used to determine the optimal battery charging/discharging scheduling based on forecasted load profiles.To alleviate the performance loss incurred from load-forecasting error, real-time control block calibrates the day-ahead scheduling.Then we estimate the marginal power for calibration using the relationship between forecasting errors and the required compensations.The estimated marginal power is then added to the day-ahead scheduling.In doing this we have two different scales of forecasting errors: 15 min interval and 1-min interval, which will be explain in detail in Section 4.2.

Day-Ahead Optimization
In this section, we describe day-ahead optimization based on the forecasted load profile, instead of the real load profile.

Internalized Pricing-Based Approach
Let L t denote the real load profile and L t denote the day-ahead forecasted load profile at time slot t ∈ T = {1, 2, . . ., T} where T is a scheduling horizon.Typically, scheduling time horizon is one day but can be longer depending on applications.Let p t denote the battery power (p t > 0 is charging and p t < 0 is discharging) at time slot t.Then, we have the following constraint for scheduling time horizon T .Let P max denote the maximum charging/discharging power of ESS, and we have − P max ≤ p t ≤ P max for ∀t ∈ T .
Let s t ∈ [0, 1] denote the battery SoC, and S min and S max denote the minimum and maximum SoC of ESS, respectively.Then, S min ≤ s t ≤ S max for ∀t ∈ T .
In case power injection from ESS into the grid is not allowed (e.g., under the regulation in Korea), the net power seen by the utility should be nonnegative, and thus Then, minimizing the total cost by using ESS can be formulated as follows [3], where P is a feasible set of battery power that satisfies ( 1)-( 4), µ t is an electricity price given by TOU, and α is a parameter determining the tradeoff between the peak cost that comes from (max t∈T L t + p t ) and the energy cost.One can solve (5) using linear programming [3] unless the SoC-dependent battery degradation cost is the concern.Battery degradation, however, cannot be ignored, and to accommodate the SoC-dependent battery degradation cost, we consider the following state transition cost function from the state s t to the state s t+1 : where β is a battery degradation coefficient [7], and ω(s) is the degradation density function that captures the state-dependent battery degradation cost.The shape of ω(s) depends on the battery characteristics and can be constructed using cycle life vs DoD data [16,17].The next battery SoC s t+1 is determined by the current state s t and the battery power p t as follow: where E max is the battery capacity.By using the transition cost function and the feasible set P, we formulate the state value function as follow: Then, an optimal battery scheduling is determined by solving the Bellman Equation ( 8) [19].However, the subtlety lies in that the solution based on dynamic programming does not consider the cost incurred by the peak load.A simple method is to impose peak power threshold by preventing abrupt state transition [7].However, determining a proper threshold requires additional process, which might be vulnerable to load-forecasting error.To overcome this limitation, we propose a new cost function by adding a penalty term γ( L t + p t ) into µ t such as where γ is a weight parameter that determines the cost of peak load.Please note that ( 9) is not restricted to the peak management under KCI tariff but can be applicable to other tariffs by properly selecting µ t and γ.Then we have a new Bellman equation as follows: The intuition of constructing ( 9) is such that TOU is replaced by a user-defined TOU which has high price when the net load (load plus battery power) becomes high.
It turns out that slight tweaking the cost function using the internalized pricing dramatically contributes to reducing the peak.Traditional peak minimization of (5) when α = 0 is the min-max operation (or L ∞ norm minimization) of L t + p t , ∀t ∈ T and can be solved by using linear programming.However, it cannot consider the state-dependent battery degradation and can be vulnerable to load-forecasting error.Finally, the day-ahead optimal battery power p t is given by

Implication of Internalized Pricing
The implication of solving ( 10)-( 11) using the internalized pricing is as follows.When β = 0 in (9) and γ is sufficiently large, both of battery degradation and the energy cost from TOU are ignored and solving (11) leads to minimizing ∑ t∈T ( L t + p t ) 2 , i.e., L 2 norm minimization, instead of L ∞ norm minimization, of L t + p t , ∀t ∈ T .Now we motivate the use of L 2 norm minimization for peak management.For simple illustration we consider a vector x = (x 1 , x 2 ) in some feasible region S ⊂ R 2 .
Let x ∞ be the solution of min ||x|| ∞ subject to x ∈ S and x * be the solution of min ||x|| 2 subject to x ∈ S. In Figure 2, we plot the feasible region S and the contours that have the same L 2 norm (dash) and L ∞ norm (dash dot).As illustrated in Figure 2a, the solutions of L 2 norm minimization and L ∞ norm minimization can be identical.That is, in this case, using L 2 method has the same peak power as L ∞ method.However, in Figure 2b, the solutions of L 2 norm minimization and L ∞ norm minimization can be different, i.e., L 2 norm minimization has higher peak power than L ∞ minimization.Back to our problem, x lies in P, which is determined from the constraints of battery power (1)-( 4), and it is of interest to know under which circumstance L 2 norm minimization is sufficient to minimize the peak load.The following proposition summarizes the result in terms of the battery capacity E max , and the initial SoC s 0 .
Proposition 1 (Sufficient condition of solution equivalence).Given the scheduling time horizon T , if the battery capacity E max and the initial SoC s 0 are given by and L 2 norm minimization has the same solution of L ∞ norm minimization.
Proof.Let L avg denote the average load in the scheduling horizon T .The battery power to make the net load equal to the average load is Since p t is provided by the battery, the summation of ( 14) over time is the energy stored in the battery.Thus, we have and By subtracting ( 16) from ( 15), E max is given by In addition, s 0 can be obtained by inserting E max into (15), which completes the proof.
Example 1. Figure 3 compares two methods for various battery capacities.In our case, surprisingly, the peak power of L 2 norm minimization is same to that of L ∞ norm minimization for all battery capacities.This is possible because Proposition 1 is the sufficient condition that L 2 minimization equals to L ∞ minimization.

The Proposed Marginal Power
So far we have discussed the day-ahead optimization based on the forecasted load profile L t .However, load-forecasting inevitably has error, and thus day-ahead optimization cannot be optimal for real load profile L t ; the optimal scheduling for the real load profile can be done only after the scheduling time horizon T is over, and thus it is indeed offline scheduling.Since day-ahead optimization cannot be the same to the offline optimization, additional mechanism is required to calibrate it.If one can estimate the power required to mimic the offline optimization by observing the load-forecasting error for some past period, real-time calibration can be possible.In this regard, we propose the marginal power, denoted by m t , as follow: where p * t is the battery power obtained by the offline optimization based on the real load profile L t .Hence, day-ahead scheduled battery power plus the marginal power can be equal to the offline optimal battery power.Let e t denote the load-forecasting error: Then, based on the offline scheduling, we investigate the relationship between the load-forecasting error e t and the offline marginal power m t .For illustrative purpose, we synthesize a forecasted load profile L t by simply adding filtered Gaussian noise to L t .In Section 5, we will use LSTM-based load-forecasting to validate the proposed approach.Surprisingly, as can be seen in Figure 4, offline marginal power turns out to have a strong linear relation with load-forecasting error.By applying linear regression, we can estimate the marginal power from the load-forecasting error as follows: where mt is the estimated marginal power, θ 0 is a bias term and θ 1 is the slope; θ 0 and θ 1 should be learned from historical dataset.The calibrated real-time battery power pt is then given by pt = p t + mt .(21) One subtlety still lies in that, however, the forecasting error e t can be measured only after time slot t ends and the real load profile L t is observed.

Leveraging Sub-Time Slot for Marginal Power Allocation
Recall that the load is measured every time slot such as 15 min.However, if the load can be measured more frequently, e.g., in every minute, which is much shorter than the duration of one time slot, we can reduce the delay to measure the load-forecasting error, and thus can allocate the estimated marginal power effectively.Indeed, there are commercially available smart meters that can sense the power consumption very frequently, e.g., every sub-second [20].Hence, the estimated marginal power per sub-time slot is allocated in the next sub-time slot.The delayed allocation of one sub-time slot is allowed as long as it belongs to the original time slot.Figure 5 illustrates how the marginal power can be estimated and allocated per sub-time slot.In this way, the proposed method performs seemingly real-time calibration.

Case Studies
In this section, we provide the experimental results of the proposed system using the real load profiles measured per sub-time slot.The time slot duration is 15 min, and the sub-time slot duration is 1 min.Since both fully charged and discharged SoC impairs the battery health, the minimum and the maximum of SoC are set as 10% and 90%, respectively.Other parameters for the experiments are summarized in Table 1.Since load-forecasting is essential, we use the ResNet/LSTM-based load-forecasting as in our previous work [18].The load-forecasting is done for every time slot as typically used in practice.The real load profiles and the forecasted load profiles are shown in Figure 6, and the original peak load is 131.5 kW.We use the load profiles of manufacturing facility, and 185 days are used for training/validation to forecast the load profiles from March 1st to 20th.One may think this data is not sufficient for reliable forecasting.However, our intension is not to accurately forecast load profiles but test our proposed algorithm under forecasting errors.Load-forecasting itself is not the scope of this paper, and please refer to [18] for the details of load-forecasting.Indeed, the proposed method can be combined with other load-forecasting techniques and would be more effective if forecasting is not accurate.Figure 7 shows the probability density function of forecasting error; the mean of error is almost zero, and the mean absolute percentage error (MAPE) is 10%.As can be seen in Figures 6 and 7, load-forecasting is not quite accurate sometimes, which is the motivation of our work.For example, in the 11th day (around time slot 1000), the forecasted load is very low, but the real load is unexpectedly high.Figure 8 is the TOU price under the KCI tariff in March.Using the parameters and the load profiles introduced so far, the proposed method calculates the optimal battery schedule considering TOU, peak load, and battery degradation under uncertainty.

Single Day Operation
The result for single day operation is presented in Figure 9.As explained in Section 4.1, we estimate the marginal power from the forecasting error using linear regression as shown in Figure 9a.For linear regression we use the data from the same day of one week ago considering the weekly periodicity of load profiles.As can be seen in Figure 9a, the marginal power is well estimated by linear regression.
In Figure 9b we present the load profiles based on various ESS schedulings.We see that from time slots 30 to 70, the forecasted load profile (purple line) deviates from the real load profile (black line), and consequently the result of day-ahead optimization (green line) is quite different from the offline battery scheduling (red line).The original peak load is 109.2kW, and the day-ahead optimization reduces it to 87.4 kW, i.e., 20.0% of peak reduction.By contrast, the proposed method achieves 60.1 kW of peak power, which is 31% of improvement from the day-ahead optimization.Please note that the peak power of offline scheduling is 55.7 kW, so the proposed method is almost as good as the optimal result.Table 2 summarizes the cost/profit analysis.The proposed technique earns $419, which is more than double compared to the profit of the day-ahead optimization, and is also close to the offline optimal profit of $452.6.

Robustness of the Proposed Method
Next we investigate the robustness of the proposed method.Specifically, we consider the case when the forecasted load profile has time misalignment, which may happen when the operation of manufacturing facilities unexpectedly shifts in time.Figure 10a shows the relationship between the load-forecasting error and the marginal power, and the marginal power can be well estimated from the load-forecasting error.Figure 10b shows the load profiles based on various ESS schedulings.In the case of day-ahead optimization, the failure of load-forecasting results in very unwanted load profile where the peak power is not reduced at all.Indeed, as seen in Table 3, the peak power slightly increases from 109.2 kW to 109.4 kW, and ESS becomes of no use.By contrast, the proposed method reduces the peak power to 67.2 kW while the optimal peak power from offline scheduling is 55.7 kW.Thus, the proposed method works well even under the unexpected load shift in time.Furthermore, the profit earned by the proposed method is $368.6, which recovers 81.4% of the profit from the offline scheduling.

Multiple Day Operation
Finally, we apply the proposed method for multiple day operation.As in the case of single day operation, we use the data from the same day of one week ago for linear regression.In single day operation, the starting and the ending SoC are both set as s 0 for all three methods as shown in Table 1.This still holds for day-ahead optimization and offline optimization because the sum of battery power in one day is zero.In the case of the proposed method, however, the battery power is adjusted by the marginal power, whose sum in one day may not be zero.Hence, we cannot enforce s 0 for both the starting SoC and the ending SoC. Thus, we set the target SoC as s 0 only at the end of each day.As can be seen in Figure 11 and Table 4, the proposed method outperforms the day-ahead method and closely follows the offline optimization.The original peak load is 131.5 kW without using ESS, and the day-ahead method reduces the peak to 97.2 kW while the proposed method further reduces it to 69.0 kW, which is also comparable to the offline optimal peak of 55 kW.It should be noted that the scheduling horizon is just one day for the proposed method.Thus, the proposed method for multiple days is applied by repeating the scheduling horizon.In the case of the offline optimization, however, the scheduling horizon covers the whole days to compute the theoretical minimum bound.In terms of profit for the proposed method, the battery degradation cost is $141.4,and the profit is $595.1.By contrast, the day-ahead method has only $151.7 of profit.The result for a longer period of operation can be obtained when more data measured in sub-time slots become available.

Conclusions
In this paper, we proposed a two-phase approach to minimize the total cost including the peak cost, the energy usage cost from TOU and the battery degradation cost.The first phase is the day-ahead optimization based on the forecasted load profile and exploits the concept of internalized pricing, which enables us to effectively manage the peak load without bothering to select the peak cut threshold.The proposed method is based on dynamic programming and thus can accommodate the

Figure 1 .
Figure 1.Overall framework of the proposed ESS scheduling.

Figure 3 .
Figure 3. Comparing L 2 and L ∞ norm minimization in terms of peak management.

Figure 4 .
Figure 4. Relationship between marginal power and load-forecasting error.

Figure 5 .
Figure 5. Real-time control technique in sub-time slots.

Figure 9 .
Figure 9. Result of single day operation.(a) Forecasting error and marginal power, (b) Load profiles obtained by various ESS schedulings.

Figure 10 .
Figure 10.Result of single day operation with unexpected load shifts in hours.(a) Forecasting error and marginal power, (b) Load profiles obtained by various ESS schedulings.

Figure 11 .
Figure 11.Result of multiple day operation.

Table 2 .
Cost and profit analysis of single day operation.

Table 3 .
Cost and profit analysis under unexpected load shift in time.

Table 4 .
Results of the proposed method for multiple day operation.