Economic Value Assessment and Optimal Sizing of an Energy Storage System in a Grid-Connected Wind Farm

This study identifies the optimal management policy of a given energy storage system (ESS) installed in a grid-connected wind farm in terms of maximizing the monetary benefits and provides guidelines for defining the economic value of the ESS under optimal management policy and selecting the optimal size of the ESS based on economic value. Considering stochastic models for wind power and electricity price, we develop a finite-horizon periodic-review Markov decision process (MDP) model to seek the optimal management policy. We also use a simple optimization model to find the optimal storage capacity and charging/discharging capacity of the ESS. By applying our analytic approach to a real-world grid-connected wind farm located in South Korea, we verify the usefulness of this study. Our numerical study shows that the economic value of the ESS is highly dependent on management policy, wind electricity variability, and electricity price variability. Thus, the optimal size of ESS should be carefully determined based on the locational characteristics and management policy even with limited investments. Furthermore, this study provides a meaningful policy implication regarding how much of a subsidy the government should provide for installing ESS in a wind farm.


Introduction
As greenhouse gas emission reduction has recently received extensive attention, renewable energy resources have been rapidly integrated into the electricity sector around the world.Several countries, including South Korea, Britain, Italy, Poland, Belgium, and Chile, as well as most states of the U.S., have aggressively adopted renewable policies such as renewable portfolio standard (RPS).According to a recent report published by the International Energy Agency (IEA), renewable energy resources will account for the largest portion of total primary energy consumption in the global electricity sector in 2030 [1].The report projects that wind energy will have the largest contribution to the penetration.
As the penetration level of the wind energy in an electric power system increases, the critical weak points of the wind energy-intermittency and non-dispatchability-have posed more challenges in the operation of the electric power system in terms of the quality of power, liability, and so on.As attempts to overcome these challenges, new technologies have been developed, such as a smart grid and/or an energy storage system (ESS).In particular, with recent technological advancement and reduced costs, was installed at the 153 MW Notrees wind power project, 2 MW and 1 hr duration ESS was installed at the 18 MW Bosch wind power project in the northern Germany, and five serial ESS projects, in total 16 MW/71 MWh, were invested for a hybrid system with 500 MW wind and 100 solar PV capacities in Zhangbei, China.
In South Korea, the government has developed a plan to actively expand the use of renewable energy in the electricity sector.One goal of the plan announced in 2014 is to increase the portion of renewable energy resources in the electricity sector from 3.66% in 2012 to 13.4% in 2035.Among renewable energy resources, wind energy is expected to account for the largest portion (more than 30%) in 2030 [3].Accordingly, a national renewable energy policy mandates the RPS for power producers whose installed capacity is over 500MW.In particular, the Jeju Island, home to 600,000 people, has a plan to generate electricity from only renewable energy sources and considers wind turbines as the primary renewable sources.
As a way to resolve the weak points of the wind energy, the Korean government has encouraged the adoption of ESS by giving a much higher subsidy to a wind farm that has its own ESS.In 2016, a power supplier with ESS connected to a wind farm can receive five RECs (Renewable Energy Certificates) for a unit of electricity generated during the peak period under the Korean RPS policy.On the other hand, a wind farm without ESS receives only one REC for a unit of electricity generated.In 2015, the Jeju Island legislated that a new wind farm must have its own ESS, and its charging/discharging capacity should be larger than 10% of the nameplate capacity of the wind farm.
Due to the high upfront costs of ESS installation, the decision-making problems related to the ESS in a wind farm has recently received considerable attention.Making reckless decisions on the ESS size and its management policy could limit the economic benefits.This paper particularly attempts to address the following decision problems: (1) identifying a management policy for optimally operating the ESS, (2) defining and assessing the economic value of the ESS, (3) identifying the factors that affects the economic value, and (4) identifying the optimal size of the ESS that maximizes the economic value.This study mainly focuses on making suggestions on how to economically install and operate the ESS in practice.
We consider a wind farm integrated with ESS, which sells the electricity into the grid to maximize the economic benefits under the Korean regulatory framework.
The rest of this paper is organized as follows.We first review the relevant literature on wind-ESS hybrid systems and the related optimization models and emphasize our new contributions in Section 2.
We introduce a finite-horizon Markov Decision Process (MDP) model to identify an optimal management policy for operating the ESS in Section 3, followed by the structural analysis of the optimal management policy in Section 4. Section 5 describes the economic value of the ESS under the optimal management policy and how to decide the optimal size of the ESS based on the economic value.In Section 6, for a verification purpose of our analytical results, we conduct an extensive numerical study with real data compiled from a wind farm located in South Korea and the Korean electricity market.Lastly, conclusions and discussions are in Section 7.

Literature Review
This section summarizes the recent research progress in assessing the economic value and determining the optimal size of the ESS in a grid-connected wind-ESS hybrid system.Comparing with previous literature, we clarify the contributions of this research and briefly describe the regulatory framework on the ESS operation and relevant studies in Korea.

Summary of methodological advance
Since the mid-2000s, several studies has considered the effective use of ESS regarding the interplay of wind turbines, energy storage, and transmission capacity and the evaluation of its economic value [4][5][6][7].These studies were based on deterministic sample paths of electricity price and wind energy dynamics by analyzing historical data.By conducting extensive sensitivity analysis for various sizes of the ESS in the hybrid system, they evaluated the economic value for a specific ESS size and found the optimal size of the ESS through the simple cost-benefit analysis.The types of ESS considered in the papers are the compressed air energy storage(CAES) [4,5,7] and the battery energy storage [6,7].
However, they have not considered the effect of the management policy of the ESS, which can vary its economic value.
To find the optimal management policy of the ESS in the hybrid system, several studies have employed deterministic optimization models.The deterministic models assume that the future values of electricity spot price, demand load and wind energy generations are known.Korpaas et al. [8] characterized an optimal strategy for ESS operation and sizing an on-site ESS with given capacities of a wind farm and transmission to the external grid, and a known demand distribution.They used a dynamic programming approach to analyze the strategy.Brekken et al. [9] considered a large wind farm integrated with an on-site zinc-bromine flow battery, with the objective of meeting an hour-ahead predicted power output to a large grid.They focused on the total costs of the entire grid rather than the hybrid system and ignored the transmission capacity between the hybrid system and the grid.It was shown that an optimal operation strategy could result in significantly lower costs than a simple strategy.Zhang and Li [10] used a two-scale dynamic programming scheme and considered the least-cost management policy of a wind-ESS hybrid system, assuming that local demand was known and the ESS was allowed to charge the electricity from the utility grid.Luo et al. [11] and Bridier et al. [12] also optimally determined the ESS size and its management policy for a system similar to the system considered in this study (see Section 3).However, both studies applied heuristic approaches to solve their deterministic models.
Taking account of the uncertainty associated with electricity spot price and wind energy, several papers have used MDP models and stochastic dynamic programming approaches to find an optimal ESS management policy and evaluate its economic value based on the policy.Shu and Jirutitijaroen [13] found the optimal policy from their stochastic MDP model.They showed that the policy could lead to considerably higher profits than an optimal policy derived from a deterministic model because the deterministic model underestimated the economic value of the ESS.Kim and Powell [14] derived a mathematical form for an optimal management policy of an ESS in a wind farm assuming simple probability distributions for uncertain factors.They used the policy to study the economics of the storage capacity.Similarly, Zhou et al. [15] suggested an optimal policy for operating a wind-ESS hybrid system with limited transmission capacity and quantified the economic value of the ESS under the policy.
However, the optimal ESS size under uncertainty has not been mainly considered in the studies, which could be critical for installing a new ESS.Harsha and Dahleh [16] defined the economic value of the ESS as reductions in the long-run average cost by using the ESS based on an infinite-horizon MDP model and examined the trade-off between the value and capital costs of the storage in a simple convex optimization problem to find the optimal ESS size.
In accordance with the recent trend, we focus on a stochastic dynamic programming approach to assess the economic value of the ESS in a grid-connected wind-ESS hybrid system.With the goal of minimizing the cost of operating the ESS, we formulate a MDP model and find an optimal policy.
Our paper contributes to the literature by introducing a way to determine the optimal ESS size that maximizes the economic value of ESS.Table 1 summarizes the literature review.
We mainly refer to the work in [15,16] when formulating an MDP model in Section 3. Contrary to the previous studies in [13][14][15][16], we consider that the ESS cannot charge the electricity transmitted from the utility grid because it is more suitable for the Korean regulatory framework (see Section 2.2).It is also important to note that we find similar structural properties between our model and traditional inventory control models with space and injection/withdrawal capacity limits [17].Therefore, we refer to papers in the field of inventory management [18][19][20] when formulating the problem in Section 3.

Studies in Korea
Under the Korean regulatory framework, a grid-connected wind farm is allowed to sell its generated electricity through a wholesale electricity market, operated by Korea Power Exchange (KPX), if it has more than 1,000kW nameplate capacity.The electricity generated by a wind-ESS hybrid system is sold at the SMP, along with additional benefits such as high tradeable RECs and investment and production tax credits [21,22].Thus, wind-ESS hybrid systems are not allowed to purchase electricity from the grid to receive the benefits.If the ESS is able to charge electricity from the grid to provide an arbitrage opportunity, the ESS becomes an individual power provider who is likely to increase its profit only by trading electricity with the grid instead of being a tool to promote the wind energy.Several previous studies in Korea considered a situation where the electricity transmission from a wind-ESS hybrid system to grid is only allowed but the reverse is not [23][24][25].
To the best of our knowledge, few research has simultaneously considered the economic value assessment and optimal sizing of the ESS in Korea.Only a few studies have suggested a simple method for determining the ESS size.Lim [26] presented a simple linear programming model to design the optimal ESS size in a hybrid system consisting of solar PV, wind and tidal.Cho et al. [27] proposed a heuristic method for the optimal sizing of a demand side customer's battery storage system.Therefore, this study contributes to the limited literature and supports decision-makers by providing tangible research outcomes based on real data from a wind farm in Korea.

Optimal ESS Management Policy Model
In this study, we consider a grid-connected hybrid system as shown in Figure 1.The hybrid system has several wind turbines (a wind farm) and an ESS, and the system is connected to a large utility grid via a transmission line.The system can sell all amounts of electricity transmitted to a wholesale market on the grid assuming that the amount of electricity transmitted from the system to the grid is negligible compared to the total amount of electricity on the grid.Thus all amounts of electricity transmitted from the system to the grid is sold at SMP on the market, regardless of the demand level on the grid, and do not affect the SMP.It is also assumed that the electricity can not be transmitted from the grid to the hybrid system through the transmission line as following the Korean regulatory framework described in Section 2. Here, the electricity generated from the turbines cannot be accurately anticipated due to uncertain wind speed, but the ESS will save some electricity from the farm to provide the electricity to the grid when the wind turbines cannot generate enough electricity.Thus, the stored electricity can be sold to increase the profit of the wind farm when the SMP is relatively high.
For this situation, we first identify the optimal operating policy of the ESS in the hybrid system.
The operator of the system needs to periodically make decisions on the optimal level of stored electricity subject to the wind power availability, ESS capacity, SMP price, and transmission capacity.
That is, the system makes decisions periodically over a finite horizon, at each time t in the finite set T := 0, τ, ..., T − τ (that is, the length of time interval is τ).Therefore, we consider this problem as a period-review inventory management problem where decisions are made at equally spaced points in time, and we develop a finite-horizon periodic-review MDP model as follows. Parameters: • S: storage capacity of the ESS (in energy unit, e.g.MWh).
• W: nameplate capacity of the wind farm (in energy unit/period, e.g.MW).
• R i , R o : charging and discharging capacity of the ESS in a specified period, respectively (in energy units/period, e.g.MW).
• C T : transmission capacity (in energy unit/period, e.g.MW).
• η: transmission efficiency, the ratio of energy dissipated by the load to the transmission line.
• δ: one-period risk-free discount rate (0 Generally, charging and discharging capacities are the same, R i = R o , so we assume that

State variables:
The period t is defined as the time interval [t, t + τ).We assume that the state variables with subscript t are realized at the beginning of period t.For example, the amount of wind electricity generated in the time interval [t, t + τ) is known at time t, denoted by w t .It is assumed that the w t is followed by a well-known stochastic process after t.In this work, we use an exogenously defined Markovian process to model w t .Also, the electricity price at time period t, denoted by p t , is assumed to follow a pre-defined electricity SMP pattern.Thus, the state at time t is defined by the following variables.
• x t : the level of available electricity in the ESS at the beginning of period t (in energy unit, e.g.MWh) • w t : the wind electricity generated in time period t (in energy unit, e.g.MWh) (w t ∈ [0, τW]).
• p t : the electricity price at time t, which will not be changed during the time period [t, t + τ) (in currency unit/energy unit, e.g.$/MWh) The tuple E t = {x t , w t , p t } forms the state of our problem.

Decision variable:
• a t : the amount of electricity to charge/discharge at time period t (in energy unit, e.g.MWh).
It is positive when the electricity is stored (if a t > 0, charging) and negative when a part of stored electricity is withdrawn to be sold (if a t < 0, discharging).Assume that there is no way to buy and store electricity from the grid, that is, electricity generated only by wind turbines can be stored.
State transition: The level of available electricity in the ESS at time period t + 1 changes depending on the amount of electricity to charge/discharge at time period t.The state variables w t and p t evolve to w t+1 and p t+1 according to their respective exogenous stochastic process, expressed as known functions g 1 The first case is when a part of wind electricity generated at time t is stored.The selling amount can be smaller than the generated electricity at time t.If a part of the stored electricity is extracted and delivered, which is the second case of the equation above, the total amount of electricity sold at time t becomes the generated wind electricity plus the electricity extracted from the storage at time t.In both cases, the amount of electricity to sell cannot be greater than the given transmission capacity.To specify feasible values of a t , we consider several constraints as follows.
• When a t ≥ 0, Objective function: Our objective is to maximize the total discounted expected cash flows, monetary benefits that the grid operator can make by selling the electricity into the grid, over all feasible decisions: where π is a feasible policy which is a sequence of decisions, and Π is the set of all feasible policies.
The expectation is taken with respect to the distribution of the random state E t in time period t.
The exogenously determined stochastic models for wind electricity and electricity price induce the distribution, and the value function of the optimal management policy for state E t in each time period t is defined as Also, we assume that the remaining electricity in the ESS at the end of time period T is worthless, that is,

Analysis of Optimal ESS Management
We now proceed to investigate structural properties of the optimal management policy of the ESS.In particular, we analyze charging/discharging action, a * t (E t ), of the MDP model defined in section 3, with policy can be easily determined by using the deterministic dynamic programming.However, since we assume that both the electricity price and the wind electricity generated follow the exogenous stochastic processes, the ESS management policy should consider their variabilities.Also, the ramp constraints from the charging/discharging capacities of the ESS and the maximum electricity sale constraint from the transmission capacity raise other concerns about the optimal management policy.
The structure of the optimal management policy can be established in a way that is similar to previous studies [15,16].To derive the optimal management policy, a * t (E t ), the well-defined stochastic models for wind electricity generation and electricity price are incorporated into the MDP model so the optimal ESS management decision at each time period can be found using standard backward recursion.
Before we analyze the optimal management policy, in a way that is similar to the proofs of Lemma 1 and Proposition 2 in the study of Zhou et al. [15], we could see a property that the value function is a non-decreasing and concave function in the level of available electricity in the ESS, x t , given any wind electricity generated w t and the electricity price p t .This property implies that without the holding cost for the electricity stored in the ESS, the more electricity the ESS has is always monetarily beneficial and the marginal benefit decreases with the higher level of the electricity.With this property, we show that the optimal management policy has different dual-threshold structures depending on the two state variables x t and w t as follows, illustrated in Figure 2. We also show that these dual-threshold levels are functions of two state variables w t and p t .
Case 1, (x t , w t ) ∈ (A1), represents a situation where the amount of the wind electricity generated in time period t is very large; so, even though the ESS is charged as much as possible, the remaining electricity is still larger than the maximum transmittable amount of electricity through the transmission line to the grid.Secondly, case 2, (x t , w t ) ∈ (A2), represents a situation where the amount of the wind electricity generated in time period t is large but, if the ESS is charged as much as possible, all of the remaining electricity can be transmitted to the grid.Lastly, case 3, (x t , w t ) ∈ (A3), represents a situation where the wind electricity generated in time period t is less than the maximum transmittable amount of electricity through the transmission line to the grid.Proposition 1 establishes the structure of the optimal management policy and its proof is provided in Appendix A. Proposition 1.The optimal charging/discharging action at time period t, a * t (E t ), is determined by two threshold functions, x t (w t , p t ) and x t (w t , p t ), as follows: where, when y t = x t + a t (the ending level of available electricity in the ESS), the two thresholds can be defined as follows: storage generation up-to level: sell down-to level: Note that when we have the charging/discharging efficiencies, ρ i , ρ o , the optimal management policy has a two-thresholds structure.If ρ i = ρ o = 1, then the two thresholds are the same, x t = x t .
Under the two-thresholds structure, the optimal charging/discharging action in Proposition 1 implies the following situations.In case 1, (x t , w t ) ∈ (A1), a part of the large amount of the wind electricity generated at the time period t will be transmitted to the grid as the maximum transmission capacity, τC T , another part of the wind electricity will be charged into the ESS as much as possible, and the remaining part of the wind electricity will be curtailed.In case 2, (x t , w t ) ∈ (A2), of the amount of the wind electricity generated at the time period t, at most τC T amounts will be transmitted into the grid and at least (w t − τC T ) amounts will be charged into the ESS.If the level of available electricity in the ESS at the beginning of the time period is low enough, then the ESS will be charged up to the storage generation up-to level.In case 3, three possibilities exist.First, some parts of the wind electricity generated at the time period t will be charged and the other parts of the wind electricity will be transmitted into the grid when the level of available electricity in the ESS at the beginning of the time period is low enough.
Second, all wind electricity generated at the time period t will be transmitted into the grid when the level of available electricity in the ESS at the beginning of the time period is between the storage generation up-to level and the sell down-to level.Last, not only all wind electricity generated at the time period t will be transmitted into the grid but also some parts of the electricity stored in the ESS will be discharged and transmitted into the grid when the level of available electricity in the ESS at the beginning of the time period is high enough.Figure 3 shows the structure of optimal charging/discharging action of the ESS in case 3.

Economic Value and Optimal Sizing of ESS
This section presents the way to assess the economic value of ESS under the optimal storage management policy established in the previous section and to find its optimal size based on the economic value.The profit at a wind farm is made by selling electricity generated by wind turbines or stored in storage facilities.For a given size of ESS, the amount of electricity sold at time period t can be optimally determined taking into account the variabilities of wind electricity and electricity price as described in the previous section 4. It is obvious that the value function without ESS-that is the case of S = 0-is less than that with ESS (S > 0).We estimate the economic value of ESS by computing the difference between the value functions with ESS and without ESS.Assuming that the investment cost for ESS depends not only on storage capacity S but also charging/discharging capacity R, we define the cost function in a simple way, as in previous studies [9,28] where c s and c c are the unit capital costs for the storage capacity and the charging/discharging capacity of ESS, respectively.Here, β t represents the charging/discharging efficiencies, i.e. −1/ρ i if a t ≥ 0 and −ρ o if a t < 0. Since the wind electricity and electricity price are not deterministic, we take the average profit under the optimal management policy of ESS.The first term of the function above is the average profit made by selling electricity with ESS and the second term is the average profit without ESS-that is, the case that all amounts of electricity generated at time period t are sold at the current price p t .As a result, the difference between the first term and second term is defined as the economic value of ESS.
The third term is the investment cost for storage over the time horizon, T.
In general, the value of the first term cannot be simply estimated because the optimal value of a t with a specified size of ESS must be first obtained as we described in section 4. Thus, we consider the two-stage optimization approach.At the first stage, a specified size of ESS, (S, R) is fixed, and then the optimal management policy of the ESS is determined at the second stage to compute the average values in (11).By testing various ESS sizes, the optimal ESS size will be searched.For our analysis, it is useful to define the following function with a * t (S, R), which is the optimal a t with a given size (S, R): In fact, this function is equivalent to the optimal value of the first term of (11) and indicates the average profit with the optimal ESS management policy under a given size (S, R).According to Theorems 11 and 12 of Harsha and Dahleh(2015) [16] and our numerical results in section 6, we could see a property where f (S, R) is non-decreasing and concave in S and R. With this property, it is also easy to see that the objective function in (11) is concave because the second term is not dependent of S and R and the third term c s S + c c R is a linear combination of S and R. Thus, the optimal ESS size -storage and charging/discharging capacities, (S * , R * )-can be obtained by any two-dimensional search method such as the method of gradient descent.

Numerical Study
In this section, we apply our analytic approach to find the optimal ESS size based on the appropriate evaluation of economic value for a real-world grid connected wind farm-Shinan wind farm 1 located in the South Korea-as a numerical study.The wind farm consists of three Mitsubishi MWT-1000A turbines and the total generation capacity of the wind farm is 3 × 1 MW = 3 MW.The wind farm started operation in December 2008.Using historical wind electricity generation data of the wind farm and historical electricity price data of the Korean electricity market, we develop two stochastic models for wind electricity and electricity price.Since both historical data are hourly-based data, the time unit in this numerical study is set to be one hour.Relative to the smallest capacity of the general transmission line to the grid (about 20 MW), the total generation capacity of the wind farm is too small.As a result, the transmission capacity term, C T , in our model turns out to be ignored for this numerical study.Hence, the optimal charging/discharging action in Proposition 1 is not affected by the C T , but the two threshold values of x t (w t , p t ) and x t (w t , p t ) for each w t and p t still exist.

Experimental Setup
In this numerical study, the economic value of ESS is evaluated by testing several storage capacities, S, and charging/discharging capacities, R. The storage levels are discretized in 0.01 MWh increments.
The storage efficiency parameters are fixed as ρ i = 0.9 and ρ o = 0.95, so the roundtrip efficiency is 0.85.
The annual risk-free discount rate is assumed as 10%.
The unit time period in the MDP model is one hour, but we compare the expected annual economic value and the annualized investment cost of ESS in order to determine its optimal size for the given wind farm.With this time scale, in order to avoid the computational burden, we calculate the expected annual economic value of ESS as follows.First, based on monthly differentiated wind electricity and electricity price models, we obtain the expected economic value of ESS for one day in a specific month from the increment between the value function of the MDP model with ESS and without ESS under the optimal management policy.The analysis for one day can incorporate the hourly variations of wind electricity and electricity price, sufficiently.However, in order to avoid the effect of the terminal condition usually occurring in finite MDP models, we use the increment of the value function between time period 0 and time period 24 after setting the analysis time horizon, T, of the MDP model as 48 hours (i.e. two days).
Thus, the economic value of ESS for one day in a specific month, D m (S, R), can be calculated as follows.
where V * t (S, R) is the value function of the optimal management policy at time period t under given S and R when T = 48.Here, we calculate the 'expected' economic value for one day with different levels of initially stored electricity.
With the above economic values for one day in each month, the expected economic value for each month is calculated by multiplying the numbers of days in the corresponding month where M m is the number of days in the month m).After that, the sum of each expected economic value for each month becomes the expected annual economic value (i.e.Y(S, R) This approach can incorporate monthly variations of wind electricity and electricity price explicitly.Figure 4 shows the wind electricity generated in January 2011 and June 2010, and indicates that the amount and variability of wind electricity are different for each month.The mean and standard deviation of each month are shown in Figure 5.As shown in these figures, the monthly difference is clearly observed.In Figure 6, the mean values of wind electricity per hour for each month are shown and hourly patterns seem obvious.Even though the yearly variation exists, it is not as significant as monthly and hourly variations.Thus, we focus on monthly and hourly variations to build a stochastic wind power model.
The wind electricity has often been modeled as an autoregressive model (AR) in literature [15,29], so we test its fitness to our data by testing several orders of autoregressive models.Before we develop the autoregressive model for the wind electricity, we perform the data preprocessing as follows.First, we transform the raw data, shown to be a non-stationary and non-Gaussian process, into the data, which follow a stationary and Gaussian process.Then, to remove the seasonal and diurnal effects of the transformed data, each of the data is standardized and stabilized by (13), which is similar to standardizing normally distributed data.
where φ is the AR coefficient and ∼ N(0, σ 2 Z ), which are estimated as φ = 0.9335 and σ 2 z = 0.1265 for our data.
To develop a discretized version of the dynamic programming for our MDP model, we discretize the wind electricity generation and construct the trinomial model for each month by the method described in Jaillet et al.(2010) [30].In our numerical study, the trinomial lattice is built to model z t for 2 days each month, i.e., 48 hours, to coordinate the analysis time horizon of our MDP model and the corresponding wind electricity, w t , is obtained by reversing the equation ( 13) and limiting each value between maximum (3MW) and minimum values (0MW).Here, since AR(1) is a discrete-time analogue of the mean-reverting process, the wind electricity model is basically a Markov process.

Electricity price model
In similar way to construct the wind electricity model, we collected historical data of electricity price in the Korean electric power system, in order to construct the electricity price model.We collected hourly based SMP data during last three years, from 2012 to 2014, from a web-based database system operated by Korea Power Exchange, Electric Power Statistic Information System (EPSIS) [31].The data exhibit the minimum and maximum SMPs during the years at 34.51 KRW/kWh and 281.76 KRW/kWh respectively.Figure 7 shows the estimated average SMPs and its standard deviations at each h and each m, used for this numerical example.Peer-reviewed version available at Energies 2018, 11, 591; doi:10.3390/en11030591

Preprints
By using the historical data, for this numerical example, we develop a simple stochastic model, which contains two main characteristics of electricity price-hourly and monthly seasonality as well as volatility, as follows.
where, p(m,h) is the estimated sample mean SMP at corresponding h and m, σ (m,h) is the estimated sample standard deviation of SMPs at corresponding h and m, and e t is the independent and identical standard normal distributed random number (e t ∼ i.i.d.N(0, 1)).By discretizing the random number e t into seven numbers with the probabilities from the standard normal distribution, we also discretize the electricity price model.

Costs and Capacities of the ESS
In order to find the optimal ESS size, the capital costs of ESS should be incorporated as we described in section 5.Even though the ESS capital costs are very uncertain yet, we use the reference capital costs based on recent, highly cited literature [32,33].Among several types of storage technologies, we focus on the capital costs of the lithium-ion battery as our reference costs data in this numerical study, which are 600 ∼ 2, 500 USD/kWh for storage capacity and 1, 200 ∼ 4, 000 USD/kW for charging/discharging capacity.Even though it has higher costs compared to other technologies, we choose the technology because it has many advantages such as high energy density, high charge/discharge currents, and high efficiency so it is the most popular in several recent real-world ESS projects in wind farms [2].According to the projects, we further assume that its discharging duration time can vary from 15 minutes to 6 hours, that is 0.25R ≤ S ≤ 6R.In addition, we assume that the lifetime of the ESS is 10 years.
Using the reference costs of the lithium-ion battery, we simply calculate the equivalent annual cost (EAC) for c c and c s as follows.

Expected annual economic value
The results of numerical analysis show that the annual economic value of the ESS, Y(S, R), depends on the sizes of storage capacity, S, and charging/discharging capacity, R. economic values obtained by applying two management policies for operating the ESS, one using the proposed MDP model and the other using a simple rule developed similar to the literature [6,7].The simple rule uses the means of historical data of wind electricity and electricity price as their deterministic profiles, allows only one time of charge/discharge cycle per day, and makes the ESS fully charged at the lowest electricity price and fully discharged at the highest price.
For a comparison purpose, we run a test for a situation where S and R are 1.5MWh and 1.5MW, respectively.The test shows that operating the ESS by the simple rule provides much lower economic value than using an optimal policy proposed in this study.The economic value based on the simple rule is approximately 17 ×10 6 KRW/year, which is only about 40% of the economic value expected from our method.This simple comparison strongly supports the importance of an optimal policy for operating the ESS.Furthermore, it implies that wind farm owners are likely to make an erroneous decision on the ESS size determination when they do not follow an optimal management policy but use a simple rule.

Optimal size
With above annual economic value of ESS, the optimal storage size S * and optimal charging/discharging capacity R * in (11) can be obtained when the financial cost of storage (c c R + c s S) is specified.With cost ranges we refer to in section 6.4, we notice that it is impossible to make a profit from the ESS in the wind farm.We further note that if the reference costs come down by a factor of 10, then the wind farm may make a profit from the installation of ESS with appropriate sizes of S and R. Therefore, we consider the revised reference costs (reduced by a factor of 10) below.To help select the optimal values of S and R, Figure 9  Peer-reviewed version available at Energies 2018, 11, 591; doi:10.3390/en11030591q q q q q q q q q q 0.3 0.6 0.9  On the other hand, Figure 10 shows the expected annual economic value, Y(S, R), and is compared to the cost range of c s , [9.8, 39.1] ×10 6 KRW/MWh/year, with different values of S for a fixed R = 1.5 MW (c c = 20 × 10 6 KRW/MW/year), which is also a 2-dimensional cross-section of Figure 8. Due to the high cost of R (20 × 10 6 KRW/MW/year), the profit cannot be earned until S is greater than about 1.8 MWh with the investment cost close to the lower bound (i.e.c s = 9.8 × 10 6 KRW/MWh/year).To determine an appropriate choice of the ESS size, a careful comparison of c c and c s should be made to maximize the economic benefit from the ESS.From the results in Figure 9 and Figure 10, it can be seen that S needs to be at least greater than R and R needs to be greater than S/5, i.e., R < S < 5R.
From the above comparisons between economic values of ESS and costs ranges, we establish a guideline for a wind farm owner who is considering the installation of the ESS, about how he or she can decide appropriate sizes of the ESS, S and R, in his or her farm.Furthermore, the comparison results give other guidelines to ESS manufacturers and policy makers.ESS manufacturers can identify how much they need to reduce costs of ESS in order for the ESS to have cost-competitiveness in South Korea, and, at the same time, the policy makers can identify how much subsidy the government needs to provide for the installation of the ESS in wind farms under the given electricity wholesale market, wind electricity potential, and costs of the ESS.

Sensitivity analysis
In our numerical study, we notice that the economic value of the ESS is affected by several factors such as ESS management policy, wind electricity variability, and electricity price variability.First of all, the management policy of the ESS can affect the economic value.Peer-reviewed version available at Energies 2018, 11, 591; doi:10.3390/en11030591q q q q q q q q q q 0.3 0.6 0.9  q q q q q q q q q q q q q q q q q q q q 0.3 0.6 0.9 1.2 1.5 1.8 2.1 2.4 2.   S, R) of the optimal policy vs. that of the naive policy with different storage capacity size from increasing the capacities of the ESS quickly drop to zero under the naive management policy.Thus, the economic gains obtained by the optimal management policy over the naive management policy increases as S and/or R increases.This result indicates that the choice of the ESS management policy is critical when the optimal ESS size is determined.Consequently, the benefit from ESS can be maximized not only with the optimal size of ESS and but also with the ESS management policy carefully determined.
To see the impact of wind electricity variability, Figure 12  Peer-reviewed version available at Energies 2018, 11, 591; doi:10.3390/en11030591q q q q q q q q q q 0.5 1.0 1.5 2.0 2.5  To remove the monthly effect of w t , we fix the mean and standard deviation of w t , i.e., w(m,h) t and σ (m,h) t in (13).As a result, the trinomial model for w t becomes the same for each month.With 3 MW of R, the Y(S, R) increases as more w t is produced, but when the mean of w t is greater than the storage capacity size S = 1.5 MWh, the increment of Y(S, R) becomes smaller, as expected.It is interesting to note that the larger variability of w t does not help improve the economic benefit of the ESS.This is because a larger variability of w t might give less of a chance to take advantage of the price variability, compared to a smaller variability of w t , i.e., it would be unable to charge more at a lower price and discharge more at a higher price.With 0.3 MW of R, the distribution of w t does not affect Y(S, R) much since the use of the ESS is very limited due to R.
Lastly, Figure 13  To remove the monthly effect of p t , we fix the mean and variance of p t for each month and each hour in (14).We should note that a larger variability of p t improves the economic benefit of the ESS because the electricity range becomes wider with the larger variance of p t .

Conclusions
This paper describes how to identify the optimal management policy of the ESS installed in a grid-connected wind farm in terms of maximizing economic benefits, and, more importantly, provides an analytic guideline for defining the economic value of ESS under the policy and selecting its optimal size.Furthermore, a numerical study is carried out to show its usefulness.
In this paper, we define the economic value of ESS as the difference between the average profit made by selling electricity from a wind farm with ESS and one without ESS, and prove that the economic value of ESS is non-decreasing and jointly concave with respect to the storage capacity and charging/discharging capacity sizes.By the numerical results, we show that the economic value Peer-reviewed version available at Energies 2018, 11, 591; doi:10.3390/en11030591q q q q q q q q 10 30 50 70 of ESS increases but its marginal increment decreases as the size of ESS increases.In addition, we find that the economic value of ESS can be affected by the management policy, wind electricity variability, and electricity price variability.This result implies that, even with specific investment costs of ESS, the optimal size of ESS can vary depending on the locational characteristics (about wind electricity and electricity price) and the management policy of the wind farm, so there is no specific solution for the optimal size of ESS.Hence, this study could help the wind farm owners, who are considering the installation of the ESS, make their decisions.
We find that the current investment cost level of the ESS is too high to make a profit by its use, but battery technology has been growing rapidly, driven by the popularity of electric vehicles and renewable energies.Therefore, its costs are expected to decrease significantly in a few years, and then this study will become more valuable.Furthermore, even though we have evaluated the value of the ESS in terms of economic perspective, ESS is, in practice, also utilized for other objectives such as wind power quality and reliable electricity supply.Thus it would be interesting for our paper to expand the study to the optimal ESS size with multi-objectives.

Figure 2 .
Figure 2. the cases of two state variables x t and w t

Figure 7 .
Figure 7.Estimated sample mean and one standard deviation range of SMPs at each hour on each month based on 3 year historicial data in the Korean electric power system

Figure 8 .
Figure 8. Annual economic values Y(S, R) for different storage sizes S and different charging/discharging capacities R.
presents the expected annual economic value, Y(S, R), and is compared with the cost range of c c , [19.5, 65.1] ×10 6 KRW/MW/year, with different values of R for a fixed S = 1.5 MWh (c s = 10 × 10 6 KRW/MWh/year), which is a 2-dimensional cross-section of the 3-dimensional values in Figure 8.Even with the revised reference costs, the installation of ESS is not profitable with most cases.Nevertheless, by selecting R between 0.3 MW and 1.5 MW, the installation of ESS into the given wind farm can be profitable with the investment cost close to the lower bound (i.e.c c is close to 19.5 ×10 6 KRW/MW/year).In addition, notice that Y(S, R) stays at the same value when R is greater than 1.5 MW because the storage size S is set to 1.5 MWh, which indicates that R does not have to be greater than S. Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: 22 January 2018 doi:10.20944/preprints201801.0192.v1

Figure 9 .
Figure 9. Annual economic values Y(S, R) for different values of R and the range of unit cost per year c c when S = 1.5 MWh and c s = 10 × 10 6 KRW/MWh/year.
Figure 11 illustrates the effect of the management policy.It shows the difference between the expected annual economic values of the ESS, Y(S, R), under the optimal management policy and a naive management policy by varying the storage capacity size, S, with two charging/discharging capacity sizes of R, 0.3 MW and 3 MW.The naive management policy is one of the simplest policies where the ESS is charged or discharged as much as possible if the current electricity price is the minimum or maximum in our price model, respectively, and do nothing otherwise.As shown in Figure 11, the differences of Y(S, R) between two management policies are notably large with most combinations of S and R. Note that the marginal economic benefits Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: 22 January 2018 doi:10.20944/preprints201801.0192.v1

Figure 10 .
Figure 10.Annual economic values Y(S, R) for different storage sizes and the range of unit cost per year c s when R = 1.5 MW and c c = 20 × 10 6 KRW/MW.

Figure 11 .
Figure 11.Annual economic values Y(S, R) of the optimal policy vs. that of the naive policy with different storage capacity size shows the expected annual economic values of the ESS, Y(S, R), when the wind electricity model has a different mean and standard deviation.Preprints (www.preprints.org)| NOT PEER-REVIEWED | Posted: 22 January 2018 doi:10.20944/preprints201801.0192.v1 Production E(wt) (MW) Annual economic value Y(S,R) (

Figure 12 .
Figure 12.Annual economic values Y(S, R) for different mean and standard deviations of wind electricity model, w(m,h) t compares the expected annual economic values of the ESS, Y(S, R), when the electricity price model has a different standard deviation with a fixed mean, p(m,h) t = 150 KRW/kWh.It also shows results when the storage capacity size, S, is set to 1.5 MWh and charging/discharging capacity sizes, R, are set to 0.3 MW or 3 MW.

Posted: 22 January 2018 doi:10.20944/preprints201801.0192.v1 Peer
. As the storage capacity, S, and the charging/discharging Preprints (www.preprints.org)|NOT PEER-REVIEWED | -reviewed version available at Energies 2018, 11, 591; doi:10.3390/en11030591 Figure 3.The structure of optimal charging/discharging action of the ESS in the case 3, (x t , w t ) ∈ (A3).capacity,R, of the ESS are considered as the decision variables, the optimization problem to maximize the net profit can be formulated as follows.t=0δ t p t min[(w t + a t β t ) + , τC T ] • η − E p t ,w t T−1 ∑ t=0 δ t p t min(w t , τC T ) − (c s S + c c R)

preprints.org) | NOT PEER-REVIEWED | Posted: 22 January 2018 doi:10.20944/preprints201801.0192.v1
Peer-reviewed version available at Energies 2018, 11, 591; doi:10.3390/en11030591Average wind electricity generation per hour for each month where w t is the amount of wind electricity at time t, G(•) is a transformation function such as the box-cox transformation, m is the index for month during a year (m = 1 ... 12), h is the index for one hour [h, h + 1) 13)Preprints (www.t and σ (m,h) t are the sample mean and standard deviation of wind electricity generated at corresponding hour, h, on month, m.By testing several statistical tests, we conclude that the resulting z t with G(•) = √ • for the hourly data of two years can be effectively modeled by AR(1) as follows.zt = φz t−1