This section presents the proposed framework for accounting for uncertainties due to DG beyond the daily planning horizon in the optimal scheduling of ESSs in distribution systems. The components of the framework are described in the following subsections:
Section 3.1 formulates a multi-period optimal power flow model for a distribution system with energy storage and distributed generation. The objective function of this MPOPF model includes a novel term explicitly accounting for the expected future value of stored energy but is otherwise comparable to many of the deterministic models found in the literature and reviewed in the previous section.
Section 3.2 introduces the parameterization chosen for the expected future value function, and
Section 3.3 describes the method based on stochastic dynamic programming for determining the parameters of this value function. To determine these parameters, the framework requires stochastic models that can use measured (historic) data to generate synthetic time series representing possible realizations of future DG power output.
Section 3.4 and
Section 3.5 present relatively basic examples of such models for wind and solar power, respectively.
3.1. Multi-Period Optimal Power Flow Model for Distribution System with Energy Storage
The objective function of the multi-period optimal power flow problem considering ESS and DG can generally be stated as
where
is the operational cost for time step
and
is the future value of storing the energy
at the terminal time step
of the operational planning horizon, given the value
of the state variable underlying distributed generation at the end of the planning horizon. Here, the terms of the objective function are given in units of €, and the electricity price parameter
with units €/MWh is used to set the scale of the cost contributions of
. It is assumed that each time step has duration
. In this article, the operational cost
represents the total cost of operating the distribution system:
Minimizing corresponds to optimising the social welfare for the system. In this objective function, the first two terms are the cost and revenue associated with importing or exporting electric power, respectively, with the constraints and imposed on the fictitious import and export generators. The cost of grid losses is implicitly accounted for through these terms, as an increase in grid losses typically increases the imported power or reduces the exported power and thus increases the operational costs. The third term represents the cost associated with rationing or shedding load for all load buses . The power prices and are the prices of electric energy imported to the system and exported from the system, respectively, at time step . We regard the electricity prices to be exogenous variables. The unit cost associated with load rationing is denoted by . All the price parameters , , and are dimensionless and measured relative to the electricity price parameter so that all terms of are given in units of MW. The decision variables of the MPOPF problem include the power output for all generators (real or fictious) and the energy stored in the ESS for all time steps .
The distribution system is assumed to have distributed generation connected to buses
that satisfy the constraints
The maximal theoretical DG output at each time step is a function of , which is the value of the DG resource variable, i.e., either wind speed or solar irradiance. DG power curtailment is represented by solutions with . As is common in MPOPF models previously reported in the literature, we assume the DG time series within the planning horizon to be deterministic. However, as our objective is to consider uncertainties beyond the current planning horizon, the stochasticity of for will be accounted for in the next subsections.
Energy storage dynamics, i.e., the evolution in time of the energy stored in the ESS, is expressed by the energy balance equation
where
and
is the power charged and discharged at time step
, respectively. The total efficiency of charging and discharging the ESS, also including inverter losses etc., is denoted
and
, respectively. The amount of energy in the ESS can never be negative or above the energy capacity
:
In addition, we require that
i.e., that the amount of energy stored at the end of the planning horizon should be at least at a minimum value
. Furthermore, the ESS is subject to power capacity limits for charging and discharging,
The grid constraints we consider are the AC power flow equations as given in Reference [
51], voltage limits, and apparent power flow limits, and the power system should be within its operational limits at all times. This means we enforce the upper and lower voltage magnitude limits for all buses
,
and the upper and lower limits for apparent power for all branches
,
3.2. Expected Future Value Function for Stored Energy
The value function
denotes the expected future value of the energy stored in the ESS at the end of the planning horizon,
, given the value
of the stochastic variable underlying distributed generation. In the context of a multi-stage decision problem, with the first stage being the planning horizon
,
can be understood as the profit-to-go function and
is to be understood as a (non-positive) cost-to-go or future expected cost function. To use the analogy with hydropower scheduling [
11], the slope of this function,
corresponds to the incremental water value of a hydropower reservoir, i.e., the expected shadow price or marginal value of an additional unit of water added to the reservoir at the end of the current planning horizon. The so-called incremental water value method of hydropower scheduling [
11] accounts for the expected future value through the marginal value in Equation (10). In that case, the stochastic variable
underlying hydropower generation would typically correspond to reservoir inflow. In our formulation, we assume that a single state variable
for each planning horizon can be used to describe the stochasticity of the DG resource variables
(wind speed or solar irradiance time series) in the next planning horizon. In this article, we limit ourselves to a distribution system with one ESS, but the framework can be extended to consider multiple ESSs similarly to how multiple reservoirs are treated in hydropower scheduling [
10,
11].
Explicitly including an expected future value term in the objective function may avoid myopic operation such as unnecessarily depleting the ESS at the end of each planning horizon when the energy could be more valuable in the next planning horizon. Taking into account DG stochasticity through an explicit dependence on could also avoid unnecessarily filling the ESS in the current planning horizon when large DG output is expected in the next planning horizon; in that case, it could be better to have more capacity available to store the excess energy and then export it during hours of higher electricity prices. Thus, the inclusion of a value function can be understood as a control measure for ensuring (on average) optimal operation also beyond the current planning horizon.
The functional form of the value function is, in general, unknown and must be determined for each problem. Based on Reference [
9] we here assume a simple quadratic functional form for the value function and determine its coefficients through the method described below. Conventional approaches to representing the value function include piecewise linear functions formed by generating Benders cuts [
10,
52], but in Reference [
9] it was shown that the value function was well approximated by a quadratic function in a similar case as those considered here. Furthermore, this functional form has the additional advantage that it is simple to implement and interpret. Thus, we propose to parameterize the value function in the following way as a quadratic function of stored energy
:
In this parameterization, the dependence of the future expected value on the distributed generation in the current planning horizon is contained in the parameters and . For a given value of , the parameter can be interpreted as the average unit value of stored energy of a fully charged ESS, as . The parameter determines the curvature of the value function for a given value of , with giving a linear function where the value of the stored energy is proportional to the amount of stored energy. The parameter needs to fulfil to avoid becoming non-concave and the term in the objective function becoming non-convex.
With this parameterization, the marginal value of stored energy defined in Equation (10) takes the form
For a linear value function with , the marginal value of stored energy equals for all values of , whereas for the marginal value of stored energy decreases as increases. A decreasing marginal value could represent that higher storage level increases the chance for spillage of distributed energy resources, for example, because of curtailment of DG due to grid constraints. A similar trend is also generally valid in the case of market operation in unconstrained grids: Since the highest price variations during the planning horizon are exploited first, the marginal value of moving a unit of energy from one time step to another is decreasing as more energy is added to the ESS.
3.3. Determining the Value Function
To present how we set the end-value of stored energy, i.e., determine the value function, we consider in this section the multi-period OPF problem described above as a multi-stage decision problem, where each planning horizon corresponds to one stage. The objective is to maximize social welfare when operating the distribution system, not only for the current planning horizon
, but also for multiple planning horizons (
) into the future:
Here,
corresponds to the term of the objective function for time step
and planning horizon
. For a given planning horizon
, the objective function for the first-stage problem can be written as
where
is the future value of the energy
stored at the beginning of planning horizon
. Solving the multi-stage decision problem amounts to determining
, which corresponds to an optimal scheduling policy for each planning horizon.
For the first-stage problem, is known, and the problem is to determine the schedule given the value of stored energy at the end of the planning horizon, . The second-stage problem is to determine the schedule for planning horizon , , given a realization of the uncertain DG resource variables () and given the initial stored energy determined by solving the first-stage problem. In this two-stage decision problem, we choose as state variables the stored energy and in addition a DG state variable describing the stochasticity of the underlying DG resource variables. These state variables are being passed from one stage to the next, meaning that and that is determined by and by a stochastic process governing the transition between the stages.
Our method builds upon a stochastic dynamic programming approach as presented in Reference [
11] in the context of hydropower scheduling. The main principle here is solving a recursive Bellman equation on a form similar to
Here
is the optimal objective value for planning horizon
, including also the future value term. The expectation is taken over possible realizations of
, which represents stochastic variables for the next planning horizon
that are assumed to be known (realized) in planning horizon
but uncertain in the current planning horizon
. We require that the stochastic processes underlying the variables
are stationary and do not depend on
. For our application, this condition is fulfilled when considering sufficiently short time periods, e.g., within the same month or season. This needs to be checked when pre-processing historic energy resource time series used to generate
. Since the physical system is static between different planning horizons, the optimal scheduling policy is also stationary. For stationary processes, one can regard the multi-stage decision problem as a static infinite-horizon problem with
. Solving this infinite-horizon problem by backwards recursion corresponds to iteratively solving a series of identical two-stage decision problem, as illustrated in
Figure 1: For each next iteration, one inserts for
the function
found in the previous iteration. For the first iteration, one needs to make a guess at an initial value function, in our case
. This iteration procedure is repeated until reaching convergence, i.e.,
. with sufficient accuracy. The value function determined through this iteration procedure can then be used in the objective function (Equation (1)) to represent the optimal scheduling policy for any planning period within the time period (e.g., month or season) it is determined for.
Similarly to the incremental water value method described in Reference [
11], we construct the future value function to use in the next iteration using the marginal values of stored energy
instead of the values
from Equation (15) directly. Assuming a concave form of the value function as in Equation (11), one can construct a value function to use for the first-stage problem from dual values obtained from the second-stage problem. This approach is analogous to constructing Benders cuts of the decomposition of stochastic multi-stage optimization problems [
52]. From the optimal solution for the second-stage planning horizon, we extract the dual value
corresponding to the energy balance constraint Equation (4) for
:
This corresponds to the marginal value of energy stored at the beginning of planning horizon . The dual value as a function of the initial amount of stored energy is evaluated for the discrete set of values for a number of realizations of , and parameter values and to use for the next iteration are estimated by fitting the linear function of Equation (12) to the data. This is repeated for different values of , and the iterations proceed until acceptable convergence is reached for and . The procedure is described in the Algorithm 1 below.
Algorithm 1: Determining Value Function for Stored Energy |
Input: Grid data for the distribution system model in Section 3.1, historic DG resource (wind speed or solar irradiance) time series data, selection of a discrete set Sx of DG state variable values ; selection of a discrete set of initial stored energy values Output: Estimates of value function parameters and for all DG state variables 1: Initialize value function parameters , for all 2: Generate synthetic time series yk for the stochastic DG resource variables for each value of 3: while and not converged for all 4: for do 5: Set DG state variable to the th value in 6: for do 7: Set initial stored energy to the th value in 8: for do 9: Use DG resource time series yk for state variable value 10: Solve second-stage problem for planning horizon () with value function parameterized by and for initial stored energy and DG resource time series yk 11: Evaluate dual value π(x, E0,p+1) 12: end for 13: end for 14: Fit dual values to a linear function of 15: Determine updated values of and 16: end for 17: end while |
The methods for generating synthetic time series representing possible realizations of stochastic DG variables in the next planning horizon
given the state variable
will be made more concrete in
Section 3.4 and
Section 3.5 for wind power and solar PV, respectively.
3.4. Modeling Stochasticity of Wind Power Generation
In this section, we consider the case of wind power generation where the stochastic variables in the next planning horizon are the wind speed time series . The mathematical expectation in Equation (15) is therefore taken over possible realizations of this wind speed time series. In determining the value of stored energy at the end of the current planning horizon, we want to account for the time correlations between the wind speeds at the end of the current planning horizon and at the beginning of the next planning horizon . High values of are correlated with high wind speeds and thus high wind power output in the next planning horizon. This, in turn, increases the probability of wind power curtailment if the ESS does not have the capacity to accommodate the part of this distributed generation that cannot be exported from the grid. One would therefore expect that the future value of stored energy at should decrease with increasing , which motivates including a state variable in Equation (15).
As in Reference [
9], we use the terminal wind speed as state variable, i.e.,
and capture the time correlation and stochasticity in wind speed by generating synthetic wind speed time series for planning horizon
using a discrete-state Markov chain model. Wind speed time series do not generally satisfy the Markov property, but Markov chain models may reproduce the autocorrelation of historic wind speed time series with acceptable accuracy if the timesteps are longer than around 40 min [
53]. This condition is satisfied in our case.
Due to the potentially strong seasonal patterns in wind speed variation, transition matrices
are first constructed separately for historic data for each month or season (e.g., combining data from multiple months if they represent similar wind speed statistics). An element in these transition matrices is the estimated probability that the wind speed in the next time step (the stochastic variable
) has the value
, given that the wind speed in the current time step has the value
. Next, for each value of
, synthetic time series
for
are generated using the transition matrix for the season and initializing the Markov chain from
. This model for the stochasticity and time correlations of wind speed is illustrated schematically in
Figure 2 for a few possible wind speed realizations
. Note that the purpose in this work is emphatically not to exactly represent accurate wind speed forecasts, but rather to generate a representative set of realizations of future (
) stochastic wind power output, capturing time correlations sufficiently accurately for the estimation of the value the energy stored at
. The time series for wind power output are generated based on the time series for the wind speed
, a power curve function
, and the rated power
of the wind turbine at bus
, i.e.,
3.5. Modeling Stochasticity of Solar PV Power Generation
To model the stochasticity and time dependence of PV generation for the purpose of estimating the future value of stored energy, we have chosen a Markov chain model with regime-switching for the cloud cover (clearness) conditions, inspired by References [
38,
54], respectively. Following Reference [
54], we assume that the stochasticity of the actual (ground-level) solar irradiance
follows a regime-switching process in which periods of the time series for
wt can be classified as belonging to one of three clearness regimes
: overcast
, partly cloudy
or sunny
. For time series for
within each regime, we have, following Reference [
38], assumed that the stochasticity can be described by a Markov chain model for the clearness
, defined as
together with a deterministic time series
for the expected irradiance given sunny conditions and no cloud cover. Note that in some other works [
54], a clear-sky-index
is used instead of the clearness
to describe the relationship between the actual and expected irradiance.
To generate synthetic time series for
based on historic data for
, we first find the time series for the expected irradiance
over a period of a day. Instead of calculating the deterministic
profile theoretically based on geometry, latitude, date, etc., we estimate it empirically using the simple model as proposed in Reference [
55]:
First, for each month of the year and each time step of the day from sunrise () to sunset (), the highest observed value in the irradiance data set is found. Sunrise and sunset for the month are determined empirically as the first and last time steps, respectively, for which all days in the data set have nonzero irradiance. Next, Equation (19) is fitted to the time series to determine the parameters and . For the purposes of this study, this is found to be a robust and sufficiently accurate approach given that there are no substantial systematic shading effects.
A procedure based on that proposed in Reference [
54] is used to classify the cloud cover conditions (clearness regimes) for each day of the historic irradiance data set: A day
is classified as sunny if the normalized error
is below a certain threshold,
If not, one calculates
and classifies the day as overcast if
and
. Otherwise, the day is classified as partly cloudy. Time series for the clearness regime value
are thus created for the historic irradiance data set for all sets of subsequent days, and these time series are used to construct
transition matrices
for the regime-switching process. Here,
denotes the stochastic variable for the clearness regime for day
.
Transition matrices
for the Markov chain model for clearness within each day are constructed separately for each of the three clearness regimes. Here,
denotes the stochastic variable for the clearness for time step
. As in Reference [
38], clearness values
are discretized in a set of
nclr values
. As the daily expected irradiance time series
may vary substantially from one month to the next, transition matrices for the regime-switching process between days and the Markov process for clearness within days are estimated separately for each month.
As the state variable underlying the PV generation in planning horizon (day)
, we use the clearness regime during this planning horizon, i.e.,
. To generate a set of irradiance time series representing possible realizations of uncertainty in the next planning horizon
within a given month and given a current clearness regime
, we (1) use the transition matrix
for the regime-switching process estimated for the month to draw a pseudo-random regime value
, (2) draw a pseudo-random clearness value
for the sunrise time step of the next day from the clearness probability distribution for clearness regime
and the given season, (3) use this as the initial value in a Markov chain
generated using the clearness transition matrix
for regime
, and 4) calculate the synthetic irradiance time series
from
with
as estimated for the given month. This representation of the stochasticity and time correlations of the energy resource variables underlying PV generation is illustrated schematically in
Figure 3 for a few possible realizations
and
for different clearness regimes.
Finally, to calculate the PV power output at bus
in time step
from the solar irradiance
, we use the simple model [
56]
Here,
is the total efficiency of the PV systems, and
is the total PV panel area connected to bus
. Such a simple model is sufficient for the purpose of this article, but for more detailed models we refer to References [
56,
57].