This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (
The optimal design of offering strategies for wind power producers is commonly based on unconditional (and, hence, constant) expectation values for prices in realtime markets, directly defining their loss function in a stochastic optimization framework. This is why it may certainly be advantageous to account for the seasonal and dynamic behavior of such prices, hence translating to timevarying loss functions. With that objective in mind, forecasting approaches relying on simple models that accommodate the seasonal and dynamic nature of realtime prices are derived and analyzed. These are all based on the wellknown Holt–Winters model with a daily seasonal cycle, either in its conventional form or conditioned upon exogenous variables, such as: (i) dayahead price; (ii) system load; and (iii) wind power penetration. The superiority of the proposed approach over a number of common benchmarks is subsequently demonstrated through an empirical investigation for the Nord Pool, mimicking practical forecasting for a threeyear period over 2008–2011.
In parallel with the rapid growth in electricity generation by renewable energy sources, producers are increasingly urged to sell their generation though the market. This has triggered the interest of both researchers and practitioners in proposing optimal bidding strategies for renewable energy producers as balance responsible parties (BRPs) in deregulated electricity markets. Especially, wind power appears to have a leading role among recently emerging renewable energy sources. The wind power producer's point of view will also be considered here for the sake of an example. Other market participants may be interested in the forecasting of regulation market quantities for optimization of their market participation strategies. Wind power production is a stochastic process with limited predictability on a dayahead basis. Accordingly, the offering strategies of wind power producers are derived in a stochastic optimization framework. The resulting optimization problems aim at minimizing the expected costs induced by their (involuntary) participation in the realtime imbalance market (also referred to as regulation market), potentially also accounting for risk aversion [
The unit regulation costs (per MWh of imbalance) in the realtime market are dynamic and asymmetric [
The objective of this paper is to rigorously model and predict imbalance penalties, at the lead times of interest, when trading wind power in a dayahead market, in an exponential smoothing framework. The very nature of regulation markets makes that realtime electricity price modeling ought to differ substantially from conventional time series modeling approaches. Excessive volatility and skewness aside, antigaming policies have in many cases caused these prices to exhibit unique dynamic features. Firstly, the widely adopted dual pricing scheme for regulating power (which implies different pricing for producers in surplus and deficit) induces a regimeswitching behavior in the prices, driven by the net balance of the system. Additionally, some markets have realtime prices that are bounded by the dayahead price (e.g., the Danish and the Spanish ones) or capped in another manner (e.g., the PennsylvaniaNew JerseyMaryland—PJM, and the New England markets in the USA) to discourage gaming even further. This feature magnifies the state dependency of the prices, since they either share characteristics with the dayahead prices or have their own unique dynamics. Furthermore, this policy arguably inflates the volatility in the realtime market through reduced supply.
Conditioning models for the realtime prices on the net balance of the whole system is paramount for capturing relevant price dynamics. This, in turn, makes popular time series models, like the seasonal AutoRegressive Moving Average (ARMA) model used in [
The idea behind the prediction approach introduced here is to take advantage of the formulation of imbalance costs as the product of two variables: (i) a categorical one representing the sign of the net system balance; and (ii) a penalty part describing the magnitude of imbalance penalties when active. Combining predictions for these two variables by applying the law of total expectation then yields the expected imbalance costs. Both variables are modeled using Holt–Winters models [
The remainder of the paper is structured as follows: Section 2 gives an overview of the empirical context of the paper and describes the data used for the analysis. Afterwards, the necessary definitions and data analysis are presented in Section 3. The models and parameter estimation procedures are given in Section 4, followed by the results of our empirical investigation in Section 5. Finally, concluding remarks are gathered in Section 6.
Dayahead electricity markets are the result of a general movement around the world towards deregulation in the electricity industry over the past two decades. Although the actual implementation varies between countries and regions, the basic functionalities remain similar among those who have adopted these policies. Even though emphasis is placed here on the Nord Pool electricity market in Scandinavia, the proposed models and forecasting methodologies may certainly be appropriate for other electricity markets (with some variations in forecast performance). After first presenting some of the specifics of the Nord Pool, the dataset available for our modeling and forecasting study is described in detail.
The Danish electricity market is a part of the Nord Pool market, which is the world's first multinational electricity market. It was founded by Statnett SF and Svenska Kraftnät (the Norwegian and Swedish Transmission System Operators (TSOs), respectively) in 1996. Since then, it has been gradually expanded to encompass Denmark, Finland and Estonia, as well. In 2002, physical exchange activities were transferred to a separate company, Nord Pool Spot ASA, which operates the markets. It is jointly owned by Statnett SF, Svenska Kraftnät (30% each), Energinet.dk and Fingrid Oy (20% each). Nord Pool's market share is one of the highest in the world with 72% of the total Nordic consumption traded on its markets in 2009 [
The market region is divided into 10 price areas, bordered by bottlenecks in the transmission grid and with internal transmission capacity practically infinite. Denmark comprises two of these areas, one on either side of the Storebælt channel.
Nord Pool runs two different markets for the physical exchange of electricity, Elspot and Elbas. Elspot is a dayahead market with gate closure at noon on the day before delivery. Here, producers and retailers bid for the sale and purchase of electricity in hourly intervals. Once the bids are collected, a system price is determined as the intersection between the aggregated supply and demand curves. The system price is the price at which the physical exchange is settled during hours where transmission across the grid bottlenecks does not reach capacity. Besides, the system price serves as a reference price for almost all financial derivatives linked to Nord Pool. In case the desired transmission across price area borders reaches its capacity, multiple area prices are defined. An area price is determined by the same procedure as the system price, though only considering the bids from the particular area, and the full usage of the connections to the surrounding areas as priceindependent bids. Elbas is opened for trading once Elspot prices have been published and has gate closure one hour prior to delivery. Elbas is a bidask market where bids for either the purchase or sale of electricity are placed. The bids are prioritized according to price and submission time. They are settled at the bid price. The acceptance of a bid across price areas is subject to capacity availability. Almost the entire exchange takes place on Elspot, which is responsible for more than 99% of the total exchanged volume each year. A more detailed description of the market environment and price settlement is provided in [
In addition, the national TSOs each run a realtime market in their countries for balancing the transmission grid. Markets for supplier imbalances and retailer imbalances are run separately and with different pricing schemes. For the retailers, a single price is defined at which those short of their contracted volume buy the surplus of those having overestimated their consumption at the time. In contrast, two prices are defined on the suppliers' market. The producers short of their contracts buy their deficits at the upregulation price, while those overproducing sell their surplus at the downregulation price. The regulation prices are bounded by the Elspot prices, so that the upregulation price can never deceed it and the downregulation price can never exceed it. Furthermore, imbalances that negate the total system imbalance are not penalized. This implies that at any given time, either the upregulation or the downregulation price is equal to the spot price.
The data for the empirical investigation covers the period from November, 2008, to December, 2011. The dataset includes hourly spot and regulation prices for the Western Danish price area of Elspot (DK1) along with predicted system load, wind power production and spot prices for the same area. All forecasts are issued before noon on the day before delivery. They also have an hourly temporal resolution. The observed prices and the load forecasts are publicly available on the website of Nord Pool (
The purpose of the data analysis that follows is to establish the desirable features of the models for imbalance costs and to identify relevant exogenous variables. Only results that appeared to deserve further inspection are presented.
Let
[
The regulatory framework of Nord Pool implies that the following conditions are fulfilled at all times:
For the system operator, these states are not necessarily the same, since the system operatoris working with a higher temporal resolution with the actual physical balance of the system. Such an actual physical balance can actually change sign within the hour and is seldom precisely zero. The hours without regulation thus comprise both hours of no physical regulation and hours where the balancing power has been inexpensive enough not to prompt a penalty for imbalances. However, since the producer's main objective is to maximize his revenue, regardless of the exact physical balance of the system, it is primarily price differences that are relevant to him. Consequently, the following modeling efforts are all done in terms of price differences and not of physical imbalance.
By applying the law of total expectation,
Let us support a bit more of our decision to divide the problem in such a way. Firstly, it was found that the statistical models we designed were better at mimicking the actual behavior of observed penalties, for both down and upregulation cases. In parallel, such a division made it easier to incorporate exogenous effects into the models. For instance, the potential effect of predicted wind power generation on penalties is to differ for down and upregulation cases, since the characteristics of forecasting errors, which then induce imbalances, are known to be asymmetric [
From a modeling perspective, working with penalties, in contrast to actual prices, yields advantages, as well. Indeed, such penalties have a constant lower boundary at zero, which, to some extent, can mitigate high positive residuals. Most importantly though, the penalty forecasts are applicable at all horizons, both dayahead and intraday. Combined with the current available information about spot prices, they form a prediction for the regulation prices. Finally, a model for the penalties is more versatile in terms of applications. When, for instance, the strategy in [
For estimation and analysis purposes, three additional variables are defined. Following standard estimation procedures for a multinomial regression model,
For the penalties, let:
The penalty series,
Cleaned versions of the series are shown in the bottom row of the same figure. They reveal considerable timevariations in the first and second order moments of the penalties. The mean constantly fluctuates, and volatile periods with rather frequent price spikes are followed by periods of relative tranquility.
Let
However, as illustrated by
For both values of λ, the observed series deviate considerably more from the empirical probabilities and seems to drift distinctively away from them over shorter periods. The plots therefore provide a clear indication that greater knowledge about the next day's imbalance sign can be obtained by considering more dynamic approaches to the modeling of such probabilities.
Mainly driven by the daily and weekly cycles of consumption, electricity prices are generally known to have the same seasonal cycles. It therefore seems natural to examine the seasonalities of
Penalties have an obvious diurnal seasonality, where upregulation penalties are generally above average during the day and below average during the night, while downregulation penalties exhibit the opposite pattern. Given the high share of hours where
The topleft plot in
The topleft plot indicates a significant impact of predicted wind power penetration and imbalance sign. besides, the topright plot shows that both penalties are significantly affected by the predicted spot price. Even though some of the middle points are not so different from their neighbors, there are nonnegligible differences between points that are located further apart. Similar conclusions can be drawn for the impact of predicted load on upregulation penalties and for the impact of predicted wind power penetration on downregulation penalties. The predicted load, however, does not seem to have an influence on downregulation penalties. Likewise, upregulation penalties seem to be merely affected by predicted wind power penetration.
The previously demonstrated dynamic and seasonal variations of
The HoltWinters model (HW model) is a pragmatic approach to model seasonal time series. The standard formulation of the HW model, suitable for time series with a single seasonal pattern, was introduced by [
For a stochastic process,
Similarly, a multiplicative HW model for
By defining the one step prediction error as:
In compliance with the common formulation of the HW model in the literature,
Should a single seasonal pattern be sufficient,
In order to account for the effects of external variables, e.g., predicted spot price, wind power penetration and load, the previously described HW models can be conditioned on exogenous variables. Let
The robust conditional HW model thus consists of
Similarly, the updating scheme for the
For a value of
Although, in principle, the number of variables used to condition upon could be infinity, one should be cautious in doing so, due to the risk of sparse updates in the points located close to the edges of the multidimensional space.
Let
Due to the binary nature of the
After the updating and forecasting steps, the inverse logittransformation is applied for deriving the posterior probabilities for the imbalance sign,
For the case of regulation penalties, both the multiplicative and the additive formulations of the HW models are tested. The estimation procedure is the same for both models. An estimate of
In the estimation process, the practical situation of dayahead market participation is mimicked. More specifically, the models are fit in the context of forecasts made at 11:00 on the day before realization. Therefore, estimates for the hour between 00:00 and 01:00 are found using
Various versions of the model described in the previous section are fit to the data in order to conclude on the their most appropriate structure. The first 14 months of the data set (
Exponential smoothing,
Single seasonal HWmodel with a daily period,
Single seasonal HWmodel with a weekly period,
Double seasonal HWmodel with daily and weekly periods,
Unconditional models are fitted for all three series, as well as conditional ones. For the imbalance penalties, all possible combinations of exogenous variables,
For all models, only one set of parameters is estimated across all lead times and fitting points. Although this along with the somewhat arbitrarily chosen fitting points might lead to suboptimal results for individual lead times and fitting points, the obtained results are decisive enough to serve the objective of this paper. Should the model be implemented in practice, however, the extra effort of estimating more locally optimal parameters would most certainly be worthwhile.
For a more intuitive comparison of the models, also for different periods, the discrete ranked probability skill score (RPSS
The estimated parameters along with the corresponding RPSS
First and foremost, the score values in
In
For both models and periods, the predictions cannot be deemed probabilistically unreliable at the 5% significance level. Both unconditional and conditional models appear to perform similarly overall. Hence, it must be concluded that the forecast wind power penetration has no significant impact on the imbalance sign; at least, not when defined in terms of price differences and under such a family of models.
The reason for the lack of improved forecasting skill most likely lies in the time variation of imbalance sign probabilities; both periodical variations and drift. Since they are not taken into account in
As a final remark, it should be noted that the segmentation of predicted probabilities implies that their position in the horizontal dimension is subject to uncertainty just as their vertical position. Especially this uncertainty could be large for the bins, including the highest and the lowest probabilities, as the distribution of elements in these bins is likely to be skewed, positively and negatively, respectively. This property could, however, only inflate the reliability uncertainty and, thus, cannot impact our conclusions. Although an interesting problem, the construction of these twodimensional consistency bars is beyond the scope of this paper.
The imbalance penalty forecasts are evaluated in terms of the root mean square error (RMSE),
The best performing models are the conditional ones, more precisely those involving conditioning upon predicted spot price. Moreover, the models include either a mean term only or both mean and daily seasonal terms. However, none of these models performs particularly well: their residual RMSE is only slightly less than the series standard deviation. In light of the optimal smoothing parameters, which yield long model memory, this is not surprising.
The long memory, combined with the high value of
Finally, the models' ability to provide optimal quantile values for the design of optimal wind power offering strategies is examined. Now, recall that the bid that optimizes the hourly expected revenue of a price taking wind power producer is the
The market design implies that the realization of
In order to evaluate the quality of
Our work has shown that the analysis of realtime electricity markets can significantly improve by building models aiming to described shortterm variations in market dynamics. This may be beneficial in both the design of offering strategies and in the appraisal of potential profitability if gaming electricity markets. Despite the fact that our empirical results are somewhat area specific, a number of relevant similarities between the specifics of the Nord Pool and of other electricity markets suggests that similar results would be obtained for other markets, as well.
Our results were derived in a framework tailored for the dayahead offering of wind power, with its specific lead times (between 13 and 37 hours ahead). Even though conditioning models upon predicted wind power penetration did not result in significant improvements, the conditional Holt–Winters framework is likely to be more relevant for other variables and for shorter lead times. For instance, one interesting aspect to investigate in the context of shortterm prediction would be to include updated wind power and load forecasts when they become available.
Although the forecasting skill of the proposed models might seem low to some readers, one has to bare in mind that the process it describes is mainly driven by errors from welltuned forecasting models and unforeseeable events, like plant malfunctions. Consequently, the observed forecasting quality is, in our opinion, quite satisfactory. From the market operator's point of view, however, the results of this paper might be worrying, since they hint that profitable gaming on the electricity market is possible. Thus, it is clear that antigaming policies have to be revised if the desire is to maintain a gamingfree market. Given the current carbon emission curbing targets in various parts of the world, however, the potential benefits of gaming for producers of and systems highly penetrated with electricity by renewable sources have to be considered while structuring such reforms.
The models presented in this paper are adapted to the purpose of facilitating the bidding strategy in the spirit of those presented in [
Pierre Pinson is partly supported by the Danish Council for Strategic Research through the project “5sFuture Electricity Markets”, No. 12132636/DSF. Two reviewers are acknowledged for their comments on an earlier version of this manuscript.
The authors declare no conflict of interest.
Time series plot of the down and upregulation penalties (
Exponentially smoothed state probabilities of the observed
The intraday and intraweek variations (
The empirical frequency of each imbalance sign as a function of predicted wind power penetration (
Stacked bar plot showing the posterior imbalance sign probabilities estimated at 10:00 on 12 November 2010, for November 13. The most likely imbalance sign and the observed one are marked.
Reliability diagrams for both unconditional (
The empirical probabilities of
↓  ∼  ↑ 

39.00%  29.30%  31.70% 
Estimated parameters and RPSS

 

Unconditional  I  —  0.0018  —  —  0.5063  0.4796 
II  —  0.0114  0.1161  —  0.5154  0.4815  
III  —  0.0188  —  0.0838  0.5105  0.4778  
IV  —  0.0126  0.1174  0.0626  0.5152  0.4806  
 
Conditional  I  0.9365  0.0022  —  —  0.5052  0.4811 
II  0.9629  0.0090  0.1541  —  0.5141  0.4816  
III  0.8837  0.0203  —  0.0924  0.5072  0.4778  
IV  0.9886  0.0104  0.1460  0.0608  0.5140  0.4808 
Standard deviation of the down and upregulation penalties.
Training  109.81  97.56 
Test  135.85  118.73 
The estimated model parameters along with performance measures for the best performing upregulation penalty models. Insample measures are for the training set, and outofsample ones relate to the test set.

 

0.2647  793  0.0082  —  92.68  0.0974  116.93  0.0301  
 
Additive  0.5372  625  0.0028  0.0447  91.93  0.1120  116.87  0.0311  
0.5223  701  0.0030  0.0533  92.87  0.0938  114.47  0.0705  
 
Multiplicative  0.4252  695  0.0050  0.0202  93.57  0.0801  117.13  0.0268 
The estimated model parameters along with performance measures for the best performing downregulation penalty models. Insample measures are for the training set, and outofsample ones relate to the test set.

 

—  —  1197  0.0040  —  108.46  0.0243  133.07  0.0405  
 
Additive  —  —  599  0.0022  0.0465  107.22  0.0466  132.35  0.0508 
0.9634  74  0.0005  0.1419  110.18  −0.0067  133.15  0.0393  
0.4491  584  0.0007  0.1523  106.52  0.0590  129.24  0.0948  
0.6975  611  0.0006  0.0936  107.30  0.0452  131.04  0.0695  
 
—  —  600  0.0023  0.0451  106.51  0.0592  132.10  0.0544  
Multiplicative  0.3768  574  0.0418  0.0587  106.38  0.0614  129.59  0.0900 