- freely available
- re-usable
Energies 2014, 7(5), 3304-3318; doi:10.3390/en7053304
Published: 20 May 2014
Abstract
: Accurate forecasting tools are essential in the operation of electric power systems, especially in deregulated electricity markets. Electricity price forecasting is necessary for all market participants to optimize their portfolios. In this paper we propose a hybrid method approach for short-term hourly electricity price forecasting. The paper combines statistical techniques for pre-processing of data and a multi-layer (MLP) neural network for forecasting electricity price and price spike detection. Based on statistical analysis, days are arranged into several categories. Similar days are examined by correlation significance of the historical data. Factors impacting the electricity price forecasting, including historical price factors, load factors and wind production factors are discussed. A price spike index (CWI) is defined for spike detection and forecasting. Using proposed approach we created several forecasting models of diverse model complexity. The method is validated using the European Energy Exchange (EEX) electricity price data records. Finally, results are discussed with respect to price volatility, with emphasis on the price forecasting accuracy.1. Introduction
During the last twenty years, the traditional vertically integrated electric utility structure has been deregulated and replaced by a competitive market. The deregulated power market is an auction market with market clearing prices. Companies that trade in the electricity market today make extensive use of price prediction techniques to stay competitive. Along with forecasting electricity prices, producers and traders can develop bidding strategies to maximize profits and minimize risks and allocate purchases between long term bilateral contracts and spot prices.
Electricity today is not storable in economically significant quantities and as a result electricity prices are volatile. Aside from volatility (elaborated in detail in Section 4), liquidity is another major market parameter. Market liquidity is an asset's ability to be sold without causing a significant change in the price and with minimum loss of value [1]. The essential characteristic of a liquid market is that there are available willing buyers and sellers at all times. In a non-liquid market, the accuracy of a price forecasting procedure can significantly vary depending on the position of the dominant player. Without knowing its position or the parameters affecting it, it is very challenging to forecast electricity prices in such market [2].
The European Energy Exchange (EEX) is the most important energy exchange in central Europe which provides a spot market for power derivatives and emission trading in Germany, France, Austria and Switzerland. EEX is a highly liquid market affected by domestic and regional power system factors. Due to the importance and regional influence of the price of the EEX, it is important to find a suitable model for electricity spot market price forecasting [3]. The proposed methodology will be tested on the EEX price history data. Many attempts have been made to predict electricity prices, ranging from traditional time series approaches to artificial intelligence, such as fuzzy systems and artificial neural networks (ANN) [4]. Auto regressive integrated moving average [5], dynamic regression and transfer function [6,7] and generalized auto regressive conditional heteroscedasticity [8,9] are the most widely used time series algorithms. Time series techniques exhibit good performances, however due to the use of linear modelling most of them have difficulties in predicting the hard nonlinear behaviors and rapid changes of the price signal [10–12]. ANN has been extensively used by many researches on similar problems. In problems with adequate data for ANN training and straight-forward selection of input-output samples, ANN are a powerful and flexible tool for forecasting and provide more accurate results than time series models. For example in [13] the ANN approach for weekly price forecasts has outperformed the time series ARIMA technique and the native procedure in all of the observed weeks.
Accuracy of a certain method can be evaluated by mean absolute percentage error (MAPE). Although the concept of MAPE sounds very simple and convincing, it has two major drawbacks in practical applications. If there are zero values (which can happen as EEX allows prices in the range from −500 € to 3,500 €), a division by zero will occur. On a perfect fit, however, MAPE will be zero. With regard to its upper level, MAPE has no restriction. When calculating an average MAPE for multiple time series, a few numbers in the series that have a very high MAPE might distort a comparison between averages MAPE of time series fitted with different methods [14].
Our goal is to reuse existing methods for price forecasting to create a hybrid price forecasting model which combines advantages of existing techniques to cover specificity of electricity price movements. Initially data is processed and filtered using statistical methods resulting with a data model of similar days. This model is then improved by using a multi-layer (MLP) neural network or price spike detection and forecasting. We validated our approach using the EEX electricity price history data, and evaluated our results by applying several measures of accuracy (MAPE, MAE and RMSE). In addition we emphasized the problem of price volatility by showing how price volatility affects the accuracy of each forecasting method.
The rest of the paper is organized as follows. Section 2 describes price forecasting framework. Section 3 is devoted to the proposed forecasting methodology while Section 4 describes price volatility. Section 5 provides our simulation results. Finally, Section 6 concludes the paper.
2. Price Forecasting Framework
Most of the existing studies of electricity price forecasting use only historical price and consumption data to forecast electricity prices over various time spans [3–6,8,10,13]. We found it important to include wind power production history data as it is the resource with the most volatile production ratio. During the last decade, wind power generation has experienced a powerful breakthrough due to incentives, preferential price, Kyoto Protocol commitments and technical achievement, as well as other renewable energy resources. In Europe, Nordic consumers benefit financially from the presence of Danish wind on the power market [15]. In recent years, Germany also has a high penetration of wind power (Figure 1) [16]. However, high volatility of the wind power can cause price drops on the power exchange to below zero values, further increasing price volatility. For example, the minimum price of −500.02 € has occurred on 10 April 2009 [17,18].
The proposed hybrid price forecasting model has both linear (similar day-SN) in Case 1 and nonlinear (ANN) forecasting capabilities in Case 2. In addition in Case 2 if possibility of price spikes (PS) are detected price is forecasted as price spike. Time horizon is day-ahead and the price is forecasted for every hour.
The price forecasting framework and methodology are presented in Figure 2. As model input, historical data for prices, consumption and wind production from Point Carbon web service [19] were used. Day-ahead forecasted consumption and wind production data were taken as additional input for increased performance.
Daily price history data is processed with data mining technique which defines:
Similar day types.
Relevant history horizon for similar days—H(T).
Hourly correlation time horizon—H(T)_{h}.
Day type can be divided in three categories: working days (Monday-Friday), Saturdays and Sundays/Holidays. Days before or after holidays also reveal distinct behavior, but due to the negligible influence on forecasting performance, these cases are ignored and not performed.
To define the relevant history horizon for similar days—H(T) we performed a historical price data analysis on an annual base, and have selected a significant hourly price correlation coefficient of 0.85. Our analysis showed that this correlation coefficient corresponds to a time period D = 28 days, in average (Figure 3):
Prior to performing the forecast, an hourly horizon is defined by applying Equations (1) and (2). Hourly horizon k^{+} and k^{−} (“significant neighbors” shown in green in Figure 4) are different for every forecasted price p(T)_{h} but generally two nearest neighbors always have the highest correlation. In some cases the maximum correlation coefficient between neighboring hours is 0.80. Therefore we decided to use this value as minimum correlation coefficient for defining the hourly horizon. Detailed preview of the hourly price correlations is shown in Figure 4, where correlations higher than 0.80 are marked in green.
3. Price Forecasting Methodology
Our approach of using hybrid forecasting model, based on similar-day analysis, improved by neural network and price spikes detection and forecasting is shown in Figure 2. Data set is filtered using a data mining technique described in Section II. Each hour is observed separately and then forecasted as price spike or normal price using a neural network. We defined the hourly price p(T)_{h} for day T as a cumulative value of linear and non-linear (neural network and price spike) components:
L_{h}—linear hourly component;
N_{h}—neural network hourly component;
S_{h}—hourly price spike component.
The hybrid forecasting model consists of:
3.1. Similar Days Methodology
Electricity price forecasting methods based on similar day's methodology were presented by Paras Mandal et al., in numerous works, such as [20–22]. Similar price days are selected based on a Euclidian norm. Euclidian norm with weighted factors was used in order to evaluate the similarity between forecasted days and similar days [1]. Due to a high correlation between consumption and price for observed market, these two parameters are chosen for day-ahead price forecasting [1]:
L_{t}—load for forecasted day;
P_{t}—price for forecasted day;
L_{pt}—load on similar day in past;
P_{pt}—price on similar day in past;
ΔL_{t}—the load deviation between forecasted day and similar days;
ΔP_{t}—the price deviation at time t;
ω—weighted factor is determined using least square method based on regression model constructed by using historical load and price data.
Input data for the proposed method includes consumption and wind forecast data taken from the Point Carbon web service [23] (underlying techniques used for consumption and wind forecasts are out of the scope of this paper). Similar days with realized prices are examined by consumption and wind production data mining method. The result of this process is the linear price component from Equation (2) defined as:
N—number of similar days;
δ—average hourly difference between forecasted hours.
The linear price component can be observed as independent because it represents a starting point for the forecasting model which can be improved by neural network and price spikes component.
3.2. Neural Network Architecture
Neural networks are applied widely for solving different problems which in general are difficult to solve by humans or conventional computational algorithms. In power systems the ANN's have been used to solve problems such as load forecasting, unit commitment, power system topology recognition, and safety analysis, price forecasting etc. [24]
For hourly neural network component used in our approach, a multi-layer feed-forward neural network is proposed as shown in Figure 5. This neural network is used to forecast hourly deviation from the similar days function with regards to a forecasted day. The neural network is composed of one input layer, one hidden and one output layer. Figure 6 shows how a relevant data set for forecasting the price deviation in hour h is defined. We have defined input variables for the ANN to be load forecast, wind forecast and hourly price, wind and load deviation from average values on similar days. The deviation is defined as:
${\mathrm{\Delta}}_{i}^{h+k}$—deviation from average value on similar days;
h—forecasted hour;
i ∈ 〈i,N〉—number of similar days;
i ∈ 〈k^{−}, k^{+}〉—number of similar hour before (k^{−}) and after (k^{+}) forecasted hour h.
The neural network is trained on 70% randomly chosen cases from the data set and tested on the remaining 30%.
Due to the fact that there is no efficient way for storing electric energy, all electricity produced has to be consumed forthwith. Imbalances between consumption and production lead to electricity price jumps (i.e., price spikes) which can be several times higher or lower than the regular price. One possible cause of imbalance are weather conditions which have an effect on consumption, production from renewable energy sources, unexpected outages and reductions of cross-border capacities. Some of these events like outages or capacity reductions happen rarely and cannot be predicted with confidence.
In the proposed forecasting model we categorize prices as either regular (normal) or price spikes. It is important to find how consumption and wind production affect the price spikes. An index was created which unifies consumption and wind production changes to create a signal, which detects possibility of price spikes. Consumption and wind index (CWI) proposed in our approach consists of two components where CC refers to the relative degree of forecasted consumption with the initial consumption on similar days and CW refers to forecasted wind production with the initial wind production on similar days:
C_{h}—consumption in hour h;
C_{sd}—average consumption on similar days;
W_{h}—wind production in hour h;
W_{sd}—average wind production on similar days;
N—number of similar days;
Applying the CWI to the whole data set indicates that high price deviations happen for lower value of the coefficient (Figure 7). If the error of forecasted wind production and consumption is lower than 1.0, a price spike may happen in the observed hour.
After calculating the CWI index of a possible price spike, the hourly price spike component S_{h} Equation (3) can be calculated. Since price spikes happen rarely, we experienced a lack of data to be used for the neural network calculation. Therefore S_{h} was calculated using a linear approximation with two variables; wind and consumption.
Importance of short-term price forecasting on one hand and its complexity on the other hand led researchers to propose various methods. Among these methods, there are three widely used approaches; time series models, artificial neural network (ANN) and hybrid methods.
4. Price Volatility
Volatility refers to unpredictable fluctuations of a process observed over time. In finance, volatility is a measure for variation of a price of a financial instrument over time [20]. Past volatility is derived from a time series of the past market prices. It is a criterion to study the risk associated with holding assets when there is an uncertainty in the future value of the assets.
In [25] past volatility is calculated as standard deviation of arithmetic and logarithmic return over a time window T. If p_{t} is a spot price for a commodity at the time t, arithmetic return over time period h is defined as:
When returns are small, the arithmetic and logarithmic returns are approximately equal:
Given the return values, the estimated value of past volatility can be calculated as:
σ_{h,T}—the estimated value of past volatility;
N_{0}—the number of r_{t,T} observations;
r_{hT}—the average over the time window T.
For this study volatility is calculated as a standard deviation of arithmetic return over a time window T because EEX prices can be negative. When the prices are negative, the logarithmic return cannot be calculated as in Equation (20).
It is interesting to observe the volatility fluctuation on EEX over the last eight years in dependence with traded volume. Figure 8 shows much higher volatility values occurring between 2005 and 2013. This period coincides with the growth of wind power production (Figure 1) so it can be concluded that for some time market prices were under influence of renewable energy production. It can also be concluded that price volatility depends on traded energy volume, as from 2010, with an increase in trade volume the price, the volatility is lower than in previous years.
5. Case Study
We tested our approach for electricity price forecasting on the EEX price history data from a time period 20 November 2010–20 July 2011. Data was processed in Microsoft Excel with Palisade Decision Tool and Visual Basic. Three cases were observed:
- I
Forecasting with similar days (SD);
- II
Forecasting with similar days and neural network (SD + NN);
- III
Forecasting with similar days, neural network and price spikes detection (SD + NN + PS).
Our results are presented in Table 1. below.
Model performance was evaluated with MAPE where y_{t} is a realized value at time t and f_{t} is forecasted value at time t in the time period T:
Price spikes brought outliers in results and that was the reason for including two additional error measures: MAE and RMSE. For example, in December price distribution was scattered and there were a large number of price spikes which increased MAPE:
Simulation results show that similar day's method with neural network and price spikes detection gives the best results. Price volatility is a dominant factor affecting the forecast model accuracy. In case of low price volatility, such as March 2011, simple models such as similar days gave adequate results with forecasting error similar to advanced models with neural network and price spikes detection. In case of high volatility, such as December 2010, advanced models gave better results. Therefore it is very important not only to evaluate the forecasting model against the forecast error but also to analyze the complexity of the result distribution, in this case defined by price volatility.
6. Conclusions
The main difference between the proposed model and other existing methods for short-term electricity price forecasting is in its data utilization approach. In the proposed approach, the data is pre-processed by statistical methods prior to each analysis. Forecasting results obtained by a combination of methods: similar days method (SD), neural network forecasting (NN) and price spikes (PS) in the following combinations; SD, SD + NN, SD + NN + PS were presented; each, respectably, proving to be more accurate. Price volatility has a significant influence on the forecasting results. Simple forecasting models produced satisfactory results in cases of low price volatility. Our method proved robust enough, even in cases of high price volatility.
This work has been supported in part by the European Community Seventh Framework Programme under grant No. 285939 (ACROSS). Part of this research was discussed at the 9th International Conference “European Energy Market” held in Florence, Italy on 10–12 May 2012.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Hachmeister, A. Informed Traders as Liquidity Providers: Evidence from the German Equity Market; Springer: Wiesbaden, Germany, 2007; Volume 66, p. 179. [Google Scholar]
- Andor, M.; Flinkerbusch, K.; Janssen, M.; Liebau, B.; Wobben, M. Negative Strompreise und der Varrang Erneurbarer Energien. Z. Energiewirtsch 2010, 34, 91–99. In German. [Google Scholar]
- European Energy Exchange. Available online: http://www.eex.com (accessed on 10 October 2011).
- Abraham, A.; Baikunth, N.; Mahanti, P.K. Hybrid Intelligent Systems for Stock Market Analysis. In Computational Science—ICCS 2001; Alexandrev, V.N., Dongarra, J.J., Juliano, B.A., Renner, R.S., Kenneth Tan, C.J., Eds.; Springer-Verlag: Berlin, Germany, 2001; pp. 337–345. [Google Scholar]
- Contreras, J.; Espínola, R.; Nogales, F.J.; Conejo, A.J. ARIMA models to predict next-day electricity prices. IEEE Trans. Power Syst. 2003, 18, 1014–1020. [Google Scholar]
- Nogales, F.J.; Contreras, J.; Conejo, A.J.; Espínola, R. Forecasting next-day electricity prices by time series models. IEEE Trans. Power Syst. 2002, 17, 342–348. [Google Scholar]
- Zhao, J.H.; Dong, Z.Y.; Xu, Z.; Wong, K.P. A statistical approach for interval forecasting of the electricity price. IEEE Trans. Power Syst. 2008, 23, 267–276. [Google Scholar]
- Garcia, R.C.; Contreras, J.; van Akkeren, M.; Garcia, J.B.C. A GARCH forecasting model to predict day-ahead electricity prices. IEEE Trans. Power Syst. 2005, 20, 867–874. [Google Scholar]
- Huisman, R.; Huurman, C.; Mahieu, R. Hourly Electricity Prices in Day-ahead Markets. Energy Econ. 2007, 29, 240–248. [Google Scholar]
- Pindoriya, N.M.; Singh, S.N.; Singh, S.K. An adaptive wavelet neural network-based energy price forecasting in electricity markets. IEEE Trans. Power Syst. 2008, 23, 1423–1432. [Google Scholar]
- Amjady, N.; Dareaeepour, A. Design of input vector for day-ahead price forecasting of electricity markets. Expert Syst. Appl. 2009, 36, 12281–12294. [Google Scholar]
- Unsihuay-Villa, C.; Zambroni de Souza, A.C.; Marangon-Lima, J.W.; Balestrassi, P.P. Electricity demand and spot price forecasting using evolutionary computation combined with chaotic nonlinear dynamic model. Int. J. Electr. Power Energy Syst. 2010, 32, 108–116. [Google Scholar]
- Catalão, J.P.S.; Mariano, S.J.P.S.; Mendes, V.M.F.; Ferreira, L.A.F.M. Short-term electricity prices forecasting in a competitive market: A neural network approach. Electr. Power Syst. Res. 2007, 77, 1297–1304. [Google Scholar]
- Shahidehpour, M.; Yamin, H.; Li, Z. Market Operations in Electric Power Systems: Forecasting, Scheduling, and Risk Management; Wiley-IEEE Press: New York, NY, USA, 2002; p. 552. [Google Scholar]
- Haugom, E.; Wesgaard, S.; Solibakke, P.B.; Lien, G. Realized volatility and the influence of market measure on predictability: Anaysis of Noed Pool forward electricity data. Energy Econ. 2011, 33, 1206–1215. [Google Scholar]
- Data of Historical Electricity Prices from European Energy Exchange. Available online: http://www.eex.com/en/Download (accessed on 2 July 2011).
- Zareipour, H.; Bhattacherya, K.; Canizared, C.A. Electricity market price volatility: The case of Ontario. Energy Policy 2007, 35, 4739–4748. [Google Scholar]
- Urllich, C. Realized Volatility and Price Spikes in Electricity Markets: The Influence of Observation Frequency. Available online: http://www.efa2009.org/papers/SSRN-id1342586.pdf (accessed on 13 May 2014).
- Data of Historical and Forecasted Consumption and Wind Production. Available online: http://www.pointcarbon.com/trading/pmteex/resources/downloads/latest/ (accessed on 3 July 2011).
- Širjaev, A.N. Essentials of Stochastic Finance Facts, Models, Theory; Advanced Series on Statistical Science and Applied Probability; World Scientific: Singapore, 1999; Volume 3, p. 834. [Google Scholar]
- Mandal, P.; Senjyo, T.; Urasaki, N.; Funabashi, T.; Srivastava, A.K. A novel approach to forecast electricity price for PJM using neural network and similar days method. IEEE Trans. Power Syst. 2008, 22, 2058–2065. [Google Scholar]
- Mandal, P.; Seoul, Y.U.; Senjyu, T.; Yona, A.; Park, J.W.; Srivastava, A.K. Sensitivity Analysis of Similar Days Parameters for Predicting Short-Term Electricity Price. Proceedings of the 39th North American Power Symposium (NAPS'07), Las Cruces, NM, USA, 30 September–2 October 2007; pp. 568–574.
- Effect of Wind Energy on Electricity Market Prices. Available online: http://www.ewea.org/index.php?id=1640 (accessed on 15 October 2011).
- Mileta, D.; Simic, Z.; Skok, M. Forecasting prices of electricity on HUPX. Proceedings of the 2011 8th International Conference on the European Energy Market (EEM), Zagreb, Croatia, 25–27 May 2011; pp. 204–208.
- Areekul, P.; Senjyu, T.; Toyama, H.; Yona, A. A Hybrid ARIMA and Neural Network Model for Short-Term Price Forecasting in Deregulared Market. IEEE Trans. Power Syst. 2010, 25, 524–530. [Google Scholar]
Table 1. Performances of the proposed hybrid model for hourly electricity price forecasting. |
Month | Forecasting Model | MAPE | MAE | RMSE | Volatility | Number of Price Spikes |
December-2010 | SD | 12.81% | 7.48 | 9.38 | 18.30 | 100 |
SD + NN | 8.50% | 7.32 | 10.43 | |||
SD + NN + PS | 6.79% | 6.45 | 10.42 | |||
January-2011 | SD | 11.36% | 6.00 | 7.44 | 15.95 | 86 |
SD + NN | 12.20% | 7.33 | 9.53 | |||
SD + NN + PS | 10.39% | 6.89 | 9.08 | |||
Febuary-2011 | SD | 19.12% | 9.38 | 8.91 | 14.22 | 101 |
SD + NN | 7.84% | 5.48 | 7.24 | |||
SD + NN + PS | 5.87% | 4.59 | 6.55 | |||
March-2011 | SD | 4.88% | 4.84 | 6.05 | 9.67 | 59 |
SD + NN | 3.73% | 4.01 | 5.31 | |||
SD + NN + PS | 3.62% | 3.89 | 4.51 | |||
April-2011 | SD | 10.86% | 5.69 | 6.98 | 11.73 | 119 |
SD + NN | 7.52% | 5.29 | 6.91 | |||
SD + NN + PS | 5.03% | 4.58 | 6.31 | |||
May-2011 | SD | 7.51% | 5.85 | 6.55 | 12.01 | 141 |
SD + NN | 6.96% | 5.45 | 6.10 | |||
SD + NN + PS | 6.11% | 5.12 | 5.95 | |||
June-2011 | SD | 13.05% | 5.49 | 6.41 | 13.20 | 74 |
SD + NN | 12.90% | 5.55 | 6.81 | |||
SD + NN + PS | 5.55% | 4.30 | 5.50 | |||
Average | SD | 10.63% | 5.90 | 7.14 | 14.05 | 681 |
SD + NN | 8.49% | 5.75 | 7.08 | |||
SD + NN+PS | 6.20% | 5.26 | 6.92 |
© 2014 by the authors; licensee MDPI, Basel, Switzerland.. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license ( http://creativecommons.org/licenses/by/3.0/).