# Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Background and Data Analysis Context

#### 2.1. Balancing Market Functioning

#### 2.2. Characterization and Predictors for the UK Market

## 3. Methodology

#### 3.1. Quantile Regression for Net Imbalance Volume

Define function: which can provide a standard error of a mean (m) given a:List (sample) confidence level (conf) |

Return: confidence interval based on: (m, m + conf, m − conf) |

Define: tolerance levels to search historical dataset variables:searchMin, searchMax = 0.3, 1.7 |

Call variables: Tolerance levels are multiplied by real readings of:Demand (itsd (MW)) Production (MW) Wind (MW) Solar (MW) |

For: a range of 0 to 48 settlement periods:Return: all NetImbVol of the historical dataset based on the variables between the tolerances [searchMin, searchMax]Create: a list (sample) with the returned values |

Call function for:(sample, 0.95 quantile) (sample, 0.90 quantile) (sample, 0.80 quantile) (sample, 0.05 quantile) |

#### 3.2. XGBRegressor

#### 3.3. Data Set Analysis

## 4. Results and Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Communication from the Commission to the European Parliament; The European Council. The European Economic and Social Committee and the Committee of the Regions; The European Green Deal COM/2019/640 final; Office for Official Publications of the European Communities: Brussels, Belgium; Luxembourg, 2019; p. 24. [Google Scholar]
- International Energy Agency. European Union 2020 Energy Policy Review, Paris; IEA/OECD Publications: Paris, France, 2020; p. 310. [Google Scholar]
- Horowitz, C. Paris Agreement. Int. Leg. Mater.
**2016**, 55, 740–755. [Google Scholar] [CrossRef] - European Commission. Clean Energy for All Europeans Package—European Commission. 20 October 2017. Available online: https://ec.europa.eu/energy/en/topics/energy-strategy/clean-energy-all-europeans (accessed on 26 June 2020).
- Kuo, P.H.; Huang, C.J. An electricity price forecasting model by hybrid structured deep neural networks. Sustainability
**2018**, 10, 1280. [Google Scholar] [CrossRef] [Green Version] - Sharma, V.; Srinivasan, D. A hybrid intelligent model based on recurrent neural networks and excitable dynamics for price prediction in deregulated electricity market. Eng. Appl. Artif. Intell.
**2013**, 26, 1562–1574. [Google Scholar] [CrossRef] - Aggarwal, S.K.; Saini, L.M.; Kumar, A. Electricity price forecasting in deregulated markets: A review and evaluation. Int. J. Electr. Power Energy Syst.
**2009**, 31, 13–22. [Google Scholar] [CrossRef] - Bunn, D.W. Forecasting loads and prices in competitive power markets. Proc. IEEE
**2000**, 88, 163–169. [Google Scholar] [CrossRef] - Weron, R. Electricity price forecasting: A review of the state-of-the-art with a look into the future. Int. J. Forecast.
**2014**, 30, 1030–1081. [Google Scholar] [CrossRef] [Green Version] - Niu, H.; Baldick, R.; Zhu, G. Supply function equilibrium bidding strategies with fixed forward contracts. IEEE Trans. Power Syst.
**2005**, 20, 1859–1867. [Google Scholar] [CrossRef] - Ruibal, C.M.; Mazumdar, M. Forecasting the Mean and the Variance of Electricity Prices in Deregulated Markets. IEEE Trans. Power Syst.
**2008**, 23, 25–32. [Google Scholar] [CrossRef] - Borgosz-Koczwara, M.; Weron, A.; Wyłomańska, A. Stochastic models for bidding strategies on oligopoly electricity market. Math. Methods Oper. Res.
**2009**, 69, 579–592. [Google Scholar] [CrossRef] - Chatzidimitriou, K.C.; Chrysopoulos, A.C.; Symeonidis, A.L.; Mitkas, P.A. Enhancing Agent Intelligence through Evolving Reservoir Networks for Predictions in Power Stock Markets; BT—Agents and Data Mining Interaction; Springer: Berlin/Heidelberg, Germany, 2012; pp. 228–247. [Google Scholar]
- Holmberg, P.; Newbery, D.; Ralph, D. Supply function equilibria: Step functions and continuous representations. J. Econ. Theory
**2013**, 148, 1509–1551. [Google Scholar] [CrossRef] [Green Version] - Kowalska-Pyzalska, A.; Maciejowska, K.; Suszczyński, K.; Sznajd-Weron, K.; Weron, R. Turning green: Agent-based modeling of the adoption of dynamic electricity tariffs. Energy Policy
**2014**, 72, 164–174. [Google Scholar] [CrossRef] [Green Version] - Ladjici, A.A.; Tiguercha, A.; Boudour, M. Nash Equilibrium in a two-settlement electricity market using competitive coevolutionary algorithms. Int. J. Electr. Power Energy Syst.
**2014**, 57, 148–155. [Google Scholar] [CrossRef] - Bierbrauer, M.; Trück, S.; Weron, R. Modeling electricity prices with regime switching models. Lect. Notes Comput. Sci.
**2004**, 3039, 859–867. [Google Scholar] [CrossRef] [Green Version] - Cartea, Á.; Figueroa, M.G. Pricing in Electricity Markets: A Mean Reverting Jump Diffusion Model with Seasonality. Appl. Math. Financ.
**2005**, 12, 313–335. [Google Scholar] [CrossRef] [Green Version] - Bierbrauer, M.; Menn, C.; Rachev, S.T.; Trück, S. Spot and derivative pricing in the EEX power market. J. Bank. Financ.
**2007**, 31, 3462–3485. [Google Scholar] [CrossRef] - Karakatsani, N.V.; Bunn, D.W. Forecasting electricity prices: The impact of fundamentals and time-varying coefficients. Int. J. Forecast.
**2008**, 24, 764–785. [Google Scholar] [CrossRef] - Janczura, J.; Weron, R. An empirical comparison of alternate regime-switching models for electricity spot prices. Energy Econ.
**2010**, 32, 1059–1073. [Google Scholar] [CrossRef] [Green Version] - CAlbanese; Lo, H.; Tompaidis, S. A numerical algorithm for pricing electricity derivatives for jump-diffusion processes based on continuous time lattices. Eur. J. Oper. Res.
**2012**, 222, 361–368. [Google Scholar] [CrossRef] - Weron, R.; Zator, M. Revisiting the relationship between spot and futures prices in the Nord Pool electricity market. Energy Econ.
**2014**, 44, 178–190. [Google Scholar] [CrossRef] [Green Version] - Misiorek, A.; Trueck, S.; Weron, R. Point and Interval Forecasting of Spot Electricity Prices: Linear vs. Non-Linear Time Series Models. Stud. Nonlinear Dyn. Econ.
**2006**, 10, 1–36. Available online: https://econpapers.repec.org/RePEc:bpj:sndecm:v:10:y:2006:i:3:n:2 (accessed on 27 June 2020). [CrossRef] [Green Version] - Weron, R.; Misiorek, A. Forecasting spot electricity prices: A comparison of parametric and semiparametric time series models. Int. J. Forecast.
**2008**, 24, 744–763. [Google Scholar] [CrossRef] [Green Version] - Lanne, M.; Lütkepohl, H.; Maciejowska, K. Structural vector autoregressions with Markov switching. J. Econ. Dyn. Control
**2010**, 34, 121–131. [Google Scholar] [CrossRef] [Green Version] - Weron, R. Modeling and Forecasting Electricity Loads and Prices: A Statistical Approach; Wiley: New York, NY, USA, 2006. [Google Scholar]
- Trück, S.; Weron, R.; Wolff, R. Outlier Treatment and Robust Approaches for Modeling Electricity Spot Prices. Munich Pers. RePEc Arch.
**2007**, 4711. Available online: https://mpra.ub.uni-muenchen.de/4711/ (accessed on 27 June 2020). - Boogert, A.; Dupont, D. When Supply Meets Demand: The Case of Hourly Spot Electricity Prices. IEEE Trans. Power Syst.
**2008**, 23, 389–398. [Google Scholar] [CrossRef] [Green Version] - Fanone, E.; Gamba, A.; Prokopczuk, M. The case of negative day-ahead electricity prices. Energy Econ.
**2013**, 35, 22–34. [Google Scholar] [CrossRef] - Janczura, J.; Trück, S.; Weron, R.; Wolff, R.C. Identifying spikes and seasonal components in electricity spot price data: A guide to robust modeling. Energy Econ.
**2013**, 38, 96–110. [Google Scholar] [CrossRef] [Green Version] - Shahidehpour, M.; Yamin, H.; Li, Z. Market Operations in Electric Power Systems; Wiley: New York, NY, USA, 2002; p. 552. ISBN 978-0-471-44337-7. [Google Scholar]
- Geman, H.; Roncoroni, A. Understanding the Fine Structure of Electricity Prices. J. Bus.
**2006**, 79, 1225–1261. [Google Scholar] [CrossRef] [Green Version] - Panapakidis, I.P.; Dagoumas, A.S. Day-ahead electricity price forecasting via the application of artificial neural network based models. Appl. Energy
**2016**, 172, 132–151. [Google Scholar] [CrossRef] - Keles, D.; Scelle, J.; Paraschiv, F.; Fichtner, W. Extended forecast methods for day-ahead electricity spot prices applying artificial neural networks. Appl. Energy
**2016**, 162, 218–230. [Google Scholar] [CrossRef] - Karabiber, O.A.; Xydis, G. Electricity price forecasting in the Danish day-ahead market using the TBATS, ANN and ARIMA methods. Energies
**2019**, 12, 928. [Google Scholar] [CrossRef] [Green Version] - Olsson, M.; Soder, L. Modeling Real-Time Balancing Power Market Prices Using Combined SARIMA and Markov Processes. IEEE Trans. Power Syst.
**2008**, 23, 443–450. [Google Scholar] [CrossRef] - Jónsson, T.; Pinson, P.; Nielsen, H.A.; Madsen, H. Exponential smoothing approaches for prediction in real-time electricity markets. Energies
**2014**, 7, 3710–3732. [Google Scholar] [CrossRef] [Green Version] - Kratochvíl, Š.; Bejbl, J. The system imbalance prediction. In Proceedings of the 19th International Student Conference on Electrical Engineering, Prague, Czech Republic, 14 May 2015; pp. 1–6. [Google Scholar]
- Dumas, J.; Boukas, I.; de Villena, M.M.; Mathieu, S.; Cornelusse, B. Probabilistic Forecasting of Imbalance Prices in the Belgian Context. Int. Conf. Eur. Energy Mark. EEM
**2019**, 2019. [Google Scholar] [CrossRef] [Green Version] - Brijs, T.; de Vos, K.; de Jonghe, C.; Belmans, R. Statistical analysis of negative prices in European balancing markets. Renew. Energy
**2015**, 80, 53–60. [Google Scholar] [CrossRef] [Green Version] - Ziel, F.; Steinert, R.; Husmann, S. Efficient modeling and forecasting of electricity spot prices. Energy Econ.
**2015**, 47, 98–111. [Google Scholar] [CrossRef] [Green Version] - Maciejowska, K. Fundamental and speculative shocks, what drives electricity prices? In Proceedings of the 11th International Conference on the European Energy Market (EEM14), Krakow, Poland, 28–30 May 2014; pp. 1–5. [Google Scholar] [CrossRef] [Green Version]
- Amjady, N.; Hemmati, M. Energy price forecasting—Problems and proposals for such predictions. IEEE Power Energy Mag.
**2006**, 4, 20–29. [Google Scholar] [CrossRef] - ELEXON. Available online: https://www.ELEXON.co.uk/ (accessed on 27 June 2020).
- ELEXON. Available online: https://www.ELEXON.co.uk/about/about-ELEXON/ (accessed on 27 June 2020).
- European Commission. Final Report of the Sector Inquiry on Capacity Mechanisms; Office for Official Publications of the European Communities: Brussels, Belgium; Luxembourg, 2016; p. 18. [Google Scholar]
- ELEXON Limited. Loss of Load Probability Calculation Statement; Balancing and Settlement Code Version 2.0; ELEXON Limited: London, UK, 2019; pp. 5–10. [Google Scholar]
- Breiman, L. Random Forest. Mach. Learn.
**2001**, 45, 5–32. [Google Scholar] [CrossRef] [Green Version] - Friedman, J.H. Greedy function approximation: A gradient boosting machine. Ann. Stat.
**2001**, 29, 1189–1232. [Google Scholar] [CrossRef] - Chen, T.; Guestrin, C. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD16), San Francisco, CA, USA, 13–17 August 2016; pp. 785–794. [Google Scholar]
- SKlearn. Available online: http://scikit-learn.org/stable/ (accessed on 31 October 2018).

**Figure 1.**Taxonomy of electricity price forecasting approaches based on [9].

**Figure 5.**Feature importance of each predictor for all three algorithms XGBoost, Gradient Boosting (GB) and Random Forest (RF).

**Figure 6.**Partial dependency from the NetImbVol (

**left**), Production (

**center**) and DRM_12 h predictors (

**right**).

**Figure 8.**Model price prediction mean and NetImbVol 95% quantile vs. real data for 48 SP (test on 23 June 2020).

Parameter | NetImbVol | Production | Wind | Solar | Price | Itsd | LOLP_12 h * |
---|---|---|---|---|---|---|---|

min | −1534.00 | 0.00 | 0.00 | 0.00 | −60.00 | 18,209.00 | 0.00 |

25% | −226.00 | 17,266.00 | 2475.00 | 0.00 | 27.00 | 25,559.00 | 0.00 |

Std. | 314.11 | 6738.22 | 2860.97 | 1933.91 | 20.94 | 6426.33 | 199.08 |

mean | −40.77 | 22,375.83 | 4948.92 | 1256.96 | 41.65 | 30,538.80 | 4.37 |

50% | −28.00 | 21,712.00 | 4574.00 | 13.00 | 40.00 | 29,843.00 | 0.00 |

75% | 146.00 | 26,820.50 | 7069.00 | 2050.00 | 55.00 | 34,680.50 | 0.00 |

max | 2017.00 | 44,493.00 | 14,090.00 | 9712.00 | 136.00 | 48,697.00 | 19,615.00 |

^{−6}.

Methods | HyperParameters |
---|---|

RandomForrest | {‘n_estimators’: 100, ‘min_samples_split’: 10, ‘min_samples_leaf’: 2, ‘max_features’: ‘sqrt’, ‘max_depth’: 90, ‘bootstrap’: True} |

GradientBoosting | {‘subsample’: 1, ‘n_estimators’: 642, ‘min_samples_split’: 7, ‘min_samples_leaf’: 1, ‘max_depth’: 14, ‘learning_rate’: 0.2, ‘alpha’:0.5} |

XGBossting | {‘subsample’: 0.8, ‘seed’: 578, ‘n_estimators’: 4183, ‘min_child_weight’: 7, ‘max_depth’: 119, ‘colsample_bytree’: 0.5} |

Metrics/Method | XGBoost | Gradient Boosting | Random Forest | XGBoost without LOLP (Baseline) |
---|---|---|---|---|

R^{2} | 76.8% | 78.3% | 80.4% | 68.2% |

Mean absolute error (£) | 7.89 | 7.49 | 6.97 | 9.44 |

Mean squared error (£) | 124.74 | 116.30 | 105.38 | 154.88 |

Explained variance score | 0.7702 | 0.7838 | 0.8059 | 0.7326 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Lucas, A.; Pegios, K.; Kotsakis, E.; Clarke, D.
Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression. *Energies* **2020**, *13*, 5420.
https://doi.org/10.3390/en13205420

**AMA Style**

Lucas A, Pegios K, Kotsakis E, Clarke D.
Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression. *Energies*. 2020; 13(20):5420.
https://doi.org/10.3390/en13205420

**Chicago/Turabian Style**

Lucas, Alexandre, Konstantinos Pegios, Evangelos Kotsakis, and Dan Clarke.
2020. "Price Forecasting for the Balancing Energy Market Using Machine-Learning Regression" *Energies* 13, no. 20: 5420.
https://doi.org/10.3390/en13205420