Probabilistic Electricity Price Forecasting Models by Aggregation of Competitive Predictors

: This article presents original probabilistic price forecasting meta-models (PPFMCP models), by aggregation of competitive predictors, for day-ahead hourly probabilistic price forecasting. The best twenty predictors of the EEM2016 EPF competition are used to create ensembles of hourly spot price forecasts. For each hour, the parameter values of the probability density function (PDF) of a Beta distribution for the output variable (hourly price) can be directly obtained from the expected and variance values associated to the ensemble for such hour, using three aggregation strategies of predictor forecasts corresponding to three PPFMCP models. A Reliability Indicator ( RI ) and a Loss function Indicator ( LI ) are also introduced to give a measure of uncertainty of probabilistic price forecasts. The three PPFMCP models were satisfactorily applied to the real-world case study of the Iberian Electricity Market (MIBEL). Results from PPFMCP models showed that PPFMCP model 2, which uses aggregation by weight values according to daily ranks of predictors, was the best probabilistic meta-model from a point of view of mean absolute errors, as well as of RI and LI . PPFMCP model 1, which uses the averaging of predictor forecasts, was the second best meta-model. PPFMCP models allow evaluations of risk decisions based on the price to be made.


Introduction
The dynamic nature of most of the natural or artificial systems and the human necessity to ensure their correct performance in the future have led to the development of forecasting techniques. A decision maker needs forecasts of his/her variables of interest if there is uncertainty about their future values. Forecasting involves the use of available information to predict the likely value of these variables. In the electric power sector, the implantation of liberalized electricity markets has changed the behaviour of producers and costumers. In the past, before the new regulation, power producers had the minimal financial risks, as customers were not able to acquire freely electricity. Now, with a competitive market agents can sell or buy energy like any other commodity.
On the other hand, the growing penetration of renewable energy generation into the power systems has increased the uncertainties about the short-term generation values. These uncertainties are caused by the intermittent nature of wind, solar or hydro based generation, and their associated risks are also influenced by the own uncertainties in the demand of customers. In a competitive electricity market, agents' bidding strategies are very dependent of the expected electricity prices.
Electricity markets show very volatile and dynamic behaviours as they depend on a large number of variables, such as electricity demands, prices of the raw materials used in thermal generation, the • Historical simulation, which uses sample quantiles from the empirical distribution of one step ahead prediction error (difference between the real value and that provided by SPF models). Different kinds of SPF models have been used in historical simulation, from autoregressive models [7,32] to semiparametric models [32]. • Distribution-based probabilistic forecasts, which basically assume a probability density function (PDF) for the prediction errors; the standard deviation of the error density is calculated and the lower and upper bounds of the PIs are adjusted to selected quantiles of the distribution. Gaussian and t distributions (normal and skew) have been used to calculate PIs [33,34]. • Bootstrapped PIs, which use bootstrap resampling [35] to develop the models, and the PIs are calculated from the quantiles of the bootstrap sample [36,37]. • Quantile regression averaging models, which apply quantile regression to a set of spot forecasts from different predictors [38][39][40].
• Daily market of MIBEL, for the day-ahead, where the corresponding prices are determined by the Iberian Energy Market Operator-Spanish Division (OMIE) [44] for each of the 24 h of the following day.

•
Markets for term contracts, as well as for average daily, weekly, monthly and quarterly periods, managed by the Iberian Energy Market Operator-Portuguese Division (OMIP) [45], which offers different types of contracts: • Future Contracts, where the energy buyer agrees to purchase an amount of energy, in a given delivery period, and the seller agrees to supply that same amount of energy for a specified price. Due to the existence of price fluctuations during the negotiation of the contract, the resulting economic gains or losses are settled on a daily basis. • Forward Contracts, which work as Future Contracts, with a difference in the settlement of economic gains or losses, which is made per month. • Swap contracts, where there is an exchange of a variable price position for a fixed price position, or vice versa.
• Intraday spot markets, managed by OMIE, that constitute adjustment markets: they provide flexibility of operations to the agents involved in such markets according to 6 intraday sessions. • Bilateral contracting markets, managed by OMIP, where there is a direct negotiation among the agents involved.
Within the framework of the 13th European Energy Market Conference held in Porto, Portugal, in June 2016, the electricity price forecast competition, EEM2016 EPF competition [46], a real-time price forecast competition in the environment of the MIBEL, was launched. A total of forty four competitors (predictors) participated in the EEM2016 EPF competition. They provided spot (point) price forecasts for the MIBEL daily market. Thus, predictors were SPF tools used by the participants in the EEM2016 EPF competition.
In EEM2016 EPF competition an extensive set of available data for the predictors was provided to the participants. This data included hourly prices in the previous days, forecasts of hourly demand and wind power generation for the Spanish area, weather forecasts (hourly wind speed, wind direction, precipitation, temperature and irradiation) for the day-ahead in the region (mainland of Portugal and Spain), regional-aggregated hourly power demands and hourly power generations of most of the types of electricity production in the previous day, hourly power demands in the previous week, and chronological data. Mean absolute errors for the competition period led to the selection of the twenty best predictors (predictors P1 to P20).
The novel PPFMCP models presented in this article utilize aggregation of the best predictors (P1 to P20) of the EEM2016 EPF competition. The expected and variance values associated to the ensemble of the point (spot) price forecasts of the predictors, for each hour, directly determine the parameter values of the PDF of a Beta distribution for the output variable (hourly price). These parameter values are the expected and variance values which are obtained by means of three aggregation strategies of the price forecasts of the predictors corresponding to three PPFMCP models. PPFMCP model 1 uses aggregation by the averaging of price forecasts of the predictors; PPFMCP model 2 uses aggregation of price forecasts of the predictors by means of weight values according to a daily rank of the predictors; and PPFMCP model 3 uses aggregation by means of optimized weights. Thus, from the PDFs, the expected value of the hourly price variable of each PPFMCP model is determined, and a multiplicity of practical quantitative results of the probabilistic forecast of each probabilistic model can be obtained.
The real-life case study of the MIBEL was used to test the PPFMCP models satisfactorily. A Reliability Indicator (RI) and a Loss function Indicator (LI) were introduced to assess the uncertainty associated with probabilistic price forecasts of PPFMCP models. The RI values and LI values, as well as the mean absolute error values for such PPFMCP models in the case study of the MIBEL showed that PPFMCP model 2 was the best probabilistic model. PPFMCP model 1 was the second best probabilistic model.
PPFMCP models and practical probabilistic information from their PDFs for hourly price forecasts can be useful for MIBEL agents of the day-ahead market as well as for other agents of the power industry. The probabilistic forecasts provided by PPFMCP models can allow the agents in the MIBEL to select the trading strategies according to their level of tolerated risk.
The structure of this article is the following: Section 2 contains a description of the EEM2016 real-time electricity price forecast competition. Section 3 describes probabilistic price forecasting models by aggregation of competitive predictors. Section 4 presents the analysis of the results of probabilistic price forecasting models in the real-life case of the MIBEL. Finally, the conclusions of this article are presented in Section 5.

EEM2016 Real-Time Electricity Price Forecast Competition
The main characteristics of the EEM2016 EPF competition [45] are presented in this section. The characteristics of the data available for the predictors used by the participants in the competition are described afterwards. Lastly, the performance of the predictors is analysed.

Main Characteristics of the EEM2016 EPF Competition
Several competitions for short-term forecasts in the field of electric power systems have been carried out in the last years such as the Puget Sound Power and Light Company load forecast competition in 1990 [47], the EUNITE Peak Load Forecast competition in 2001 [48], the Global Energy Forecasting Competition 2012 (GEFCom 2012) [49], the AMS Solar Energy Prediction Contest 2013-2014 [50], the Global Energy Forecasting Competition 2014 (GEFCom 2014) [51] and the Global Energy Forecasting Competition 2017 (GEFCom 2017). In these competitions, different competitors used their best forecasting models (predictors) trying to be the winners.
The 13th International Conference on the European Energy Market (EEM2016) was held on 6th-9th June in Porto, Portugal, organized by INESC TEC [52] with the collaboration of the Faculty of Engineering of the University of Porto. The EEM2016 Organizing Committee in collaboration with the enterprise Smartwatt launched a real-time price forecast competition (EEM2016 electricity price forecast competition) in April 2016 in a Competition Platform (COMPLATT) which corresponded to an active and realistic framework, analogous to real-world situations. The open competition intended to involve diverse actors such as industry professionals, the academic community and scientific researchers, with the objective of dealing with a real-world price forecasting case in the framework of the MIBEL. Such platform was launched on the 15th of February in order for competitors (with their predictors) to start to know and work on COMPLATT.
The main challenge of this competition with respect to those mentioned above, was to forecast the spot price of the MIBEL for the 24 h of the 5 days ahead, on a daily rolling basis and in a real-time environment. It is important to observe that, as far as we know, no real-time competition platform has been used to evaluate the performance of price forecasting models before the EEM2016 EPF competition. The EEM2016 EPF competition constituted the first time that a real-time competition about the MIBEL was carried out. Competitors submitted their forecasts (obtained from their predictors) for the 120 h ahead, before 10:00 UTC for each day of the competition. The day-ahead clearing hourly prices of electricity were provided by OMIE at 12:00 UTC. These prices were accessible for the competitors at 13:00 UTC.
Historical hourly data, corresponding to year 2015, was available to the participants. Along the competition rolling period, a daily update of data (rolling data) was also available. These data are described with detail in Section 2.2. Furthermore, competitors could make use of any other additional data emulating real-life situations. The performance of the predictor forecasts was evaluated by calculating mean absolute errors in €/MWh as described in Section 2.3.

Characteristics of Data
Different kinds of historical information in 2015 were available for the competitor predictor training. The historical information was the following: i.
Chronological data (hour, day of the week). ii.
Actual hourly data prices of the daily electricity market (MIBEL). iii.
Actual hourly data of Portuguese and Spanish power systems: power demand, hydropower generation, wind power generation, thermal generation (nuclear, coal and natural gas), solar power generation, cogeneration and power exchanged with France. This data was obtained by aggregating a very large amount of information from the websites of REE, the Spanish Transmission System Operator (TSO) [53], and REN, the Portuguese TSO [54]. iv.
Data of hourly forecasts of wind power generation and power demand for the Spanish area. v.
Data of hourly weather forecasts: wind speed and direction, temperature, irradiance and precipitation. These forecasts were provided for a mesh of points covering the Iberian Peninsula.
Each day of the competition, competitors could access the most recent data on a rolling window basis. The rolling data for the rolling inputs of the predictors had similar characteristics to those described above. Such rolling data were available for the competitors, starting on the 15th of February, 2016, until the end of the EEM2016 EPF competition. As mentioned above, the competition took place in April 2016 and all the competitors had therefore the opportunity to test and reconfigure their predictors at an early stage.
In order to understand the quality of predictor forecasts, an analysis of the values of historical data for the years 2015 and 2016 was carried out. Monthly electricity generation values, as well as the demand and real price of electricity values, corresponding to 2015 and to the first ten months of 2016 are plotted in Figures 1 and 2, respectively.
Both figures show the monthly average values of power demand, hydropower generation, renewable (wind and solar) and cogeneration power generation, coal power generation, nuclear power generation, combined cycle power generation and power exchanged with France. Monthly average electricity prices in the daily session of the MIBEL are also represented in both figures. Figures 3 and 4 show hourly mean values of electricity generation through the most varied sources, demand and real price of electricity corresponding to April 2015 and April 2016, respectively.
As indicated in Figures 3 and 4, the month of April 2015 shows quite significant changes compared to the same month in 2016 (shown in Figure 4). These are meaningful months in the production of electricity by the hydroelectric plants, which increased significantly in 2016 as a consequence of a higher rainfall than that of year 2015. As a result, the production of electricity by coal and natural gas power plants decreased, and the special regime production (where mini and micro-hydro productions are included) increased. All these facts led to a 40-50% decrease in electricity prices, which means that there was an increased difficulty in the price forecasting capabilities of the predictors of the for April 2016, due to the changes of generations with respect to those in April 2015.
Energies 2018, 11, 1074 6 of 25 took place in April 2016 and all the competitors had therefore the opportunity to test and reconfigure their predictors at an early stage.
In order to understand the quality of predictor forecasts, an analysis of the values of historical data for the years 2015 and 2016 was carried out. Monthly electricity generation values, as well as the demand and real price of electricity values, corresponding to 2015 and to the first ten months of 2016 are plotted in Figures 1 and 2, respectively.   Figure 4). These are meaningful months in the production of electricity by the hydroelectric plants, which increased significantly in 2016 as a consequence of a higher rainfall than that of year 2015. As a result, the production of electricity by coal and natural gas power plants decreased, and the special regime production (where mini and micro-hydro productions are included) increased. All these facts led to a 40-50% decrease in electricity prices, which means that there was an increased difficulty in the price forecasting capabilities of the predictors of the for April 2016, due to the changes of generations with respect to those in April 2015.    Figure 4). These are meaningful months in the production of electricity by the hydroelectric plants, which increased significantly in 2016 as a consequence of a higher rainfall than that of year 2015. As a result, the production of electricity by coal and natural gas power plants decreased, and the special regime production (where mini and micro-hydro productions are included) increased. All these facts led to a 40-50% decrease in electricity prices, which means that there was an increased difficulty in the price forecasting capabilities of the predictors of the for April 2016, due to the changes of generations with respect to those in April 2015.

Predictors Performance
The evaluation of the forecasts provided by the predictors of the EEM2016 EPF competition was mainly based on mean absolute errors expressed in €/MWh. The final rank of the EEM2016 EPF competition was computed after competition closure by calculating the hourly Mean Absolute Error for the best 10 days of forecast, for the entire 5 days forecast horizon. In spite of a better use of the limited number of forecast days of the competition, in this article we evaluate the performance of predictors for the 14 days of such competition, and thus maximizing the utilization of the available information.
In this article, the rank of predictors (from predictor P1 to predictor P20) was carried out for the day-ahead (D + 1) taking into account the forecasts provided by the competitors at day D, for a better analysis of the probabilistic price forecasting models presented later, in Section 3.
The mean absolute error MAE(D + k) corresponding to the forecast for day D + k (k = 1, 2, 3, 4, 5), provided at day D, is defined by Equation (1): is the forecasted price (provided at day D) for hour h of day D + k, and ( + )ℎ is the real price for hour h of day D + k.
The predictors were globally ranked, from higher ranks to lower ranks, in the competition period, according to the MAE for "all days" of the competition period, ( + 1) , which corresponds to the average of the mean absolute errors for day D + 1 associated with the forecasts provided during the days of the competition period.
( + 1) for all the competition period (14 days) is given by Equation (2): Figure 5 shows the forecasts |( ) for the twenty best predictors (P1 to P20), provided by the competitors at day D for each hour h of day D + 1, during the days of competition in April 2016. Furthermore, the real price is also shown in Figure 5.

Predictors Performance
The evaluation of the forecasts provided by the predictors of the EEM2016 EPF competition was mainly based on mean absolute errors expressed in €/MWh. The final rank of the EEM2016 EPF competition was computed after competition closure by calculating the hourly Mean Absolute Error for the best 10 days of forecast, for the entire 5 days forecast horizon. In spite of a better use of the limited number of forecast days of the competition, in this article we evaluate the performance of predictors for the 14 days of such competition, and thus maximizing the utilization of the available information.
In this article, the rank of predictors (from predictor P1 to predictor P20) was carried out for the day-ahead (D + 1) taking into account the forecasts provided by the competitors at day D, for a better analysis of the probabilistic price forecasting models presented later, in Section 3.
The mean absolute error MAE(D + k) corresponding to the forecast for day D + k (k = 1, 2, 3, 4, 5), provided at day D, is defined by Equation (1): whereP D|(D+k)h is the forecasted price (provided at day D) for hour h of day D + k, and P(D + k)h is the real price for hour h of day D + k. The predictors were globally ranked, from higher ranks to lower ranks, in the competition period, according to the MAE for "all days" of the competition period, MAE g (D + 1), which corresponds to the average of the mean absolute errors for day D + 1 associated with the forecasts provided during the days of the competition period. MAE g (D + 1) for all the competition period (14 days) is given by Equation (2): Figure 5 shows the forecastsP D|(D+1)h for the twenty best predictors (P1 to P20), provided by the competitors at day D for each hour h of day D + 1, during the days of competition in April 2016. Furthermore, the real price is also shown in Figure 5.  Figure 6 shows the histogram of daily mean errors of the price forecasts provided by the predictors for each of the two weeks of the competition, based on day-ahead hourly price forecasted values minus real hourly price values. Thus, the mentioned daily mean errors (for each predictor i) are calculated according to expression (3): where, , |( ) is the price forecasted at day D for hour h of day D + 1 by the predictor i (point forecast, for hour h of day D + 1), and ( + 1)ℎ is the real price for hour h of day D + 1. The average price during the competition period was 20.5 €/MWh, and the real price values ranged from 0 to 42.7 €/MWh. Observe that MAPE (mean absolute percentage error) corresponds to MAE divided by this average price during the competition period.     Figure 6 shows the histogram of daily mean errors of the price forecasts provided by the predictors for each of the two weeks of the competition, based on day-ahead hourly price forecasted values minus real hourly price values. Thus, the mentioned daily mean errors (for each predictor i) are calculated according to expression (3): where,P i,D|(D+1)h is the price forecasted at day D for hour h of day D + 1 by the predictor i (point forecast, for hour h of day D + 1), and P(D + 1)h is the real price for hour h of day D + 1.  Figure 6 shows the histogram of daily mean errors of the price forecasts provided by the predictors for each of the two weeks of the competition, based on day-ahead hourly price forecasted values minus real hourly price values. Thus, the mentioned daily mean errors (for each predictor i) are calculated according to expression (3): where, , |( ) is the price forecasted at day D for hour h of day D + 1 by the predictor i (point forecast, for hour h of day D + 1), and ( + 1)ℎ is the real price for hour h of day D + 1. The average price during the competition period was 20.5 €/MWh, and the real price values ranged from 0 to 42.7 €/MWh. Observe that MAPE (mean absolute percentage error) corresponds to MAE divided by this average price during the competition period.    The average price during the competition period was 20.5 €/MWh, and the real price values ranged from 0 to 42.7 €/MWh. Observe that MAPE (mean absolute percentage error) corresponds to MAE divided by this average price during the competition period. Figure 6 shows that the daily mean error with the highest frequency in the first week is around 5 (€/MWh), representing about 25% of the average price during the competition period. In general, the predictors forecasted price in excess, possibly caused by the exceptional weather conditions in the competition period, with more hydro and wind generation than those of the historical data used to build the predictors. In the first week of the competition, the bias was more significant than in the second week, indicating an improvement of the predictors' performance during the competition. This element of improvement is also evident in Figure 7, with the exception of the 18 April 2016, with a high penetration of renewable generation. Figure 7 shows the values of MAE(D + 1) and MAPE(D + 1) for each day of the competition period as well as the values of MAE g (D + 1), that is, MAE for "all days" of the competition period, of the nine best ranked predictors, P1 to P9. Observe that MAPE(D + 1) corresponds to MAE(D + 1) divided by the average price during the competition period.
Energies 2018, 11, x 9 of 25 Figure 6 shows that the daily mean error with the highest frequency in the first week is around 5 (€/MWh), representing about 25% of the average price during the competition period. In general, the predictors forecasted price in excess, possibly caused by the exceptional weather conditions in the competition period, with more hydro and wind generation than those of the historical data used to build the predictors. In the first week of the competition, the bias was more significant than in the second week, indicating an improvement of the predictors' performance during the competition. This element of improvement is also evident in Figure 7, with the exception of the 18 April 2016, with a high penetration of renewable generation. Figure 7 shows the values of MAE(D + 1) and MAPE(D + 1) for each day of the competition period as well as the values of ( + 1), that is, MAE for "all days" of the competition period, of the nine best ranked predictors, P1 to P9. Observe that MAPE(D + 1) corresponds to MAE(D + 1) divided by the average price during the competition period. The performances of the first and second best ranked predictors (P1 and P2) for "all days" of the competition period (with ( + 1) of 2.6 and 2.8 €/MWh, respectively), are better than those of the third predictor, P3, and the remaining predictors (which have ( + 1) around 3.5 €/MWh).
It should be noted that the EEM2016 EPF competition was a real rolling price forecast environment, with a total of forty four real predictors (SPF tools) and a large set of input data. Very limited information about predictors, and the data used by them, was available. However, we knew that predictors P1 and P2 were used by professional traders with access to privileged information and with high experience in electricity price markets. Predictors P3, P4 and P5 were utilized by academics who used only part of the public data available for the EEM2016 EPF competition and applied neural network modelling and advanced forecasting techniques. These facts are important to observe that predictors of the academics using limited information are not as good as the predictors of professional traders, but the forecasts of the predictors of the academics seem to be close to the forecasts of the best predictors. Figure 8 shows values of ( + 1) to ( + 5), as well as the corresponding values of ( + 1) to ( + 5), of the predictors P1 to P9, for day D + 1 to day D + 5, respectively, during the period of the competition, according to Equation (4) with (k = 1, 2, 3, 4, 5): The ( + ) value corresponds to the ( + ) value divided by the average price value during the competition period. The performances of the first and second best ranked predictors (P1 and P2) for "all days" of the competition period (with MAE g (D + 1) of 2.6 and 2.8 €/MWh, respectively), are better than those of the third predictor, P3, and the remaining predictors (which have MAE g (D + 1) around 3.5 €/MWh).
It should be noted that the EEM2016 EPF competition was a real rolling price forecast environment, with a total of forty four real predictors (SPF tools) and a large set of input data. Very limited information about predictors, and the data used by them, was available. However, we knew that predictors P1 and P2 were used by professional traders with access to privileged information and with high experience in electricity price markets. Predictors P3, P4 and P5 were utilized by academics who used only part of the public data available for the EEM2016 EPF competition and applied neural network modelling and advanced forecasting techniques. These facts are important to observe that predictors of the academics using limited information are not as good as the predictors of professional traders, but the forecasts of the predictors of the academics seem to be close to the forecasts of the best predictors. Figure 8 shows values of MAE g (D + 1) to MAE g (D + 5), as well as the corresponding values of MAPE g (D + 1) to MAPE g (D + 5), of the predictors P1 to P9, for day D + 1 to day D + 5, respectively, during the period of the competition, according to Equation (4) with (k = 1, 2, 3, 4, 5): The MAPE g (D + k) value corresponds to the MAE g (D + k) value divided by the average price value during the competition period.  Figure 8 shows an increase in the ( + ) error as k increased, that is, the performance of the forecast for the day D + k worsens as the considered day D + k is farther away from day D. The forecasts provided at day D for day D + k with k = 2 to 5 can be important for a forecast user when no new forecasts are available at the days following day D due to (for example) fails in technology supporting the communications between such forecast user and the forecast provider, or due to any kind of energy storage, which might need to know prices over the next few days to decide operation now. Figure 8 proves that forecast performances up to k = 5 have an acceptable quality and, therefore, the forecasts for k = 2 to 5 (at day D) could be considered for practical strategic decisions, especially when they are the only information available (i.e., due to fails in technology, as specified above). Furthermore, the price forecast for k = 2 to 5 can be useful for some decisions associated with shortterm bidding strategies in electricity markets, as for example bidding in week-ahead future electricity derivative products. In Figure 8 the increase of ( + ) values, with respect to those of ( + 1), does not seem very significant for k = 2; but the increase of ( + ) values is evident for k = 5. It should be noted that the best predictors, P1 and P2, which are the best predictors according to MAE for "all" the competition period (that is, according to ( + 1)), are not always better than the other predictors from the point of view of ( + ) for k ≠ 1. For example, it can be observed that predictor P3 (academic competitor) is the best predictor for k = 2 and k = 3 from a point of view of ( + ).

Probabilistic Forecasting Models by Aggregation of Competitive Predictors
Forecasts of the predictors are point (spot) forecasts. Therefore, the ensemble of these predictor forecasts can be the basis for a probabilistic representation of the set of point forecasts by using probabilistic price forecasting models.
A detailed modelling of PDFs of Beta distributions for each hour of probabilistic price forecasts is first described in this Section 3, as well as two indicators of uncertainty of probabilistic forecasts. Afterwards three probabilistic price forecasting models (PPFMCP models) by aggregation of competitive predictors are presented in Sections 3.1-3.3.
Let's say that point forecasts for hour h of day D + 1 of n predictors (forecasts provided at day D), are , |( ) (i = 1, 2,… n). Let's say that the maximum forecasted price value for hour h of day D + 1 from the set of forecasts of n predictors is represented by  Figure 8 shows an increase in the MAE g (D + k) error as k increased, that is, the performance of the forecast for the day D + k worsens as the considered day D + k is farther away from day D. The forecasts provided at day D for day D + k with k = 2 to 5 can be important for a forecast user when no new forecasts are available at the days following day D due to (for example) fails in technology supporting the communications between such forecast user and the forecast provider, or due to any kind of energy storage, which might need to know prices over the next few days to decide operation now. Figure 8 proves that forecast performances up to k = 5 have an acceptable quality and, therefore, the forecasts for k = 2 to 5 (at day D) could be considered for practical strategic decisions, especially when they are the only information available (i.e., due to fails in technology, as specified above). Furthermore, the price forecast for k = 2 to 5 can be useful for some decisions associated with short-term bidding strategies in electricity markets, as for example bidding in week-ahead future electricity derivative products.
In Figure 8 the increase of MAE g (D + k) values, with respect to those of MAE g (D + 1), does not seem very significant for k = 2; but the increase of MAE g (D + k) values is evident for k = 5. It should be noted that the best predictors, P1 and P2, which are the best predictors according to MAE for "all" the competition period (that is, according to MAE g (D + 1)), are not always better than the other predictors from the point of view of MAE g (D + k) for k = 1. For example, it can be observed that predictor P3 (academic competitor) is the best predictor for k = 2 and k = 3 from a point of view of MAE g (D + k).

Probabilistic Forecasting Models by Aggregation of Competitive Predictors
Forecasts of the predictors are point (spot) forecasts. Therefore, the ensemble of these predictor forecasts can be the basis for a probabilistic representation of the set of point forecasts by using probabilistic price forecasting models.
A detailed modelling of PDFs of Beta distributions for each hour of probabilistic price forecasts is first described in this Section 3, as well as two indicators of uncertainty of probabilistic forecasts. Afterwards three probabilistic price forecasting models (PPFMCP models) by aggregation of competitive predictors are presented in Sections 3.1-3.3.
Let's say that point forecasts for hour h of day D + 1 of n predictors (forecasts provided at day D),  (10) and (11)). Thus,B D|(D+1)h is represented by the function of expression (5): A PDF,B D|(D+1)h , of a Beta distribution, supported in the interval [0, 1], corresponding to the normalized stochastic price forecast variable P D|(D+1)h can be defined. The stochastic normalized variable P D|(D+1)h is associated with normalized valuesP i,D|(D+1)h which are defined by Equation (6): Thus,B D|(D+1)h is represented by the function of expression (7):  (8) and (9), respectively: where w i,D is a weight value for predictor i at day D that depends on the model chosen among the probabilistic price forecasting models that are presented in Sections 3.1-3.3.
With the expected value E B D|(D+1)h and variance V B D|(D+1)h , obtained by (8) and (9), parameters α D|(D+1)h and β D|(D+1)h are calculated by Equations (10) and (11): be obtained by using the PDF given by expression (5) of a Beta distribution, supported in such interval. Thus, the expected value of the hourly price is given by expression (12).
Practical probabilistic values of each PPFMCP model can be also obtained from their PDF given by expression (5). These probabilistic models can then lead to the evaluation of risk decisions based on price, calculations on the probability to have prices which are higher than some price limits, calculations on the probability to obtain prices in a target interval, calculations of quantiles, etc. For example, quantile Q p;D|(D+1)h for probability value p can be obtained by using the inverse of the cumulative distribution function (represented by CF −1 ) for functionB D|(D+1)h , through expression (13): On one hand, appropriate performance error indicators, as the mean absolute error, can be used to evaluate the accuracy of explainable information of the forecasts of the PPFMCP models, based on deviations in real price values with respect to the expected values of the hourly price (point forecast of hourly price) from such PPFMCP models.
On the other hand, the accuracy of representing the non-explainable information of the forecasts of the PPFMCP models can be evaluated by suitable indicators of uncertainty of the probabilistic forecasts of such models. Two indicators of probabilistic price forecast uncertainty of the probabilistic models have been introduced in this article, which are the Loss function Indicator, LI, and the Reliability Indicator, RI, described in the following paragraphs.
Indicator LI depends on vector Indicator LI is calculated by evaluating quantile deviations with respect to the real price for each hour t. Here, the real price is represented as P t . Observe that P t corresponds to P(D + 1)h of Equation (2), (i.e., P(D + 1)h is the real price for hour h of day D + 1). Thus, the quantile derivations L Q p,t ; P t are obtained by Equation (14): where quantile Q p,t is obtained according to expression (13). It should be noted that Q p,t corresponds to Q p;D|(D+1)h during the abovementioned period. The Loss function Indicator, LI, is then computed by Equation (15) and corresponds to the average value of the quantile derivations (relatively to the real price) for the total of nq quantiles and the total of m hours: The accuracy of representing the non-explainable information of the forecasts of a PPFMCP model is better for lower values of LI. Indicator LI corresponds to a unique value which allows to make the comparison among PPFMCP models. However, this does not allow to carry out a detailed analysis. The Loss function Indicator for each quantile, LI p , can be used for a graphic representation and analysis, as shown in Section 4. LI p is computed by Equation (16): In order to calculate the Reliability Indicator, RI, some previous definitions are necessary. A reliability diagram can be obtained for the probabilistic hourly price forecasts of each PPFMCP model, as shown later in Section 4. In the horizontal axis of the diagram, probability intervals (pi) are represented. The number of probability intervals npi are defined in accordance with the desired degree of detail and with the number m of forecasts available for evaluation. Probability interval pi corresponds to the interval defined by the quantile for probability p (pi−1) and by the quantile for probability p (pi) , where (p (pi) − p (pi−1) ) is constant.
In the vertical axis of the reliability diagram, the frequency of the events (price occurrences) are represented. The target frequency is an ideal frequency of events, (100 × ftar pi ), as a percentage, which can be calculated as 100 divided by the npi number. Furthermore, in such vertical axis the observed frequency, (100 × fobs pi ), as a percentage, corresponds to the frequency of the events (price occurrences) consisting in that the real price value P t occurs into the considered probability interval pi, that is, P t is greater than (or equal to) the quantile for probability p (pi−1) for hour t and P t is lower than the quantile for probability p (pi) for hour t.
For cases when value P t occurs outside the limits of the PDF of the Beta distribution for hour t (value lower than the minimum, or value greater than maximum from the set of predictors), fobs pi frequencies are also computed. It is necessary to observe that ftar pi frequencies are zero when the values of real price occur outside those limits, which corresponds to events occurring into two additional "intervals" (indication "<min" for interval below the minimum, and indication ">max" for interval above the maximum).
Higher values of RI indicate a better accuracy for representing the non-explainable information of the forecasts of the PPFMCP models, that is, a better matching of the PDFs during the abovementioned period. Indicator RI returns a unique value for each probabilistic model; however, it is possible to evaluate details of the adequacy quality of the PDFs by observing the reliability diagram and by analyzing the frequencies of events in each probability interval.
Probabilistic price forecasts of PPFMCP model 1 correspond to PDFs of Beta distributions, with four parameters given by expression (5), for each hour h of day D + 1. Table 1 and Figure 9a  Probabilistic price forecasts of PPFMCP model 1 correspond to PDFs of Beta distributions, with four parameters given by expression (5), for each hour h of day D + 1. Table 1 and Figure 9a show examples of the results of PPFMCP model 1 for three hours, which are hours 11 h, 12 h, and 13 h, on the 14th of April, 2016. Table 1,   The three probability density functions of PPFMCP model 1, presented in Figure 9a, show different shapes for the three hours.   The three probability density functions of PPFMCP model 1, presented in Figure 9a, show different shapes for the three hours.

Probabilistic Price Forecasting Model 2 (PPFMCP Model 2) by Aggregation of Competitive Predictors
PPFMCP model 2 weights the different predictors as a function of a daily rank of each predictor. Rank R i,D is defined as the daily rank of predictor i at day D. R i,D is obtained taking into account a daily value of the mean absolute error of each predictor i for day D, which is calculated by expression (18): Weight values w i,D are then values that belong to the set {1, 1/2, 1/3, 1/4, 1/5 . . . 1/20}, taking into account the daily ranking of predictors. A vector of twenty weights was used (one weight value for each predictor), for each day D. In PPFMCP model 2, these weights were applied to Equations (8) and (9) for the twenty best predictors (P1 to P20), being n = 20. Table 2 shows weights w i,D of PPFMCP model 2 for each predictor. As expected, the best daily performing predictors have higher weights values (more green cells and less red cells in Table 2).

Probabilistic Price Forecasting Model 3 (PPFMCP Model 3) by Aggregation of Competitive Predictors
The third probabilistic model, PPFMCP model 3, uses weights u i,D obtained by a minimization process for day D, which is defined by expression (19): In PPFMCP model 3, these weights u i,D are applied to Equations (8) and (9) (u i,D is used in substitution of w i,D ), for the twenty best predictors (P1 to P20), being n = 20. The results of the optimization of expression (19) depend on the differences in the performance of the predictors. In the EEM2016 EPF competition, there were significant differences in the performance of predictors. Table 4 shows the weights values of u i,D of PPFMCP model 3 for the twenty best predictors (P1 to P20), where higher values are shown in green cells and null values are shown in red cells. In comparison with Table 2, now Table 4 shows that the optimization leads to select weights in a way that only a few predictors (in comparison with PPFMCP model 2) are used by PPFMCP model 3 for the day-ahead price forecasts. PPFMCP model 3 leads then to provide more sharpened PDFs than PPFMCP model 2. Table 4. Weights u i,D of the PPFMCP model 3 for the predictors P1 to P20. In comparison with PPFMCP model 2, PPFMCP model 3 differentiates predictors, giving more relevance to predictors with a better performance.
Examples of results of PPFMCP model 3 are shown in Table 5 and Figure 9c for hours 11 h, 12 h, and 13 h on the 14th of April, 2016. In Figure 9c, the higher sharpness of the probability density functions obtained by PPFMCP model 3 is more evident compared with those of PPFMCP model 2 and PPFMCP model 1.

Analysis of Results of Probabilistic Price Forecasting Models
The point (spot) forecasts (expected values of the hourly price) of PPFMCP models 1, 2 and 3 are the values, in €/MWh, computed by expression (12). Figure 10 presents point forecasts of PPFMCP models 1, 2 and 3; the (point) forecasts of the three best predictors P1, P2 and P3; and the real hourly price. The point forecasts of PPFMCP models 1 and 2 are fairly similar; the point forecast of PPFMCP model 3 seems to be a bit more different, but close to the point forecasts of PPFMCP models 1 and 2. functions obtained by PPFMCP model 3 is more evident compared with those of PPFMCP model 2 and PPFMCP model 1.

Analysis of Results of Probabilistic Price Forecasting Models
The point (spot) forecasts (expected values of the hourly price) of PPFMCP models 1, 2 and 3 are the values, in €/MWh, computed by expression (12). Figure 10 presents point forecasts of PPFMCP models 1, 2 and 3; the (point) forecasts of the three best predictors P1, P2 and P3; and the real hourly price. The point forecasts of PPFMCP models 1 and 2 are fairly similar; the point forecast of PPFMCP model 3 seems to be a bit more different, but close to the point forecasts of PPFMCP models 1 and 2.   Figure 11 give the mean absolute error values (MAE in Table 6 and Figure 11) of the point forecasts of the three probabilistic models and the mean absolute error values of the (point) forecasts of the three best predictors P1, P2 and P3, corresponding to the period from the 6th to 18th of April 2016, and to each day of this period.
As shown in Table 6, PPFMCP model 2 is the model with a better performance (the MAE value is 2.70 €/MWh, for the 6th to 18th of April 2016); PPFMCP model 1 is close to the former model (the MAE value is 2.78 €/MWh); PPFMCP model 3 obtains a worse performance (the MAE value is 2.93 €/MWh). Comparing the performances of PPFMCP models with those of the individual predictors, we conclude that none of the probabilistic models beats predictor P1, but PPFMCP model 2 and PPFMCP model 1 surpass predictor P2, and all three PPFMCP models beat predictor P3 as well as all the remaining predictors. Considering that predictors P1 and P2 have a significant better performance in comparison with other predictors, we consider that the best predictor P1 was expected not to be beaten by PPFMCP models. It is necessary to remember that predictors P1 and P2 are used by experienced professional price forecasters with access to non-public data. Figure 11 shows that, for the first week of the competition, PPFMCP models perform often worse than individual predictor P1; but the performance of probabilistic models is usually better during the second week than the performance in the first week. This fact can be explained because most of the predictors improved their performance progressively during the competition (with less dispersion  Table 6 and Figure 11 give the mean absolute error values (MAE in Table 6 and Figure 11) of the point forecasts of the three probabilistic models and the mean absolute error values of the (point) forecasts of the three best predictors P1, P2 and P3, corresponding to the period from the 6th to 18th of April 2016, and to each day of this period.
As shown in Table 6, PPFMCP model 2 is the model with a better performance (the MAE value is 2.70 €/MWh, for the 6th to 18th of April 2016); PPFMCP model 1 is close to the former model (the MAE value is 2.78 €/MWh); PPFMCP model 3 obtains a worse performance (the MAE value is 2.93 €/MWh). Comparing the performances of PPFMCP models with those of the individual predictors, we conclude that none of the probabilistic models beats predictor P1, but PPFMCP model 2 and PPFMCP model 1 surpass predictor P2, and all three PPFMCP models beat predictor P3 as well as all the remaining predictors. Considering that predictors P1 and P2 have a significant better performance in comparison with other predictors, we consider that the best predictor P1 was expected not to be beaten by PPFMCP models. It is necessary to remember that predictors P1 and P2 are used by experienced professional price forecasters with access to non-public data. Figure 11 shows that, for the first week of the competition, PPFMCP models perform often worse than individual predictor P1; but the performance of probabilistic models is usually better during the second week than the performance in the first week. This fact can be explained because most of the predictors improved their performance progressively during the competition (with less dispersion among their spot forecasts); thus, forecasts of PPFMCP models resulted in better performances progressively. among their spot forecasts); thus, forecasts of PPFMCP models resulted in better performances progressively.  The Loss function Indicator (LI) calculated according to Equation (15) in €/MWh, and the Reliability Indicator (RI) calculated according to Equation (17) in %, were used to evaluate the uncertainty associated with the probabilistic price forecasts of PPFMCP models. Table 7 shows the values of indicators LI and RI for the three PPFMCP models. From the point of view of representing the non-explainable information of the forecasts of PPFMCP models, the values of indicator LI and the values of the indicator RI, in Table 7, are consistent when comparing PPFMCP models. PPFMCP model 2, which utilizes aggregation of price forecasts of the predictors by means of weight values according to a daily rank of the predictors, is the best probabilistic model. The second best model is PPFMCP model 1, which uses aggregation by the averaging of price forecasts of the predictors. The last preferred model is PPFMCP model 3, which uses aggregation by means of optimized weights. This performance order of the PPFMCP models is consistent with the order obtained by the mean absolute errors analysis from Table 6. The Loss function Indicator (LI) calculated according to Equation (15) in €/MWh, and the Reliability Indicator (RI) calculated according to Equation (17) in %, were used to evaluate the uncertainty associated with the probabilistic price forecasts of PPFMCP models. Table 7 shows the values of indicators LI and RI for the three PPFMCP models. From the point of view of representing the non-explainable information of the forecasts of PPFMCP models, the values of indicator LI and the values of the indicator RI, in Table 7, are consistent when comparing PPFMCP models. PPFMCP model 2, which utilizes aggregation of price forecasts of the predictors by means of weight values according to a daily rank of the predictors, is the best probabilistic model. The second best model is PPFMCP model 1, which uses aggregation by the averaging of price forecasts of the predictors. The last preferred model is PPFMCP model 3, which uses aggregation by means of optimized weights. This performance order of the PPFMCP models is consistent with the order obtained by the mean absolute errors analysis from Table 6. The Loss function Indicator values for each quantile, LI p , are represented in Figure 12. The number of quantiles, nq, was 9, corresponding to probabilities between 0.1 and 0.9 (p = 0.1, 0.2, 0.3, . . . . 0.9). An ideal bell shape of LI p should show "central" quantiles with relatively higher values of LI p . Ideally the bell shape should be symmetrical, "centred" in quantile Q50 (which represents a probability of 0.5). Figure 12 shows that PPFMCP models 1 and 2 are quite similar, being PPFMCP model 2 slightly better. Both are clearly better than PPFMCP model 3 from the point of view of LI p . Figure 13 shows the reliability diagrams for PPFMCP models. The number of probability intervals, npi, is 10; probability intervals range from probability interval 0-0.1 to interval 0.9-1. It is important to observe that all intervals have the same length (length of value 0.1). In Figure 13, the target frequency is 10% for each probability interval, except for those with the indication "<min" or ">max" which have a null value for the target frequency. The observed frequencies, (100 × fobs pi ), as a percentage, are situated in indication "<min" of Figure 13 with notable values, as well as in some of the probability intervals with lower probability values. This fact occurs because most of the predictors generally provided forecasts with bias in excess in the competition, being this more frequent during the first week of the competition period. The observed frequencies of PPFMCP models 1 and 2 for near "central" probability intervals seem to form some parts of bell shapes roughly, which indicates that the PDFs of these probabilistic models are not sharpened enough, probably due to the influence of the predictors with worse performances. From this point of view (sharpened shape of PDFs), PPFMCP model 2 can be a bit better with respect to PPFMCP model 1.  The Loss function Indicator values for each quantile, LIp, are represented in Figure 12. The number of quantiles, , was 9, corresponding to probabilities between 0.1 and 0.9 ( = 0.1, 0.2, 0.3, …. 0.9). An ideal bell shape of should show "central" quantiles with relatively higher values of . Ideally the bell shape should be symmetrical, "centred" in quantile Q50 (which represents a probability of 0.5). Figure 12 shows that PPFMCP models 1 and 2 are quite similar, being PPFMCP model 2 slightly better. Both are clearly better than PPFMCP model 3 from the point of view of LIp. Figure 13 shows the reliability diagrams for PPFMCP models. The number of probability intervals, , is 10; probability intervals range from probability interval 0-0.1 to interval 0.9-1. It is important to observe that all intervals have the same length (length of value 0.1). In Figure 13, the target frequency is 10% for each probability interval, except for those with the indication "<min" or ">max" which have a null value for the target frequency. The observed frequencies, (100·fobspi), as a percentage, are situated in indication "˂min" of Figure 13 with notable values, as well as in some of the probability intervals with lower probability values. This fact occurs because most of the predictors generally provided forecasts with bias in excess in the competition, being this more frequent during the first week of the competition period. The observed frequencies of PPFMCP models 1 and 2 for near "central" probability intervals seem to form some parts of bell shapes roughly, which indicates that the PDFs of these probabilistic models are not sharpened enough, probably due to the influence of the predictors with worse performances. From this point of view (sharpened shape of PDFs), PPFMCP model 2 can be a bit better with respect to PPFMCP model 1.  Lastly, Figure 14 shows the probabilistic forecasts provided by PPFMCP model 2 in the last 13 days of the competition period. Observe that only the forecasts for those 13 days were available because the first day of the competition period was needed to provide the first rank used to obtain the weights for PPFMCP model 2. In Figure 14    Lastly, Figure 14 shows the probabilistic forecasts provided by PPFMCP model 2 in the last 13 days of the competition period. Observe that only the forecasts for those 13 days were available because the first day of the competition period was needed to provide the first rank used to obtain the weights for PPFMCP model 2. In Figure 14 are represented the quantiles corresponding to probability values of 0.05, 0.25, 0.75 and 0.95, as well as the point forecast provided by PPFMCP model 2 (the expected value obtained with Equation (8)), and the real price.

Conclusions
This article presents original probabilistic price forecasting models (PPFMCP models), by aggregation of competitive predictors for the day-ahead hourly probabilistic price forecasting. PPFMCP models were applied to the real-life case of the MIBEL.
The EEM2016 electricity price forecast competition (EEM2016 EPF competition), a real-time price forecast competition on the COMPLATT (competition platform) that corresponded to an active and realistic framework analogous to real-world situations, was first time launched in April 2016, where    Lastly, Figure 14 shows the probabilistic forecasts provided by PPFMCP model 2 in the last 13 days of the competition period. Observe that only the forecasts for those 13 days were available because the first day of the competition period was needed to provide the first rank used to obtain the weights for PPFMCP model 2. In Figure 14 are represented the quantiles corresponding to probability values of 0.05, 0.25, 0.75 and 0.95, as well as the point forecast provided by PPFMCP model 2 (the expected value obtained with Equation (8)), and the real price.

Conclusions
This article presents original probabilistic price forecasting models (PPFMCP models), by aggregation of competitive predictors for the day-ahead hourly probabilistic price forecasting. PPFMCP models were applied to the real-life case of the MIBEL.
The EEM2016 electricity price forecast competition (EEM2016 EPF competition), a real-time price forecast competition on the COMPLATT (competition platform) that corresponded to an active and realistic framework analogous to real-world situations, was first time launched in April 2016, where

Conclusions
This article presents original probabilistic price forecasting models (PPFMCP models), by aggregation of competitive predictors for the day-ahead hourly probabilistic price forecasting. PPFMCP models were applied to the real-life case of the MIBEL.
The EEM2016 electricity price forecast competition (EEM2016 EPF competition), a real-time price forecast competition on the COMPLATT (competition platform) that corresponded to an active and realistic framework analogous to real-world situations, was first time launched in April 2016, where a total of forty four predictors provided point (spot) price forecasts for the MIBEL on a daily rolling basis. Predictors were point price forecasting tools used by participants in the EEM2016 EPF competition. Historical hourly data, corresponding to the year 2015, were provided to the competition participants. During the competition period, a daily update of data (rolling data) was provided. The data available for the predictors were hourly prices on previous days, regional-aggregated hourly power demands and hourly power generations of most of the types of electricity production on the previous day, hourly power demands in the previous week, forecasts of hourly demand and wind power generation for the Spanish area, weather forecasts (hourly wind speed, wind direction, precipitation, temperature and irradiation) for the day-ahead in the region (mainland of Portugal and Spain), and chronological data. The predictors were globally ranked, according to mean absolute errors for the competition period, which led to the selection of the twenty best predictors (predictors P1 to P20).
The novel PPFMCP models are based on the aggregation of the best predictors (P1 to P20). For each hour, the point price forecasts of the predictors constitute an ensemble that is used to obtain the parameter values of the PDF of a Beta distribution for the output variable (hourly price). PDFs are obtained directly from the expected and variance values associated with such ensemble using three aggregation strategies of price forecasts of the predictors, which correspond to three probabilistic price forecasting models. PPFMCP model 1 uses aggregation by the averaging of price forecasts of the predictors; PPFMCP model 2 uses aggregation of price forecasts of the predictors by means of weight values according to a daily rank of the predictors; and aggregation by means of optimized weights is applied in PPFMCP model 3.
Thus, valuable quantitative probabilistic information of the behaviour of the hourly price variable can be determined from the PDF for each hour of the probabilistic price forecast of each PPFMCP model, which allows the evaluation of risk decisions based on the price to be made, as well as the calculation on the probability to have prices which are higher than some price limits, the calculation on the probability to obtain prices in a target interval, and the calculation of quantiles.
PPFMCP models were satisfactorily applied to the real-life case study of the MIBEL. Results of the PPFMCP models are described in this article, including PDFs of Beta distributions for probabilistic forecasts, as well as daily weight values of these probabilistic models.
Two appropriate indicators, called Reliability Indicator (RI) and Loss function Indicator (LI), are also introduced, which allow the uncertainty associated to probabilistic price forecasts of the PPFMCP models to be evaluated.
Results from PPFMCP models, showed that PPFMCP model 2 was the best probabilistic model from the point of view of the mean absolute error values, as well as from the point of view of LI and RI values; and results also showed that PPFMCP model 1 was the second best probabilistic model.
The probabilistic price forecasting models of this article and the pragmatic calculations applied to the PDFs associated with their day-ahead probabilistic price forecasts can be useful for business agents of electrical energy markets and other players who act in the electric power industry.  Vector of m probabilistic forecasts (of a considered PPFMCP model) B t Probabilistic price forecast for (hour) time t (t = 1, 2, 3, . . . .. m) Q p,t Quantile for probability p and for (hour) time t of forecast L Q p,t ; P t Quantile derivation, with respect to the real price, for each hour t, associated with LI LI p Loss function Indicator for the quantiles, corresponding to probability p, for m hours 100 × ftar pi Target frequency, in percentage, (ideal frequency of events) 100 × fobs pi Observed frequencies, in percentage