Comparative Analysis of Wind Energy Generation Forecasts in Poland and Portugal and Their Influence on the Electricity Exchange Prices

Abstract

Currently, the privileged position of wind energy producers is being weakened by their enforced participation in the market on equal terms. This requires accurate production forecasting. The main aim of this study is to comparatively examine the wind generation forecasts in Poland and Portugal, as well as to verify their influence on the day-ahead market prices. The statistical analysis revealed significant deviations of the forecasted and actual wind production in both countries, which referred to the corresponding spot and balancing prices caused considerable financial losses by the wind energy suppliers. In this paper, the influence of the wind generation forecasts on the spot prices has been examined through developed the auto-regressive moving average (ARMA), ARMA with exogenous inputs (ARMAX) and non-linear auto-regressive neural network (NAR), NAR with exogenous inputs (NARX)artificial neural network (ANN) models. The results have shown that the usability of the information of forecasted wind generation is not unequivocal in models developed for spot prices in Poland, mainly because of the randomness and volatility of recorded wind generation forecasts. However, in the case of Portugal, the forecasted wind generation occurred to be a valuable input in spot prices models, which results in an improvement in the models’ accuracy.

1. Introduction

Sustainable development of contemporary power system towards systems based on renewable energy sources (RES) requires facing some problems. These problems are mainly related to the inherent difficulty in predicting the availability of RES at a given time. This problem is made more difficult with the time-variation of the load and volatility of energy prices on the wholesale markets. One of the key aspects of reliable operation of the power system is proper planning of the supply versus demand, which, due to the RES generation uncertainty, becomes a struggle for responsible entities. The inaccuracy, of for instance wind generation forecasts, entails consequences of two primary natures: (i) technical—divergence of the forecasts around planned RES generation which impedes the transmission system operators’ (TSOs) ability to properly and accurately plan the fulfilment of actual demand by available power capacity; and (ii) economical—the uncertainty of the total forecasted wind power supply translates into the uncertainty of the day-ahead and balancing market prices, which can result in financial losses of the market participants.

This study will examine the economic aspect identified above. Wind power variability influences the wholesale market price. This considerable impact is caused by the relatively low marginal cost of the energy from wind farms, because wind energy is not burdened with fuels costs. Therefore, an injection of the wind energy, featured by the lowest unit cost, shifts the supply-demand curve to the right, which results in a reduction of the market prices.

In the literature, numerous publications can be found which support the hypothesis that the wind energy forecast is a valuable input in wholesale market price forecasting. Moreover, a wide range of mathematical models have been developed as well (e.g., [1,2,3])—however, their universalism has not been tested on divergent market environments, such as Polish and Portuguese ones.

The abovementioned concerns have led to the statement of several research problems which are addressed by this paper. The main contributions of the research are the following:

• The quality of overall wind power forecasts in Poland and Portugal, as well as distinguishing the most relevant features of the forecasting errors. To address this issue, an evaluation of the wind power prognoses will be made using statistical measures, which will give a quantitative picture of the accuracy, characteristics and tendencies observed. The comparison of obtained statistic description for both countries will allow to distinguish the main differences between the forecasts and point out the main sources of uncertainty.
• The cost of inaccurate wind power forecasting globally from the perspective of wind power producers in both countries. This is accomplished by using market and wind generation data available for both countries to calculate the financial loss from inaccurate wind power predictions. This will illustrate the range and importance of appropriate and accurate wind generation forecasts.
• The extent to which the information on expected wind generation improves the accuracy of the wholesale market prices forecasts in both examined countries. The verification of whether wind generation has an impact in predicting market prices requires developing mathematical models. For this study, the four models commonly used in dealing with spot market prices time series will be approached: ARMA—auto-regressive moving average; ARMAX—auto-regressive moving average with exogenous inputs; NAR—non-linear auto-regressive neural network and NARX—non-linear auto-regressive with exogenous inputs neural network. The modelling process will be carried out independently using data from Polish and Portuguese systems. This will evaluate if the inclusion of wind generation data improves the spot prices predictions in both countries, and examine if a given model can be successfully applied for the data coming from diametrically different power and market systems.

The current stage of wind energy development in Poland and Portugal is comparable. Citing the Portuguese TSO, at the end of 2019, the installed onshore wind power capacity accounted for about 5.2 GW, while, in Poland, according to the state’s Energy Regulatory Office, 2019 ended with 5.9 GW of wind power connected to the power system. So far, none of the countries has offshore wind installed capacity. After years of substantial and steady growth, in both cases, a plateau has been observed from 2016 onwards, and increases of installed power year to year are reaching no more than several dozens of MW. With regard to offshore wind power, both in Poland and in Portugal, no energy has been generated as of the end of 2019. In Portugal, a pre-commercial 25 MW offshore wind park using an innovative floating technology is being currently installed in deep waters. Though in Poland no construction works have been initiated yet, the special zone on the Baltic sea has been delimited, in order to develop offshore wind power of over 7 GW in the coming years.

The main objective of the study is to quantitatively evaluate the applicability of selected electricity price forecasting time series models in different datasets corresponding to Poland and Portugal market electricity prices. Moreover, we have verified the usefulness of wind power forecast as supplementary explaining variables of the models.

In the literature review concerning this research, a wide range of studies concerning the predictions of wind energy as well as spot market prices has been found. Although, usually, in the collected publications, a focus was put only on a single case of particular power system/power market. To the best knowledge of the authors, a multi-subject study has not been performed to reveal whether a given forecasting methodology applies to the separate cases of different national market environments, such as in Poland and Portugal. Although the Polish and Portuguese wind power capacity in the system is similar in amount when considering the share of this kind of energy the differences are substantial.

The main innovation of the paper is the use of the same methodology to analyse the influence of wind power forecasts in the market electricity prices of two differently conditioned power systems (Portugal and Poland), with a set of distinguishing features. This is accomplished by using a single data source, therefore making the comparison results robust and fully comparable.

The paper is organised as follows. To introduce the reader to the context of the abovementioned problems, Section 2 is devoted to the state-of-the-art research on the topics of wind power and spot market prices forecasting, together with particular focus on their interrelation. An overview of Portuguese and Polish RES sector and energy exchange institutions is offered in Section 3. Moreover, a description of the methodologies applied in solving the research problems stated in the introduction, including the basic statistical measures for dealing with forecasting error analysis and fundamentals of time series models constructed in terms of this study is presented in Section 4. In Section 5, the study results will be displayed, beginning from the statistical description of the wind generation forecasts error for Portugal and Poland, and related potential penalties suffered by the wind power producer globally. Finally, the spot prices forecasting results will be presented for the developed time-series models, with their comparative evaluation. At the end of this paper, the most significant findings will be highlighted in Section 6.

2. State-of-the-Art Review

Contemporarily, when the electricity is traded on wholesale markets, the uncertainty regarding the amount of wind energy supplied to the system has a direct influence on the market prices. Every excess or deficiency of energy in the system has a consequence in the financial outcomes of energy suppliers/receivers [1]. This elaboration concerns the two crucial problems in sustaining modern power systems, being a subject of intensive research within last decades, which are wind power forecasting (WPF) and electricity price forecasting (EPF). The increasing share of wind energy in the power systems, together with the tendency for equalisation of wind power producers with all the remaining electricity market participants, leads to the enhancement of the interaction between the WPF and EPF.

The impact of increased wind power penetration in power grid on the market player’s behaviour and their decisions has been studied by Usaola (2008), who estimated that the uncertainty of wind power supplied to the system leads to significant reduction of the energy producers revenues, reaching 10% of maximum obtainable revenue [1].

The same finding has been obtained by Crespo-Vasquez et al. (2017), who constructed a decision-making model of the windfarm owner which underlined the loss of incomes resulting from the uncertainty of wind power output—in this case, the losses constituting 4% of the total earnings [4].

The analysed publications indicate that the improvement of WPF is crucial from the perspective of increasing the effectiveness of market actions taken by its contributors [1,2,4]. It also should be pointed out that the uncertainty of the energy produced in a system has not only negative consequences. The presence of the day-ahead market, as well as the balancing market, may allow members to increase their revenues, who may speculate the prices and gain profits only from buying/selling the energy depending on the market or imbalance prices [1].

A comprehensive and condensed study on WPF methods has been delivered by Zhao (2011), which gives a view on numerous aspects related to this issue. Above all, two main approaches in WPF models can be distinguished: (i) physical: the models are created based on the measurements, technical data of the windmills and air parameters provided by weather prognoses; (ii) statistical: including the artificial intelligence, based on the statistical models [5].

Among the most important issues related to the WPF is its scalability, which has been pointed out by Rasheed et al. (2014). The author emphasises that, for two instances of the same weather conditions in the wind farm, the power output may differ significantly, which is caused by the complexity of the terrain on which the installation has been set up [6].

The fact that there are not universally optimal WPF models for all the windfarms has been highlighted by Banerjee et al. (2017), who demonstrated that the most appropriate modelling approach depends also on the evaluation criterion [7].

The achievable WPF errors entail the necessity of knowing the primary sources of uncertainty. In a paper published by Monforti et al. (2017) the authors have attempted to detect and weight the main factors affecting inaccurate wind power predictions. The lack of comprehensive knowledge of the technical parameters of the wind farms and insufficient information on the wind fields are mentioned as the two main sources of WPF inaccurateness [8].

The desire for knowing the market prices in advance with acceptable reliability has become the subject of many studies. A remarkable focus has been put on forecasting the spot market prices [9,10,11]. Bessec et al. (2014) distinguish three main modelling methods for predicting the energy prices on the market: equilibrium and game theory, simulation and time series forecasting methods [12]. The basic models for prediction of time series include smoothing methods like averaging, exponential smoothing (e.g., Brown’s, Holt’s and Winter’s methods). However, the results of these models suffer relatively high prediction errors [9].

Throughout numerous scientific publications, most attention is paid to the autoregressive (AR), moving average (MA) and their combination—ARMA—models, which have had success in forecasting economic phenomena [13]. The ARMA model can be useful in stationary processes, which are characterised by a constant value of the average in throughout the time domain. Since a lot of processes indicate a presence of a trend, the ARMA model cannot be applied directly—this problem is solved by extension of the ARMA model by including integration (the I term in ARIMA), which transforms the process with trend into a stationary one (ARIMA). Another difficulty comes from the seasonality of some time series in a short time, which has been addressed by the seasonal (SARIMA) model.

In the age of computers, artificial neural networks (ANN), a branch of computational intelligence (CI), has become increasingly popular in all range of predictions, mainly because of their forecasting accuracy and availability of dedicated software. Neural networks can be used successfully in circumstances which prevent statistical methods being applied. One main advantage of ANN is its adaptability for complex, dynamic and nonlinear relations. On the other hand, Weron (2014) emphasises that the neural networks can be susceptible to unexpected, rapid changes in the process [14].

Kolmek and Navruz (2012) performed the parallel forecasting processes for the Turkish energy spot prices utilising the ARIMA and ANN model, with the learning dataset constituting historical observations for time-period of 342 days. The latter model was achieved a smaller error, which was concluded as the main advantage of this method for statistical techniques (15.60% MAPE for ARIMA vs. 14.15% MAPE for ANN, calculated for a time horizon of 1 week) [15].

In his explicit study, Weron (2014) [14] lists the strengths and weaknesses of the electricity spot prices forecast in terms of the aforementioned methods, which have often been used separately a particular modelling approach. The weaknesses of each model are reduced though combining the modelling tools through dedicated computer software. The hybrid EPF models often result in performance improvements [16,17].

The most recent studies devoted to WPF and EPF strongly affirm the continuous necessity for proper projections of the wind power supplied to the system and their helpfulness in making decisions concerning the electricity market prices. Maldonado et al. [18], in their attempt to systematise the literature review concerning WPF, concludes that the field of improvement is sought more often in the direction of ANNs, which brings more promising results. However, what was highlighted by Santhosh et al. in paper [19], is that even the most advanced techniques do not solve the problem of generalisation capability, and do not assure suitability of a given model for individual windfarms. In the scope of the same study, Santhosh indicates the latest development of hybrid models and highlight the importance of probabilistic forecasting. The influence of RES generation forecast on the day-ahead market prices have been confirmed by the study by Frade et al. [20], which was conducted based on the Iberian market data.

3. Portuguese and Polish RES Sector and Energy Exchange Institutions

Portugal is a country characterised by a very high share of RES in the total energy mix. Poland, while representing a steady growth in this area, does not reach the Portuguese share by far. It should be stressed that direct comparison on the RES share is improper since, throughout the continent, countries have developed diverse and specific energy systems, depending mainly on available domestic resources. Other key factors determining the shape of energy systems are climatic conditions (e.g., temperature), as well as the maximum energy independence. Although Poland has seen a considerable increase of RES in total energy production, the dominant primary energy source is coal, which is one of the most CO₂-intensive electricity generation technologies. Contrary to Poland, Portugal’s energy mix is diverse, with no single dominant energy production technology [21,22].

The continuation of the European Union’s (EU’s) plans for clean and sustainable energy requires the adequate and optimal utilisation of the wide range of RES technologies to make it not only efficient, but also profitable. Poland and Portugal are countries separated by a vast distance, which results in different climatic conditions and different available sources of renewable energy generation. Taking into account the average annual wind speed, in both countries, the obtained measures are similar [23]. On the other hand, the solar energy in Portugal has a much higher potential for utilisation, with measures of the yearly solar irradiation being 1800 kWh/m² in Portugal and around 1200 kWh/m² in Poland.

The Towarowa Giełda Energii (TGE) was established in 1999, as the Polish government recognised the need to have an institution managing the energy trading on the emerging liberalised competitive market. Half a year after commission, the spot market was launched for market participants. Currently, TGE is the only company licensed by the state for managing the energy exchange. During the two recent decades, TGE expanded its area of activity into the emissions market exchange and origin certificates trading platform. In 2008, TGE introduced the Commodity Forward Instruments market, which allows it to maintain relatively constant prices across a wider time-horizon, as well as to optimise the costs of sales and purchases of energy [24].

While the TGE activities are limited to the within the country borders, the Iberian Electricity Market (MIBEL) is an example of the energy market expanded to the regional scale. It is a result of joint efforts of the Spanish and Portuguese governments. This convergence, having its origins in 2001, allows every market participant to make deals with other agents from all over the Iberian Peninsula. The integration of the hitherto separated energy systems required a series of undertakings, including the harmonisation of the electric network, laws, and economic environment. In 2004, both parts of the Santiago Agreement declared the creation of two sub-institutions, responsible for different aspects of the proper operation of the system as a whole. These were the OMI-Polo Português (OMIP), responsible for the forward market and OMI-Polo Español (OMIE), which was brought to existence to manage the spot market. The period between the Santiago Agreement and the launch of the Iberian Market (1 July 2007) was influenced by many factors, including political changes.

4. Methods

The research made in this paper can be divided into two main segments—the first one will refer to the evaluation of the wind power forecasts published for both Poland and Portugal in the European Network of Transmission System Operators for Electricity (ENTSO-E) Transparency Platform (https://transparency.entsoe.eu/), while the second part will concern the applicability of the ENTSO-E wind power forecast in the prediction models of the spot market prices.

4.1. Wind Generation Forecast Analysis

The analysis of wind power forecasts has been made by combining day-ahead wind generation forecasts and actual data, which has published by the ENTSO-E platform in hourly time steps. The data range has been assumed to be the calendar year 2016, which resulted in 8760 observations.

We have used the most up-to-date data available at the time of the study. In order to have a dataset representing the course of the entire calendar year, at the time of performing this analysis the most recent one was 2016. Since that time, the installed capacity in both Portugal and Poland has not changed significantly (see

Table 1. Wind power installed capacity in Portugal and Poland in GW (2016 and 2019).

GW	2016	2019
Portugal	5.1	5.2
Poland	5.75	5.9

) and the market environment remained unchanged. Therefore, we believe that subjecting updated data to the models would not affect the conclusions of the study.

The prediction error is expressed as the difference between the realisation of the variable and its corresponding forecasted value for a given instant t:

$$e_t=P_t-P_t^{*}$$

(1)

where $e_t$ is the absolute error of the forecast and $P_t,P_t^{*}$ represent the actual wind power in the system and its day-ahead prediction, respectively, expressed in MW.

The statistical analysis of the obtained forecasting error will cover the calculation of the basic statistical measures presented in

Table 2. Statistical measures for wind power forecast evaluation.

Statistical Measure	Mathematical Expression
Average $\overline{e}$, MW	$\overline{e}=\frac{1}{N}{\displaystyle {\displaystyle \sum }_{t=1}^N}{e}_t$	(2)
Standard deviation $\sigma$, MW	$\sigma =\sqrt{\frac{1}{N-1}{\displaystyle {\displaystyle \sum }_{t=1}^N}{\left(e_t-\overline{e_t}\right)}^2}$	(3)
Kurtosis $K$	$K=\frac{\frac{1}{N}{{\displaystyle \sum }}_{t=1}^N{\left(e_t-\overline{e}\right)}^4}{{\sigma }^4}-3$	(4)
Skewness $S$	$S=\frac{N{{\displaystyle \sum }}_{t=1}^N{\left(e_t-e\right)}^3}{\left(N-1\right)\left(N-2\right){\sigma }^3}$	(5)
MAPE, %	$MAPE=\frac{1}{N}{\displaystyle {\displaystyle \sum }_{t=1}^N}|\frac{e_t}{P_t}|·100\%$	(6)

where: $e_t$—forecast error [MW], $N$—total number of observations, t—time instance, $P_t$—actual value.

[13].

Moreover, the financial losses caused by inaccurate production planning and suffered by the wind energy suppliers in both countries will be estimated based on the ENTSO-E database. The calculations will be made using the actual and forecasted wind generation data, as well as their time-corresponding spot and balancing market prices for the calendar year 2016, in hourly time steps. The losses have been calculated assuming that the entire forecasted wind power is traded on the spot market, and the divergence between the wind forecast and actual production is traded on the balancing market. For this study, it was assumed that wind energy producers do not have any forward contracts. Additionally, wind producers rely entirely on the day-ahead plan and do not participate in the intraday market.

At this point, it has to be underlined that there are two divergent approaches to imbalance pricing in Poland and Portugal. In Poland, there exists only one, uniform imbalance market price. In contrast, in Portugal, two imbalance prices can be distinguished: the lower and upper imbalance price, depending on the imbalance characteristics (deficit or surplus).

Following the mentioned dissimilarities, the calculation of the financial loss has to be defined separately for both countries. The total loss of production plan deviation will be estimated as follows:

$$L={\displaystyle {\displaystyle \sum }_{t=1}^N}\left|E_t-E_t^{*}\right|\ast \left(I_t^U-M_t\right), for E_t-E_t^{*}<0$$

(7)

$$L={\displaystyle {\displaystyle \sum }_{t=1}^N}\left|E_t-E_t^{*}\right|\ast \left(M_t-I_t^L\right), for E_t-E_t^{*}>0$$

(8)

where: L—financial loss, $E_t$,$E_t^{*}$—actual and planned wind energy, respectively, $M_t$—spot market price, $I_t^U$—upper imbalance price; $I_t^L$—lower imbalance price (for the Portuguese case); $I_t^U=I_t^L=I_t$—imbalance price (for the Polish case); $N$—total number of observations; t—time instance.

4.2. Spot Market Prices Forecasting Models

To comparatively examine the impact of the forecasted wind power on the spot market prices in both countries, two representative modelling approaches have been selected from a wide range found in the literature: statistical (represented by the ARMAX model) and heuristic (the NARX model). For the evaluation of the usability of the wind energy as an input variable to these models, the corresponding values without the wind power information have been evaluated as well (the ARMA and NAR models). The comparison of ARMA vs. ARMAX and NAR vs. NARX using the data from Poland and Portugal will allow observation of what extent the models with additional external input perform better in two individual cases of Polish and Portuguese power markets.

4.2.1. Persistence Model

Persistence models, also named the naive models, are characterised by their simplicity. The forecasted value of model takes the value of the last observation.

$$y_{t+1}^{ }=y_t$$

(9)

4.2.2. ARMA and ARMAX—Autoregressive Moving Average Models

Box and Jenkins (1970) introduced a step-by-step methodology for modelling and estimation of time series by using the autoregressive (AR) and moving average (MA) models [25]. On the other hand, the ARMAX model is the extension of the ARMA model, utilising inclusion of the additional variable $u\left(t\right)$ in the model.

$$\begin{gathered} y\left(t\right)+a_{1}y\left(t-1\right)+\cdots +a_{na}\left(t-n_a\right) \\ =b_{1}u\left(t-1\right)+\cdots +b_{nb}\left(t-n_b\right)+\epsilon \left(t\right) \\ +c_1\epsilon \left(t-1\right)+\cdots +c_{nc}\epsilon \left(t-n_c\right) \end{gathered}$$

(10)

where: $a_{na}$—coefficients of the AR part of the model;

$c_{nc}$—coefficients of the MA part of the model;

$b_{nb}$—coefficients of the X part of the model;

$n_a,n_b,n_c$—polynomial orders;

$y\left(t\right)$—past values of the modelled quantity (here spot prices);

$u\left(t\right)$—past values of the external explaining variable (here wind power forecast).

If all the entities related to the external variable $u\left(t\right)$ were removed from Equation (10), the model becomes an ordinary ARMA model, relying only on the past values of the explained variable.

Successful adoption of the ARMAX model entails the necessity of fulfilling the particular requirements of the Box-Jenkins method, as well as following the 3-step procedure, which is: model identification, estimation of the parameters and model verification. To properly identify the model, the examined time series has to meet particular requirements. First of all, it has to be stationary (there is no trend observable and the variance is constant). If the stationarity is not confirmed, the time series has to be modified to achieve stationarity. Secondly, the parameters of the model have to be determined employing computational methods (e.g., the maximum likelihood estimation or non-linear least-squares estimation). Further, the estimated parameters have to be verified statistically to prove their significance to the model. In this particular case, the ARMAX model data requirements have been validated by the integration of the spot market prices time series. To remove the periodicity of the series, the spot prices series have been differenced by time-lag of 24 h.

The best model structure, i.e., polynomial orders, was obtained iteratively, using the looping capabilities of the software used.

4.2.3. NAR and NARX—Artificial Neural Network Models

The origins of artificial neural networks (ANN) are in 1943, when the first artificial neuron model was introduced by McCulloch and Pitts. Contemporarily, ANNs are commonly used in statistics and signal processing. Despite the strong development of this branch of science, the range of ANNs application is continuously expanding. The standard structure of the ANN consists of input units (vector of numbers provided by the user), hidden units (representing the intermediate calculations) and output units (vector of the model results) linked by particular weights. The weights reflect the importance of a given signal in the model—the higher the weight value, the more significant the input signal in the process of output determination. Therefore, the weights can take both negative or positive values [26].

The weights adjustment constitutes the learning process of the network, which, in the case of this study, has been made by use of the Levenberg-Marquardt algorithm, considered one of the most commonly used ANN learning techniques, and characterised by high effectiveness in feedforward networks training [27].

If the external signal is removed from the NARX model, it becomes the NAR model, with only the past values of spot market prices as the explanatory variables. The best model structure i.e., the number of hidden layers and number of input signals (here, the number of lags of time series) was obtained iteratively using the looping capabilities of the software used.

4.2.4. Forecasting Approaches

The data used in the models was the wind generation forecast and spot market prices recorded in hourly time steps for 2016, as shown in Figure 1 for Poland and Portugal.

Comparing both spot prices series, Poland’s energy price denotes high volatility, while the electric energy prices in Portugal are more clustered around a constant value. Moreover, a slightly increasing trend is noticeable in the Portuguese energy prices, especially in the second half of the year. Since the time series reveal periodicity, which eliminates the direct application of ARMA and ARMAX models, all forecasts will be made based on differenced (integrated) time series by 24 hours’ lag, as already mentioned.

It was decided to divide the data into two subsets: (i) learning dataset—consisting of observations from January to November 2016, which was used for the estimation of the models; and (ii) testing dataset—December 2016, which was used for verification of the models’ performance. Thus, the forecasts were made in a step-ahead manner for December 2016.

Moreover, the forecasting by all models was made by splitting the training data set into static and dynamic groups. The static approach uses a singular estimation of the model’s parameters, which are used in an unchanged form for the entire prediction horizon (1 month). On the other hand, in the dynamic approach, the model’s parameters are updated every prediction step, based on the modified learning dataset, which is shifted also by one step. As such, it was decided to test several cases which differed by the size of learning dataset. These were defined as follows:

• Static approach—fixed learning dataset—from January to November 2016 (case S1);
• Static approach—fixed learning dataset—November 2016 (case S2);
• Dynamic approach—“moving” learning dataset—preceding week (case D1);
• Dynamic approach—“moving” learning dataset—preceding 24 h (case D2).

The mean absolute percentage error (MAPE) has been chosen as the parameter to evaluate the quality of obtained forecasts, the has been selected and calculated for each of the abovementioned cases (Equation (6)), for Poland and Portugal, referred to the entire month of December 2016.

In order to have a perspective on the financial burden corresponding to the inaccuracy of the performed forecasting models, the total uncertainty of sales of the wind energy (SU) in December 2016 (744 h) has been calculated according to Equation (11), under the assumption that the entire amount of wind energy is traded on the spot market:

$$SU=MAPE·{\displaystyle {\displaystyle \sum }_{t=1}^{744}}{E}_t^{*}·M_t$$

(11)

where SU—total sales uncertainty (EUR), $E_t^{*}$—traded forecasted wind energy volume in hour t (MWh), $M_t$—spot market price in an hour t (EUR/MWh).

5. Results and Discussion

The wind power error value (expressed as the difference between the realisation of the variable and its corresponding forecasted value) has been calculated for each hour of 2016, and divided into ranges to present the error distribution on a histogram, as shown in Figure 2.

The main conclusion from the analysis of the above chart is that the wind generation forecasts are generally underestimated in both countries, since the result of subtracting the forecasted value from the actual value of power results in positive values, between 0 and 200 MW.

To get a deeper view of the characteristics of the error distribution, the statistical parameters of the examined dataset have been calculated. The statistical description has been carried out to quantitatively compare the inaccuracies of wind power forecasts for both countries, including the calculation of statistical measures, which are presented in

Table 3. Statistical description of the wind energy forecast error for Poland and Portugal in 2016.

Statistical Measure	Poland	Portugal
Average, MW	118.23	7.52
Standard deviation, MW	173.00	353.30
Kurtosis	7.17	13.02
Skewness	0.399	−0.307
Minimum, MW	−1189.00	−3659.00
Maximum, MW	1789.00	1934.00
MAPE, %	15.00	24.20

and are referred to in Section 3.

Analysing Table 3, one may conclude that both Polish and Portuguese forecasts are underestimated, because the average error values exceed 0 significantly, especially in Poland, reaching 118 MW. Although the average is much higher in Poland compared to Portugal, the Portuguese forecast is characterised by higher volatility, which is indicated by the values of kurtosis and standard deviation. Comparing the kurtosis of both forecasts’ errors, the higher tendency for clustering of the error near the average value occurs in the forecast prepared for Poland. In both cases, the extreme values of the forecast error reached gigawatts. Comparing the MAPE values, the Polish forecasts perform considerably better, reaching 15% on average. The skewness values of examined samples took the opposite sign for Poland and Portugal, which means that the asymmetry of the error distributions has the opposite direction. Comparing the wind generation forecasts error (up to 2 GW) with the overall generation rates in the countries (reaching up to about 10 GW in Portugal and 50 GW in Poland) leads to the question in what extent these forecasts influence the total energy system and energy markets.

The potential financial losses have been calculated assuming that the entire forecasted wind power is traded on the spot market, and the divergence between the wind forecast and actual production is traded on the balancing market. The results of the performed calculations are presented in

Table 4. Financial losses resulting from wind generation forecast deviation in 2016.

	Poland	Portugal
Total financial loss, EUR	1,900,592	20,812,563
Number of hours in 2016, when forecast deviation brought benefits	4510	814
Total volume of imbalanced energy, MWh	1,358,182	1,944,235
Unit cost of imbalanced energy, EUR/MWh	1.40	10.70

:

Although the total wind-generated imbalanced energy volume in 2016 is not significantly different between Poland and Portugal, the financial losses in the case of the Iberian country reached almost EUR 21 million, which was over ten times more than in the case of Poland. Although the yearly-aggregated forecast deviation brought losses, there were hours, when it resulted in positive income (L < 0). In Poland, more than half of hours in the year brought benefits from incorrect forecasts (4510 h in 8784 of total).

To directly compare the results, the unit costs of the forecast error was calculated by the division of the total imbalance by the total financial loss in 2016. As shown in Table 4, a 1 MWh planning error in Portugal resulted in EUR 10.7 of costs, which was almost 10 times larger when compared with Poland. Such a significant discrepancy comes from the imbalance pricing system, which is different in these countries. In Portugal, there are two levels of balancing market price, depending on the character of imbalance (surplus/deficiency). In Poland, there is only one universally valid imbalance price, which allows for market speculation. In the circumstance of a single, uniform imbalance price, there exists a considerable probability of obtaining income from selling energy on the balancing market instead of the spot market. The Polish wind energy seller may speculate on the difference between the spot and imbalance prices, which may encourage him to intentionally distort the production plan. In contrast, the Portuguese energy seller may expect only the negative outcomes from an inappropriate production plan.

As far as the influence of wind energy forecasts on spot prices is concerned, the forecasting results with the use of ARMA, ARMAX, NAR and NARX models are presented in

Table 5. Comparison of the auto-regressive moving average (ARMA) and ARMA with exogenous inputs (ARMAX) spot prices forecasting models.

POLAND
CASE	ARMA				ARMAX					SUARMA-SUARMAX
	PARAMETERS		MAPE	SUARMA	PARAMETERS			MAPE	SUARMAX
	p	q		(EUR)	na	nb	nc		(EUR)	(EUR)
S1	1	0	6.98%	3,448,565	1	0	0	6.97%	3,443,624	-
S2	1	0	6.90%	3,409,040	1	0	0	6.89%	3,404,099	-
D1	1	2	7.04%	3,478,209	2	2	0	6.79%	3,354,693	123,516
D2	1	0	8.23%	4,066,145	1	0	1	6.81%	3,364,574	-
PORTUGAL
CASE	ARMA				ARMAX					SUARMA-SUARMAX
	PARAMETERS		MAPE	SUARMA	PARAMETERS			MAPE	SUARMAX
	p	q		(EUR)	na	nb	nc		(EUR)	(EUR)
S1	5	4	3.66%	2,262,710	5	4	5	3.57%	2,207,070	55,640
S2	4	4	3.71%	2,293,622	5	2	5	3.57%	2,207,070	86,552
D1	3	3	4.24%	2,621,282	4	2	4	3.64%	2,250,346	370,936
D2	3	3	6.09%	3,765,002	3	5	3	7.77%	4,803,623	-

and

Table 6. Comparison of the non-linear auto-regressive neural network (NAR) and NAR with exogenous inputs (NARX) spot prices forecasting models.

POLAND
CASE	NAR				NARX					SUNAR-SUNARX
	PARAMETERS		MAPE	SUNAR	PARAMETERS			MAPE	SUNARX
	delays	Hidden		(EUR)	delays	ex.Input	Hidden		(EUR)	(EUR)
		layers				delays	layers
S1	1	5	6.71%	3,315,168	3	5	2	6.74%	3,329,990	-
S2	2	2	6.55%	3,236,118	5	5	2	6.63%	3,275,643	-
D1	2	3	7.54%	3,725,241	2	1	1	8.12%	4,011,798	-
D2	3	4	9.83%	4,856,647	4	4	4	8.78%	4,337,880	518,767
PORTUGAL
CASE	NAR				NARX					SUNAR-SUNARX
	PARAMETERS		MAPE	SUNAR	PARAMETERS			MAPE	SUNARX
	delays	Hidden		(EUR)	delays	ex.Input	Hidden		(EUR)	(EUR)
		layers				delays	layers
S1	5	1	3.74%	2,312,169	5	2	5	3.63%	2,244,164	68,005
S2	4	5	3.71%	2,293,622	4	5	3	3.64%	2,250,346	43,276
D1	1	1	3.88%	2,398,720	5	5	5	3.67%	2,268,893	129,828
D2	3	4	4.58%	2,831,479	5	4	1	4.20%	2,596,553	234,926

, respectively. Results where the inclusion of wind power forecasts led to better performance when compared to the case without wind power information are in bold text. The information of the key model parameters is also shown. Moreover, the uncertainty of the sale as defined in Equation (11) is also displayed.

To have a reference of the models constructed in this paper, the persistence model has been simulated as well. Applying Equation (9) to the validation dataset (December 2016), the persistence model has been constructed and evaluated by obtaining the value of MAPE, which took a value of 6.92% in Poland and 4.50% in Portugal.

The S1, S2 and D2 cases have not been highlighted, because in these examples the ARMAX model’s nb coefficient was zero. This is the order of polynomial composed of external variable values, which means that the ARMAX model becomes ordinary ARMA model.

Analysis of Table 5 and Table 6 show that when comparing the forecasting results globally, the Portuguese spot market prices are significantly more predictable than the Polish case using the models constructed for this study. Another finding is that usually, the dynamic cases perform worse than the static ones thus the model estimation should include wider ranges of data. Additionally, in the right-hand-side column, the difference between the sales uncertainty (SU) of the models with and without the inclusion of the wind generation forecast has been calculated. The SU value varied between EUR 55,000 and EUR 235,000, in the case of models based on Portuguese data.

Recalling the reference forecasts of the persistence model (6.92% MAPE in Poland and 4.50% in Portugal), it can be observed that in the case of Poland the obtained models rarely perform better than the persistence one, while, in Portugal, the MAPE obtained by time series models is often below 4%.

Comparing country-to-country, it can be observed that the wind generation forecast is valuable for predicting the spot prices in Portugal, because in seven out of eight cases, the “X” model returned more accurate results than its counterpart without information of wind generation forecast (ARMA vs. ARMAX and NAR vs. NARX). In the case of predicting the Polish spot market prices, only in two instances out of eight did the addition of wind production forecasts improve the model output. The cumulated simulation results show that wind generation forecasts influence the spot market prices in Portugal. In the case of the Polish market, has been shown that the forecasts do not have an impact on the spot market price. As a source of this difference, the authors consider the significant divergence in the wind generation share in the total electricity production. In the case of Portugal, the relative amount of the energy injected to the system from wind turbine generators is much higher, thus, may have a greater impact on the market behaviour.

As a general tendency, the Portuguese spot market prices were more predictable than the Polish time series. This is shown by the one-sided difference of the MAPE measure in all the cases.

Moreover, when comparing the results obtained from the four models for the individual learning dataset approaches (S1, S2, D1 and D2), a global result is that the models of “static” character, those being estimated on a fixed dataset, resulted in obtaining lower errors. As for the reason, the authors assume the instantaneous peaks of the prices occurring within a week or a day, which may significantly influence the estimation and impact the model’s prediction accuracy. In the situation when the time range of the data was wider (1 month, 11 months), this impact was mitigated.

The results answer the fundamental question of this study because as it is observed, the impact of wind power on the market prices is not unequivocal. An application of a given model with wind power variable as input occurred to be more accurate in the case of Portugal, while in Poland not.

The outcome of the study demonstrates consistency with former research on the topic. As highlighted in the study by Maciejowska et al. [28], the RES generation forecasts published on the ENTSO-E platform are encumbered with systematic errors, which gives space for future improvement in EPF. The analysis of WPF and EPF for Poland and Portugal confirmed the relevance of the forecasting error, which translated to the level of financial loss or gain, strongly amplifying the uncertainty of sales. The calculated values of MAPE for ARMA and ANN models, compared to the ones reported in the studies devoted to spot market forecasting (e.g., Kolmek and Navruz, [15]), can be found satisfactory. However, direct comparison between studies in the literature should be performed with caution. Our parallel study addressed this deficiency by comparatively analyse the separate cases of two countries, but with a data feed completely correspondent to one another in time and scope. That allowed to unambiguously evaluate the accurateness of EPF models and their receptivity for external data input as the wind power forecast.

6. Conclusions

The analysis of the wind power forecasts for Poland and Portugal resulted in significant forecasting errors, based on calculations made by the use of data from the ENTSO-E platform. Large discrepancies have been observed for both analysed countries. Analysis of the wind forecast error distribution revealed that there exists a difference in the estimation of the wind generation forecast among the countries. In Portugal, the forecast often predicted lower values than the actual ones. In the case of Poland, these forecast values overestimated the production compared to actual values. This finding opens a new research opportunity which would investigate the application of systematic error correction in overall wind production forecasts.

Calculation of the financial losses from inaccurate wind power forecasts led to the conclusion that the uncertainty of the wind energy production forecasting results in remarkable potential income losses for the wind energy producers, reaching millions of EUR in global (national) scale. As shown in this study, in the Polish case, there exists a large possibility of market speculation, since, in many instances, the sales on the balancing market were more profitable than in the spot market.

Analysis of the spot price forecasting models shows that in the case of Poland, the applicability of wind power as the input to a particular model is uncertain. On the other hand, a decrease of MAPE values has been demonstrated when adding the wind power input variable to the models based on Portuguese data, which translated into lower values of sales uncertainty of the generated wind power. These opposite observations reveal that the advantage of models including wind power information is not universal among the individual countries.