Energy markets (EMs) are a complex and continuously evolving reality, meaning that new players are emerging—chief among these are the producers of variable renewable energy (VRE)—and new challenges need to be managed—such as the ones associated with the participation of VRE producers in competitive markets [1
]. Indeed, recent years have witnessed a substantial increase of non-controllable or variable renewable energy, notably wind power and solar photovoltaic. VRE has several unique characteristics compared to those of conventional generation, including significant fixed capital costs but near-zero or zero production costs. VRE is also normally the marginal resource, since it is operated at maximum capacity (taking into account the weather conditions). These characteristics have a strong influence on the outcomes of EMs, reducing market-clearing prices [3
]. Accordingly, the research community has paid attention to the effectiveness of current market designs to determine if they are still efficient to deal with the increasing levels of VRE (see, e.g., [4
VRE typically involves significant forecast errors, which may result in large imbalances. The day-ahead market (DAM) closes normally at 12:00 p.m. (CET), and thus the bids of wind power producers need to be calculated by taking into account power forecasts computed 12 to 36 h ahead. As a result, an adjustment of the gate closure to a time closer to real-time operation seems to be important to enable a fair participation of VRE producers. The differences between the quantities of energy produced and the commitments resulting from the DAM need to be balanced in the intra-day market and/or the balancing market. At present, the participation of VRE producers in balancing markets (BMs) is still very limited, despite the technical feasibility and the (potential) motivation to operationalize such participation.
To address the issues associated with the participation of VRE producers in markets, adaptations to the current market structure as well as new elements of market design have been proposed by theorists and practitioners working on the area of competitive energy markets. For instance, the International Energy Agency points out that the physical transactions of electrical energy in power systems with high shares of VRE need to be made by considering auctions and centralized pools, and should not take into account feed-in-tariffs or other supporting schemes. The process of trading energy also needs to be improved by defining the terms of the transactions up to 30 min before real-time operation with an interval up to 10 min [6
]. This near real-time negotiation is also supported by the Clean Energy Package (Article 7), published by the European Commission [7
]. In this package, a new proposal for regulating the Internal Market for Electricity is presented, with the main goals of stimulating the global leadership of Europe in renewables, harmonizing markets rules, supporting the integration of VRE, and increasing the general welfare of consumers (see [8
] for a complete overview). Article 6 of the new proposal indicates that market operators should develop new products to accommodate the increasing levels of VRE and support demand-response programs.
Now, generally speaking, European markets typically allow bidding up to 5 to 30 min before real-time operation, contributing to reduce the imbalances resulting from VRE producers. The markets of North-America and Australia present some additional flexibility by including 5-min real-time (sub-)markets. Despite this, most real-world markets operate by considering power (MW) and not energy (MWh), thus allowing to some extent substantial deviations of VRE producers.
Against this background, this article presents a new bilateral energy contract, called short-term energy (STE) contract, and introduces two new marketplaces that may allow to reduce the imbalances resulting from VRE producers (we note that throughout the article the terms “new marketplace” and “new market product” will be used interchangeably). The main aim is to enable an active and competitive participation of VRE producers in energy markets, decreasing imbalances and the associated costs, and to some extent avoiding the waste of energy. The new contract and the design of the new marketplaces take into account the following aspects: legal basis, market time unit, minimum bid quantity, transaction time horizon, type of market participant, and the role of participants in the process of trading energy. The authors are aware of no similar market products in place in the real-world.
Furthermore, the article presents a simulation-based study to analyze the behavior (and test) the new contract and the design of the new market places in a real-world setting. The study involves the participation of both wind power producers (WPPs) and retailers in markets, who prepare bids according to different strategies. The simulations are performed with the help of the agent-based tool called MATREM (see [9
]). Six key performance indicators (KPIs) are considered, namely the value of wind energy to the market, the global imbalances of the system, the imbalances and costs of WPPs and retailers, and the total cost of the system.
The work presented here builds on our previous work in the areas of trading wind power in markets [11
] and portfolio optimization of retailers [13
]. Specifically, in [11
], we investigated the benefits of the participation of WPPs in BMs at both economic and technical levels. In [12
], we analyzed the impact of the wind power forecast uncertainty and the change of the day-ahead market gate closure on market outcomes. In [13
], we introduced a model for optimizing the portfolios of retailers using the Markowitz theory. In this paper, as noted, we present and test a contract and two marketplaces related to the participation of VRE producers in energy markets.
The remainder of the paper is structured as follows. Section 2
presents the main features of existing energy markets. Section 3
discusses the participation of VRE producers in balancing markets. Section 4
presents the new energy contract and Section 5
the new marketplaces. Section 6
summarizes the features of the MATREM system. Section 7
illustrates the trading behavior of WPPs by taking into account the new contract and marketplaces. Section 8
presents the simulation-based study and discusses the experimental results. Finally, Section 9
presents some concluding remarks.
2. Energy Markets and VRE Producers
Day-ahead markets close typically at 12:00 p.m., 12 to 36 h before physical delivery. Market participants trade energy on exchanges or pools using programs based on the system marginal pricing theory. Prices and quantities are calculated in a specific day D
for every hour of day
. Intra-day markets are essentially markets involving scheduling and pricing procedures a few hours ahead to facilitate balancing in advance of real-time. Such markets may involve various sessions based on auctions or may operate continuously (see, e.g., [14
]). Most American markets also include a short term market, generally referred to a real-time market, to set prices and schedules for 5-min periods (but see [3
Derivatives are financial instruments that include forwards, futures, options and swaps [15
]. These instruments are essentially contracts to buy or sell a specific amount of electricity at a certain future time for a specific price. They may span from days to several years and allow market participants to hedge against the financial risk inherent to day-ahead and intra-day prices [16
]. Also, they may be financial (involving a purely financial settlement) or physical (involving a financial settlement and the physical delivery of energy), and are typically traded in derivatives exchanges. In short, market participants submit orders to sell or buy electricity in an electronic trading platform. Orders include the quantity and the price as well as several other parameters that are deemed appropriate. The trading platform automatically and continuously matches the orders that are likely to interfere with each other (typically, for a particular type of contract and a specific energy price). Also, apart from derivatives exchanges, bilateral contracts—such as forwards and swaps—may be negotiated privately between two parties. The terms of such contracts are very flexible and can be defined to meet the objectives and needs of both parties (but see [17
] for a more in-depth discussion).
Balancing markets are imposed by the European Network of transmission system operators and allow to compensate the deviations from the schedules defined in day-ahead and intra-day markets, as well as in bilateral contracts. The players that deviate typically need to pay penalties. The system operators have access to reserve capacity for the provision of system services, namely primary reserve (or frequency control reserve), secondary reserve (or fast active disturbance reserve), and tertiary reserve (or slow active disturbance reserve). Primary reserve is the first to be activated, after grid disturbances or imbalances between production and consumption. It must be activated up to 15 s and the disturbances need to be controlled in 30 s. Secondary reserve should be fully activated in 30 s and can continue active for a maximum of 15 min. Tertiary reserve is activated manually, up to 15 min, and can continue active for hours (see, e.g., [8
Secondary and tertiary reserve are traded by system operators in day-ahead tenders. In short, these agents define the needs of the power system for up and down-regulation, receive the proposals of the authorized participants, and determine schedules and prices by using an algorithm based on the system marginal pricing theory. Typically, different simulations are performed for computing the price for up and down-regulation. Now, apart from bilateral contracts and derivatives exchanges, balancing markets are most important for the work described here, and the next section is devoted to the participation of VRE producers in such markets.
4. The Short-Term Energy Contract
The short-term energy (STE) contract is a new type of bilateral contract—and to some extent a new market product—that presents some similarities with the aforementioned bilateral contracts, although there are obvious differences. The specifications of the contract are shown in Table 1
. It has the goal of allowing agents to reduce/avoid imbalances and consequently the potential payment of penalties—that is, it is not a profit-seeking product. Accordingly, the energy price is pre-defined as the market-clearing price (DAM price) for the period under consideration. The minimal energy quantity is 0.1 MWh. Agents submit to a trading platform bids involving specific energy quantities for periods of 15 min (and not one hour). Bids may be new or associated with energy deviations and should be submitted up to one hour prior to real-time operation. Buy and sell bids likely to interfere with each other generate transactions and new contracts. To this end, the trading platform takes into account energy quantities only (quantities may be either fully or partially matched). Bids associated with deviations have priority over new bids and are matched according to a principle of equity. Physical delivery is done in strict accordance with transmission system operators, who are informed about the terms and conditions of new contracts (energy price, energy quantity, etc.).
Now, an important feature—and to the best of our knowledge—a novel feature of the STE contract is the inherent aspect of considering energy and not power. Figure 2
illustrates this aspect by depicting bids involving either power (green line) or energy (blue line). Bids are assumed to be simple and consist of quantities and other parameters that are deemed appropriate (e.g., match type). The settlement period has the duration of 15 min. Existing bilateral contracts consider typically a quantity based on power, meaning that power plants should follow a constant production schedule during the settlement period (green line of Figure 2
, corresponding to a quantity of 50 MW). This may not be adequate for wind power producers and other VRE producers due to the uncertainty and variability of renewable generation. Accordingly, the SET contract considers a quantity based on energy—that is, power plants do not necessarily need to follow a constant production schedule during the settlement period (orange line of Figure 2
, corresponding to an average quantity of 12.5 MWh). This typically leads to a decrease of the imbalances and the associated costs (but see the real-world study presented in Section 8
). Overall, despite the existence of a number of contracts traded in energy markets worldwide, such as daily future contracts or even 15-min base and peak contracts (see, e.g., [29
]), it is especially noteworthy that we are aware of no contracts similar to the short-term energy contract. At this stage, we note that an appealing alternative to the short-term energy contract involves the submission of both energy price and energy quantity (instead of energy quantity only). However, energy would be traded at different prices and, to some extent, some transactions would not be considered due to the mismatch of price.
6. Main Features of the MATREM System
MATREM (for Multi-Agent TRading in Electricity Markets) is an agent-based tool for simulating the behavior of competitive energy markets. In [9
], we present a detailed description of the system, and in [10
] we classify the system according to a number of dimensions associated with both electricity markets and intelligent agents. The remainder of this section gives an overview of MATREM.
The system supports a day-ahead market (DAM), an intra-day market (IDM), a futures market, and a balancing market (BM). The DAM is a central market where generation and demand are traded on an hourly basis [31
]. The IDM is a short-term market that involves several auction sessions. Both markets operate according to the marginal pricing theory and are controlled by a market operator agent. Two pricing mechanisms are supported: system marginal pricing (SMP) and locational marginal pricing (LMP). The futures market is a market to hedge against the financial risk (i.e., the price volatility) associated with the DAM and the IDM. It is an organized market for both financial and physical products, which may span from days to years. The balancing market is a market for the provision of system services. MATREM considers three types of reserve, namely primary reserve, secondary reserve and tertiary reserve. The stability of the power system is a task associated with a system operator agent, who is responsible for the operation of this market.
The system also supports a marketplace for negotiating tailored (or customized) bilateral contracts, notably contracts defined to cover the delivery of large amounts of electrical energy over long periods of time. Two types of contracts are considered: forward contracts and contracts for difference [32
]. The negotiating parties are equipped with a model that handles two-party and multi-issue negotiation. The negotiation process is an iterative process involving an exchange of offers and counter-offers [33
Market participants are modeled as software agents and include generating companies, retailers, aggregators, traditional consumers, coalitions of consumers, market operators and system operators. VRE aggregators allow the participation of WPPs and other VRE producers in the aforementioned markets. Coalitions of consumers are essentially alliances of end-use customers with the main goal of increasing their bargaining power. The system platform is a 32/64-bit computer running the JAVA programming language and the JADE platform [36
7. Trading Behavior of Wind Power Producers
As noted earlier, most existing energy markets, including the Iberian electricity market [37
], were not designed to deal with large levels of non-dispatchable generation. MIBEL includes a day-ahead market and an intra-day market, managed by the Spanish electricity market operator (OMIE [38
]), as well as a derivatives market, managed by the Portuguese electricity market operator (OMIP [39
]). Market participants submit hourly bids to trade energy in the day-ahead market. The IDM involves six auction sessions, with gate-closures between 1 and 5 h ahead of real-time operation. The derivatives market allows private parties to trade standardized bilateral contracts.
shows the typical trading behavior of WPPs in the DAM and the IDM, which takes into consideration the most reliable meteorological data, in order to minimize deviations. WPPs submit wind power forecasts in the day-ahead market during day
, and make commitments to produce specific quantities of energy for day D
. The bids to submit to the intra-day market are essentially deviations—that is, they are computed by taking into consideration the actual commitments and the current updates of the wind power forecasts.
In this work, the forecasts are obtained by considering the Neural Network Toolbox and the artificial neuronal networks (ANN) approach [40
]. ANN is typically organized in layers, namely the input, hidden, and output layers. Each layer covers one or more nodes that represent the basic unit of information process (also known as the neuron). The main advantage of the ANN is the capability to deal with nonlinear relationships by learning the association between the predictors and predictions.
Numerical weather prediction (NWP) models present systematic errors, which are mainly associated with the following: (i) initial and boundary meteorological conditions (ICs), (ii) a poor representation of the physics of the models, and (iii) a failure to solve sub-grid scale phenomena, such as the sea-land interaction. The data to feed the NWP model to enable wind power producers to participate in the day-ahead market are based on ICs from 06:00 UTC, representing a time horizon for the meteorological forecast ranging between 18 to 42 h ahead. For the case of the intra-day market, the time horizon for the forecast ranges from 2 to 7 h (depending on the auction session).
The meteorological data (ICs) are updated every six hours, namely at 6:00, 12:00, 18:00 and 24:00 (see the small light-blue arrows of Figure 4
). This data needs to be processed to get the wind power forecasts, involving a computation time of nearly 2 h (see the dotted green lines of Figure 4
). The updated forecasts are represented by the vertical orange arrows, meaning that the bids of WPPs to intra-day sessions 1, 3, 5 and 6 are based on updated data, and the bids to the other sessions take into account the available data only. Also, the bids of WPPs for a particular intra-day session involve the first hours of that session only—that is, the hours until the beginning of the next session (see the horizontal dark yellow arrows of Figure 4
). Such bids include a quantity and a price. The quantity refers to the excess of energy. The price is 0 €/MWh.
Overall, WPPs submit bids to the day-ahead market, make commitments to produce energy during day D, and submit their deviations to the various sessions of the intra-day market. After that, in case WPPs still expect deviations, they can make use of the new bilateral energy contract to eventually reduce the imbalances. Following this, WPPs can participate in the aforementioned marketplaces. At the end, in case WPPs still expect deviations, they need to assume their balance responsibility and (eventually) pay penalties for their imbalances.
8. Simulation-Based Study
This section presents a simulation-based study carried out by using the MATREM system. It involves the simulation of the day-ahead and intra-day markets, as well as the use of the new contract and the simulation of the new marketplaces.
8.1. Data, Agents and Scenarios
The following sources of data are considered: (i) hourly prices and quantities of the day-ahead and intra-day Iberian markets (data published by the Spanish electricity market operator, OMIE [38
]), (ii) hourly prices and quantities submitted to the Portuguese balancing market (data reported by the Portuguese system operator, REN [41
]), and (iii) hourly deviations and prices of the imbalances for producers and retailers (data reported by REN [41
]). Also, the study makes use of real wind power data from a set of wind parks located in the central region of Portugal, which is available from 1 January 2009 to 31 December 2010.
The (software) agents are 12 producers (with several production units), representing the supply-side of Portugal, five retailers, representing the demand-side of Portugal and Spain, and one aggregated wind power producer, with 249 MW of installed capacity (representing 10% of the Portuguese installed capacity in 2010). To get expressive results, the data was upscaled to 2490 MW of installed capacity, by multiplying all values by a constant factor.
presents several key features of the producer agents. As noted earlier, the forecasts for the day-ahead and intra-day markets (time horizon ranging from 18 h to 42 h, in case of the day-ahead market, and from 2 to 7 h, for the intra-day market), were obtained by considering a numerical weather prediction model coupled with an artificial neuronal networks approach. Both the numerical prediction model and the artificial neuronal approach were calibrated for the region under consideration. The normalized root mean square error of the day-ahead forecast is around 13.5%. For the new products, the wind power forecasts were obtained using the historical time series and the ANN approach.
The study involves the following seven scenarios:
S1: Wind power producers participate in the day-ahead market only (baseline scenario);
S2: WPPs participate in both the day-ahead and the intra-day markets;
S3: WPPs participate in the DAM and the IDM, and make use of the STE contract; also, retailers and conventional generation participate in the DAM and the IDM, and make use of the STE contract;
S4: WPPs participate in the day-ahead and intra-day markets, as well in the energy reserve marketplace;
S5: WPPs participate in the DAM and the IDM, as well in the renewable power band marketplace;
S6: WPPs participate in the DAM, the IDM, and the renewable power band and energy reserve marketplaces;
S7: WPPs participate in the DAM, the IDM, use the STE contract, and participate in the renewable power band and energy reserve marketplaces; also, retailers and conventional generation participate in the DAM and the IDM, and use the STE contract.
For all scenarios, we consider that the agents assume their balance responsibility, including the wind power producers, meaning that all agents pay penalties for their imbalances. To quantify the relevance of both the new contract and the new marketplaces, several important parameters are simulated, notably:
Deviations of wind power producers and retailers;
Deviation costs of wind power producers and retailers;
Wind energy value to the market.
8.2. Results and Discussion
and Table 6
, and Figure 5
and Figure 6
summarize the results obtained by simulating the behavior of the day-ahead and intra-day markets with the help of the MATREM system, as well as considering the new contract and the new marketplaces. In particular, Table 5
shows the main results for the wind power producers, namely the energy value, the deviations and the remunerations. The table indicates that the STE contract (scenario S3) substantially reduced the deviations of the wind power producers, namely 56% in relation to the base scenario (see also Figure 5
). In this way, the use of this contract decreased the cost of the deviations of WPPs by nearly 59% (see also Figure 6
). However, WPPs were remunerated by the day-ahead clearing price, which was normally less attractive than the price of the tertiary reserve market (mFRR market). The participation of WPPs in the energy reserve marketplace (scenario S4) also increased the wind energy value (16% in relation to the base scenario S1). This result is explained by the high remuneration received by WPPs in the mFRR market, for their excess of energy (according to the actual operation of the Iberian market). In relation to the participation of WPPs in the renewable power band marketplace (scenario S5), they obtained a high levelized remuneration from energy. However, in comparison with the other scenarios, WPPs trade a lower quantity of energy, which results in a lower remuneration, and a higher cost with deviations.
indicates that the use of the STE contract and the participation of WPPs in the energy reserve marketplace (scenarios S3 and S4) are particularly important, compensating their real-time lack of energy (i.e., wind power forecast underestimations). An analysis of the fifth column of the table shows that WPPs need to buy energy to fulfill their commitments with the DAM and the IDM. For the case of scenario S5, involving the participation of WPPs in the renewable power band marketplace, the benefits are not substantial for situations involving wind power forecast underestimations. In such situations, WPPs receive money for their excess of energy, instead of paying for their lack of energy.
The sixth column of the table reflects to some extent the price of the secondary reserve market (aFRR market). WPPs receive a higher remuneration for power (scenario S5, corresponding to 29.06 €/MW), in comparison with the remuneration resulting from scenario S7 (28.83 €/MW), involving the new contract and the participation in the new marketplaces. This result is associated with a reduction in deviations that is not attractive, leading to a reduced power band (and bid).
Overall, the results of Table 5
show that the use of the new market products are beneficial to wind power producers. Accordingly, both market operators and independent system operators should take them into consideration, possibly with particular adaptations and/or extensions, in order to allow an effective participation of VRE producers in liberalized markets—that is, to provide the flexibility needed to integrate the increasing levels of renewable generation, maintaining the security of the power systems.
shows the simulation results for the particular case of retailers. These agents use the STE contract only. The results indicate that the retailer agents reduce the deviations in 39% and the costs with deviations in 38%. Also, the results for scenario S7 show that the use of the STE contract and the participation in the new marketplaces by WPPs, as well as the use of the new contract by retailers, result in a significant decrease in the global imbalances (
) and the associated costs (
This article presented a new bilateral energy contract (STE contract) and new marketplaces designed to allow wind power producers to (eventually) reduce their deviations. The contract and the marketplaces were to some extent tested in a real-world study conducted with the help of the multi-agent system MATREM.
The software agents were 12 producers, representing the supply-side of Portugal, five retailers, representing the demand-side of Portugal and Spain, and one aggregated wind power producer. The data involved hourly prices and quantities of the day-ahead and intra-day Iberian markets, hourly prices and quantities submitted to the Portuguese balancing market, and hourly deviations and prices of the imbalances for producers and retailers. Seven scenarios were considered, involving the participation of wind power producers in the new marketplaces, and also using the new contract.
The simulation results indicated that the new contract and the new marketplaces are beneficial to WPPs at both technical and economical levels, reducing the imbalances and increasing the wind energy value to the market. Also, the STE contract led to a reduction in the imbalances of retailers, resulting in a reduction of their costs with imbalances. Accordingly, market operators and system operators should take into consideration the new products presented here, possibly with particular adaptations and/or extensions, to allow an effective participation of VRE producers in liberalized markets, thus contributing to a new paradigm involving no feed-in tariffs nor other supporting schemes—that is, the paradigm expected in the near future, involving near 100% renewables.