The Clean Energy Package and Demand Response: Setting Correct Incentives

Abstract

We describe how recent EU regulation affects demand response (DR) and highlight some of the remaining regulatory challenges from a legal and economic viewpoint. With the Clean Energy Package (CEP), the EU has opted for a fully market-based, consumer-centered approach for DR. The development of business models and products is left to a large extent to market forces. However, to enable the efficient development of those DR markets, network regulation has to adapt. (1) Network tariffs have to become more cost-reflective to provide correct incentives to market participants. The capacity tariffs have to increase, net-metering should be abolished, and optional tariff components for providing flexibility may need to be considered. (2) The regulation for distribution system operators (DSOs) may need to be fine-tuned to reflect their new roles. We present three scenarios: (a) a horizontal merger of unbundled DSOs under incentive regulation, (b) a DSO as a subsidiary of an integrated utility under cost plus regulation, (c) a transfer of some activities from DSO to TSO.

1. Introduction

Effective demand response (DR) can play an important role in future electricity markets (see [1] for a review). More elastic demand can limit the size of price spikes, reduce the need for investments in network and generation capacity as shown by a number of prospective simulation models. For instance, using a two-stage unit-commitment model calibrated to the Belgian market [2] shows that DR lowers costs, increases reliability, and lowers emissions. Reference [3] estimates that the monetary benefits of DR in the form of demand shifting for the Midcontinent Independent System Operator (MISO) are between $1 million and $17 million. For the EU27 system, ref. [4] simulates a capacity expansion model for the year 2050 with and without DR. It is estimated that DR increases investments in the low-voltage grid with €800 billion but leads to a reduction of €6200 billion in generation capital and operating costs. Adjusting a standard expansion program with own-price and cross-price demand elasticities and calibrating for the Danish electricity market, ref. [5] shows that DR reduces the weighted average electricity price. The previous papers all used a bottom-up model for the electricity market. Using a computational general equilibrium model (CGE) calibrated for Spain, ref. [6] shows that DR increases household welfare but lowers GDP as producers are hurt by a reduction in demand. Reviewing existing studies [7] the US Department of Energy concluded that the estimated benefits of demand response vary considerably and differences are driven mainly by analysis methods, assumptions regarding customer participation, and market characteristics. In a Stackelberg model with retailers and consumers, ref. [8] shows that how consumers and retailers share the additional surplus created by DR will depend on the strategic setting. More elastic demand can also limit the size of price spikes, eliminate the exercise of market power, and provide ancillary services. Reference [9] models the Italian wholesale electricity market as an oligopoly market and quantifies the effect of two DR programs, while [10] shows that DR can significantly reduce the cost of reserve provisions using a simulation model for the German power market.

A number of technological developments and the further decarbonization of the energy system have increased the scope for DR. With the emergence of electric vehicles and heat pumps, households can become providers of flexibility. Using a unit commitment model representing the Illinois power system in 2020, ref. [11] shows that managing the charging of plug-in hybrid electric vehicles (PHEV) in a DR program can significantly reduce the total operating cost of the electricity system. A similar study for NYSIO [12] shows that controlled charging will reduce the cost of the integration of PHEVs, especially if there is a lot of intermittent wind generation. Reference [13] assesses the impact of DR by residential heat pumps on operational costs and emissions under different tariffication schemes. In the best case scenario, with optimal coordination, operational cost reduction between 0.9% and 5.5% can be achieved, but simple price signals might be insufficient. For the Danish system with a wind power penetration above 25%, ref. [14] shows that heat pumps with intermediate thermal storage are more cost-effective than electric boilers because they can offset intermittency generated by wind power.

At the system level, traditional large-scale fossil plants are being phased out and replaced by intermittent renewable generation. In the long run, this increases price volatility and the value of flexibility and DR. However, small amounts of photovoltaics (PVs) might reduce price volatility. Using a GARCH model and a GARCH-in-mean model respectively for German electricity prices [15,16] show that average prices decreased, but volatility increased with larger penetration of intermittent wind energy. Those results may depend on the specific merit order of conventional power plants in Germany and the time frame over which volatility is measured. Reference [17] compares the effect of wind power on Danish and German prices. Weekly volatility increases with wind energy in Germany and Denmark, but daily volatility decreases in Denmark, while it increases in Germany. For Italy, estimates of quantile regression models [18] show that both wind power and PV increased volatility although large price spikes were reduced. The results for PV are however less robust.

Energy flows at the distribution level have become less predictable and bottlenecks arise. Managing congestion at the distribution grid requires local coordination. Reference [19] derives distributional locational marginal pricing (DLMP) to deal with congestion management in distribution networks, and applies it two case-studies: the high penetration of electric vehicles (EVs) and heat pumps (HPs). Reference [20] proposes a price-based coordination mechanism between fleet owner of EVs and the DSO to manage congestion on the distribution network. Reference [21] discusses the functioning of a flexibility clearing house (FLECH) for managing congestion on the Danish DSOs. The goal of this market is to reduce feeder overload and feeder voltage problems, by procuring flexibility from DERs.

Although promising, DR has not developed as foreseen in the International Energy Agency’s (IEA) Sustainable Development Scenario (SDS), as current growth rates are 10 times smaller than predicted [22]. Moreover, field experiments and empirical analysis show that DR programs can increase demand elasticities but few of those experiments have outgrown demonstration phases. Ref. [23] reviews several programs of DR worldwide, and shows that many projects remain small scale, are currently limited commercially viable and might be hard to scale up. Several studies show that consumers are price-sensitive, but those studies often focus on specific interventions that are not necessarily part of a DR program. Reference [24] represents the results of a dynamic pricing experiment for households in the District of Columbia, and shows that the dynamic pricing programs provide stable, predictable, and sizable demand reductions. Using a natural experiment [25] estimates that consumers adjust demand to their average price and not the marginal price, and finds medium long-run price elasticities of −0.08 for California. For Germany [26] estimates demand elasticities of −0.52, using an instrumental value approach. Demand elasticity is larger for informed consumers.

The IEA identifies regulatory barriers in Europe as a reason for the slow development of DR [22]. A good regulatory framework is a prerequisite for the efficient development of the DR. In this paper, we discuss the specific provisions in the EU Clean Energy Package (CEP) that affect DR, most notably the provisions regarding market access for consumers. We then look at the need for improving network tariffs and the incentives for DSOs. This regulatory framework is a building block for the efficient development of DR.

The success of DR will further depend on the development of accessible marketplaces for energy and flexibility, and innovative business models that engage consumers. Reference [27] reviews the literature on the structure and the challenges of future retail electricity markets, and highlights the role of different market actors, how such markets could be organized and simulated numerically. Reference [28] extensively reviews demand-side business models for energy efficiency and DR and identifies values related to ancillary services (frequency, interruptible load, operating reserve), capacity markets, and direct participation in the wholesale market. It also considers synergies with the generation, transmission, distribution, and retail segments. Reference [29] compares the possible market design options for developing demand response, using a simulation model calibrated to the French power system. Its main focus is whether the current market design permits the merchant development of demand response and smart metering, and concludes that the capacity market is the solution that fits the requirements of the DR operator. Reference [30] presents a new market concept, the DR exchange in which market operators, TSOs, DSOs and retailers participate. Reference [31] reviews 55 studies on DR with residential consumers and summarizes the drivers for different levels of engagements. Financial motivations, trust, perceived risk, and complexity were seen as major factors. Socio-demographic elements only have mixed effects.

The European Commission (EC) recognizes the importance of end-users and energy communities in future energy markets and the regulatory barriers that preclude the development of DR. Therefore, as part of the CEP, the EC decided to overhaul the existing market design. Our first contribution (Section 2) is to highlight the regulatory conditions for DR within the European context. The examination of the CEP shows that the EC has resolutely chosen for a fully market-based approach: it gives far-reaching market access rights to consumers, allows for new market participants (independent aggregators, and energy communities), and introduces real-time energy prices for end-users. All organized electricity markets (capacity, forward, day-ahead, intra-day, balancing, ancillary services) will have to be adapted for those access requirements.

The European Commission also recognizes the important role that the DSOs will play in this process. The second contribution of the paper is to discuss the regulatory boundary conditions for the DSO. For DR to function effectively, tariff structures need to be designed such that they do not distort incentives. Section 3 highlights possible improvements of network tariffs.

In the past, DSOs were mainly responsible for investing and maintaining their distribution network. In the new market design, they will play a more important role as a buyer of local flexibility services, as a market maker for local energy markets and as an interlocutor with the Transmission System Operator (TSO) for system-wide ancillary services. Section 4 discusses how the DSOs, or other market actors, can be incentivized to take up this role. A straightforward implementation of cost-plus regulation might be insufficient.

2. The Clean Energy Package—Demand Response

European energy markets have been liberalized since the second half of the 1990s [32,33]. European Regulations and Directives provide the regulatory framework for the internal energy market, which are then implemented by the Member States. They have been revised regularly, and the EC has just finished its third overhaul, the ‘Clean Energy for all Europeans’ package also known as the Clean Energy Package (CEP). In this section, we discuss how EU regulation affects energy consumers, and highlight some recent developments.

2.1. From Passive Consumers

The focus of the first three energy packages (1996–2009) was on founding the internal energy market: the introduction of competition for generation and supply, ensuring non-discriminatory access to distribution and transmission networks, improving cross-border trade, and establishing a governance structure (e.g., national and international regulators). The position of small energy consumers—households and small and medium-sized enterprises—has only gradually received attention. Where earlier regulations focused on consumers as rather passive agents requiring protection, newer iterations view consumers more as active market participants.

Consumers were still assumed to be rather passive in the Second Electricity Directive (2003/54/EC), which obliges the Member States to take measures regarding customer protection and public service obligations. It also allowed Member States to introduce measures for Demand-Side Management (DSM). The original goal of DSM was to improve energy efficiency on the demand side, thereby reducing the need for additional investments in production or networks. DSM already existed in the pre-liberalization period as a requirement for the regulated vertically integrated utilities. With the liberalization, those responsibilities could be given to the regulated and partially unbundles DSOs. It was defined (Art. 2.29) as a global or integrated approach aimed at influencing “the amount and timing of electricity consumption, to reduce primary energy consumption and peak loads by giving precedence to energy efficiency, or other measures”. The system operators are well placed to trade-off network investment and the cost of DSM projects. Moreover, as they are typically not selling energy products, the implementation of DSM measures by system operators does not distort competition. However, DSM only provides limited incentives for innovation of business models by suppliers.

In the recast Electricity Directive, DSM is no longer mentioned. Instead, the focus has fully shifted towards DR, and end-consumers are placed central to the discussion.

Although DSM is no longer seen as a tool to provide demand flexibility, the Energy Efficiency Directive [34] allows member states to impose Energy Efficiency Obligations (EEOs) on energy distributors and retail energy sales companies. The EEOs focus on overall energy efficiency, but not on flexibility. In 2015, five countries used such schemes: Denmark, France, Italy, Poland and UK [35].

2.2. To Responsive Market Participants

Since the Energy Efficiency Directive (2012) (Art. 15), energy consumers have been treated as active participants, whose role has been further developed by the CEP [34]. The CEP consists of a new energy rulebook which consists of four Directives and four Regulations. The most relevant for DR are the following:

(1)

The amending Directive on Energy Efficiency (2018) [36]

(2)

The new Electricity Regulation (2019) [37]

(3)

The amending Directive on Electricity (2019) [38]

The CEP brings forth an updated market design for electricity markets and introduces new measures regarding dynamic pricing, the market access of DR, the role of aggregators and energy communities, and the regulation of TSOs and DSOs.

Central to this development is the promotion of DR. It is defined in the recast Electricity Directive (Art. 2.20) as, “the change of electricity load by final customers from their normal or current consumption patterns in response to market signals, including in response to time-variable electricity prices or incentive payments, or in response to the acceptance of the final customer’s bid to sell demand reduction or increase at a price in an organised market […] whether alone or through aggregation”.

Important in this definition is that consumers react to market signals and that their response reflects short term deviations from normal consumption levels. The definition distinguishes implicit DR, where consumers observe prices and adjust their demand accordingly, and explicit DR, where consumers—possibly through intermediaries—bid into organized markets and participate directly in the price formation process.

This definition of DR is much more precise and foresees a much larger role the final consumer than the 2012 version of the Energy Efficiency Directive [34] which defined it as “a mechanism to reduce or shift consumption to improve energy efficiency.”

2.3. Implicit DR: Correct Prices

The first set of measures aims at increasing implicit DR. To implement this, consumers should be faced with dynamic prices, namely time-varying energy prices and transmission tariffs that reflect market scarcity, and smart metering systems that continuously measure energy consumption by individual consumers.

Annex XI of the 2012 Energy Efficiency Directive [34] states that network or retail tariffs may support dynamic pricing for DR measures by final customers, time-of-use tariffs, critical peak pricing, real-time pricing, and peak time rebates. The recast Electricity Directive [38] requires that the national regulatory frameworks enable suppliers to offer dynamic electricity price contracts and that the Member States ensure that final customers with smart meters installed can request to conclude a dynamic electricity price contract. The Directive [38] defines dynamic electricity price contracts in article 2.15 as “an electricity supply contract between a supplier and a final customer that reflects the price variation in the spot markets, including in the day-ahead and intraday markets, at intervals at least equal to the market settlement frequency”.

Article 19 of the recast Electricity Directive [38] requires Member States (or national regulatory authorities) to strongly recommend “electricity undertakings and other market participants to optimize the use of electricity, inter alia, by providing energy management services, developing innovative pricing formulas, and introducing smart metering systems that are interoperable, in particular with consumer energy management systems and with smart grids […].” Member States are required to ensure the implementation of smart metering systems that assist in customer participation in their territories, possibly subject to a cost-benefit assessment. A cost-benefit assessment should be revaluated at least every four years (preamble, 53).

2.4. Explicit DR: Consumer Access

The second set of measures focuses on explicit DR, where consumers and small businesses can directly participate in energy markets. The principle of market access of consumers through aggregation, or individually, was established in the 2012 Energy Efficiency Directive [34]. Article 15.8 required Member States to encourage demand-side resources (DSR) to participate alongside supply in wholesale and retail markets, and to ensure that TSOs and DSOs treat DR providers, including aggregators, without discrimination on the basis of their technical capabilities.

However, member states were slow operationalizing market access for demand-side resources. Regulatory barriers (measurement, verification, and payment schemes) remained and hampered growth [39,40]. Specific support measures were therefore introduced by the new Electricity Directive [38], which define the roles of aggregators and energy communities.

Note that consumers can participate in all organized energy markets as defined in [41]. It covers any system in which multiple third-parties buying and selling interests in energy products are able to interact in a way that results in a contract. So, this includes not only the day-ahead and the balancing markets but also the ancillary services markets and capacity markets, unless technical limitations prevent it. Organized energy markets also include all derivatives products (forward and option markets) and contracts offered by independent brokers.

2.5. Explicit DR through Aggregation

In article 2.18, the Electricity Directive [38] defines aggregation as “a function performed by a natural or legal person who combines multiple customer loads or generated electricity for sale, purchase or auction in any electricity market”, while article 2.19 defines an independent aggregator as “a market participant engaged in aggregation who is not affiliated to the customer’s supplier”. Member States are required to ensure that in producing ancillary services, TSOs and DSOs treat market participants engaged in the aggregation of DR in a non-discriminatory manner alongside producers on the basis of their technical capabilities.

Importantly, member states are required to provide aggregators with the right to enter electricity markets without consent from other market participants, to have non-discriminatory, transparent rules that assign roles and responsibilities to all electricity undertakings and customers, to make rules about data-exchange between market participants and finally, to establish a conflict resolution mechanism between market participants.

The Directive foresees that aggregators may be required to pay financial compensation to other market participants (e.g., retailers), or to the market participants’ balance responsible parties, directly affected by DR activation. However, the compensation is not allowed to limit market entry or flexibility and is thus limited in article 17(4) of the Directive. This is an important requirement as aggregators may, depending on the market design, impose negative externalities on suppliers. For example, aggregators may “cherry-pick” consumers with demand profiles which are more favorable than the profiles of the average consumer [42,43].

Market participants engaged in aggregation will be financially responsible for the imbalances they cause in the electricity system (17.3(d)). However, there is also a requirement for a “provision for final customers who have a contract with independent aggregators not to be subject to undue payments, penalties or other undue contractual restrictions by their suppliers;” in art. 17.3(e).

Similar requirements to facilitate aggregation of distributed demand and supply are put forward in the Electricity Regulation [37]: it states in its principles (article 3(e)) that consumers’ and small enterprises’ market participation must be enabled by aggregation of generation or consumption. The same Regulation’s article 6.1 indicates that all market participants should have access to balancing markets, either individually or through aggregation.

2.6. Explcit DR and Energy Communities

The recast Electricity Directive defines ‘citizen energy communities ’(CEC) in article 2.11 as a legal entity based on open, voluntary participation and controlled by members or shareholders who are natural persons or local authorities. A CEC’s primary purpose is providing environmental, economic, or social benefits to the members or shareholders of the community or to the local area where it operates. This community may engage in generation. Member states may grant CECs the right to manage distribution networks in their area of operation and establish the relevant procedures. (Art. 16.4).

Article 16.3 of the Directive obliges the Member States to ensure that CECs can access all electricity markets, are treated in a non-discriminatory manner, are financially responsible for the imbalances they cause in the electricity system, and that they are treated like active customers in accordance with article 15.2(e). According to article 16 of the Directive, Member States are required to set a legal framework that ensures that participation in these CECs is voluntary, protects the shareholders’ rights, and ensures that shareholders, or members, are allowed to leave such a community.

If a CEC manages a distribution network, then it is entitled to make agreements with the relevant DSO or TSO to which its network is connected. The community then has to pay appropriate network charges at the connection points between their network and the distribution network outside the community. It is upon the Member States to determine how those network charges will be determined in practice. The same article (16.4(c)) requires that customers connected to the distribution network operated by a community shall not be harmed or discriminated against.

Hence, CECs could in some member states (partially) fulfill the roles of DSOs (operate a local network) and suppliers. In order to become a CEC, the community needs to be controlled by natural persons or local authorities. It does not necessarily have to participate in distributed generation, and it can generate environmental, economic, or social benefits to the community or the area in which it operates. Note that there are no explicit requirements for members to be physically close to each other—however, the scope of the community has to be local. For-profit communities are not strictly ruled out and neither are private companies with minority shareholders in the community. Although communities are allowed to manage the distribution network, network ownership is not explicitly covered under the definition.

The Renewable Energy Directive [36] Art 2(16) defines a slightly narrower concept: a ‘renewable energy community’ (REC). As the CECs it is a legal entity with open and voluntary cooperation, effectively controlled by members or shareholders (natural persons, SMEs, or local authorities) and which whose primary purpose is not financial profits but rather environmental, economic, or social community benefits. RECs have to be located in the proximity of the renewable energy projects that are owned are developed by the legal entity, and the renewable energy community can produce, consume, store and sell renewable energy, share renewable energy within the community, and access all suitable markets.

2.7. Increased Role of DSOs

The CEP also adopts regulation in recognition of the larger role that DSOs are expected to play in the future. The recast Electricity Directive affirms the larger set of tasks of DSOs in chapter IV. Article 32 of the Directive obliges the Member States to provide the legal framework required to allow and incentivize DSOs to procure services in order to improve efficiency in the distribution system. The same article requires regulatory frameworks in the Member States to enable DSOs to procure services from resources such as DR in transparent, non-discriminatory manners. DSOs are required to define standardized market products of the services acquired in ensuring effective participation of all market participants, including DR. Distribution system operators shall cooperate with TSOs for the effective participation of market participants connected to their grid in retail, wholesale, and balancing markets according to article 31.9.

According to the preface of the Regulation, DSOs may require regulatory safeguards to ensure neutrality in their functions, since they may often be vertically integrated companies that are also involved in the supply or other services. The Regulation states that the EU DSO is needed to improve the efficiency of the electricity distribution networks within the EU and to ensure cooperation with TSOs and the European Network of Transmission Systems Operators (ENTSO) for Electricity.

Article 57 of the Electricity Regulation requires DSOs and TSOs to cooperate in planning and operating their networks, in particular exchanging information and data, and they must cooperate to achieve coordinated access to resources such as DR. Article 28 of the same regulation requires TSOs to cooperate through ENTSO for Electricity at Union level. Article 30 obliges TSOs to establish regional cooperation within the ENTSO for Electricity and specifies that the regional coordination centers shall complement the role of TSOs.

According to article 52 of the new Electricity Regulation, DSOs shall cooperate through a European Entity for DSOs—“EU DSO entity”—in order to promote the completion and functioning of the internal market in electricity and optimal management and coordinated cooperation of DSOs and TSOs. DSOs who wish to participate shall become registered members of the entity. Article 55 describes the rather extensive tasks of the EU DSO which vary from coordinating operation and planning of transmission and distribution networks, facilitation of integration of renewable energy resources and distributed generation and direct and indirect DR, to the digitalization of distribution networks (smart grids and smart meters), cybersecurity and data management.

3. Network Services Tariffs

In the next two sections we study how regulation of distribution networks might affect the development of DR. We start with the network services tariff and look then, in Section 4, at the incentive regulation of network operators.

Network operators are regulated monopolies that recoup their costs by charging network users. In the past, tariffs for residential customers consisted mainly of a flat energy charge proportional to the volume taken from the network (€/KWh) and a small connection charge (€/connection). The EC [44] indicates that in Europe the energy component is 69% of the final network bill. It is broadly recognized that the current tariff structure may not be optimal in a smart energy system and that tariffs should become more directly linked to costs. One suggestion is that the size of the energy volume component needs to be reduced and that the capacity component (€/KW) should become more important. CEER [45] reports: “Both the European literature review and the answers to the EC public consultation on Energy Market Design (question 15) indicate a general support for a move towards capacity-based charging, with the option of a hybrid of capacity and consumption based charging to incentivise a change in consumer behaviour.” See also [44,46].

However, more advanced tariff structures have become feasible in a smart electricity network: tariffs can become dependent on time and location and can change in response to local network congestion. In the next subsection we highlight some of the trade-offs in setting tariffs and derive some recommendations.

3.1. Tariff Principles

The determination of network tariffs typically involves a trade-off between objectives. Tariffs should among other things:

• Lead to economic efficiency: Prosumers have the right incentives to invest in and operate DER. Hence efficiency includes both static and dynamic efficiency.
• Satisfy the budget constraints of the network operator and provide it with a stable income.
• Be equitable and fair. This could mean that the largest shoulders bare the largest costs, that network users do not receive undue cross-subsidies, that the risks and benefits are well shared between the network operators and network users, or that consumers in similar situations are treated similarly.
• Be practically and politically implementable and not too complicated to administer.

Other (and longer) lists with more subtle variations of different criteria can be made, but we believe that this list represents some of the main trade-offs [44,47,48].

Those objectives are often conflicting. For instance, economic efficiency requires that network services be priced at marginal costs. However, network operators are natural monopolies: the average cost of providing network services decreases with the amount of energy distributed. Hence under marginal cost pricing, the network operator will incur losses. This violates the second condition of satisfying the budget constraint.

Marginal cost pricing in its most orthodox implementation following peak load pricing principles [49,50] implies different prices for each location and each moment in time and prices that vary minute to minute according to the level of local congestion. This requires real-time meters to be present, which is not yet the case in all Member States. It may also lead to different prices for households that are connected to different feeder lines, even if they are located in the same city. This could be seen as unequal treatment, and therefore unfair. It is also unclear whether such an extremely granular tariff structure will provide meaningful economic incentives and satisfy the budget constraint. Different fairness criteria may conflict as well. Protecting vulnerable consumers might require cross-subsidies between consumer groups, which leads to inefficient pricing.

Pollitt looks at four different pricing principles (cost-reflective pricing, traditional public service pricing, platform market pricing, and customer-focused business model pricing) and highlights how they all reflect different objectives and trade-offs [48]. It is therefore unlikely that a one-size-fits-all approach will work for the EU.

3.2. Existing Rationale for a Volume-Based Tarrif

The rationale for the historical tariff system, where tariffs are mainly based on the volume of energy consumption, was that it provided a good balance between fairness, efficiency, and feasibility. Efficiency requires that prices correspond to the marginal cost of energy services. Lacking smart meters, short-run marginal costs were hard to obtain, so long-term marginal costs were used. Those long-term marginal costs depend on the additional investment required to cope with growing peak demand. Hence, ideally, peak demand (i.e., the capacity demanded by a household precisely at moments with the highest aggregate network flows) should be priced. A capacity tariff (€/kW) would achieve this goal. As load profiles for residential consumers used to be very similar, charging consumers based on electricity volume was a good enough proxy for the peak capacity requested by each consumer. Pricing at marginal costs would break the budget constraint of the network operator, and some remaining unrecovered costs would still need to be paid for by consumers. This was done by increasing the tariff above the long-term marginal cost. By setting a mark-up on a volume-based tariff, richer consumers with larger consumption would pay a greater share of the unrecovered costs, hence the broadest shoulders would carry the largest load. Given that consumption was very inelastic, efficiency losses associated with setting this mark-up were small and were not considered.

Under this traditional pricing scheme, production by small DER would often be accounted for as “negative load”. That means that it would be subtracted from total consumption. Lowering the net electricity consumption would lower the retail and distribution bill of households with DER. Under this system, households receive implicit compensation for decentralized production that is equal to the sum of their energy retail price and distribution tariff. This principle, sometimes also called “net metering”, is relatively simple to administer and provides additional incentives for self-production, which was seen as a bonus.

3.3. Current Tariff Becomes Unsustainable

A tariff system based on energy volume is, however, not future proof. With the development of DERs, those tariffs need to be adjusted for several reasons.

Demand profiles of consumers are no longer similar to the introduction of new technologies such as electric vehicles, local battery storage, and heat pumps [51,52]. The total energy consumption is therefore no longer a good measure for network costs that consumers impose on the network. This will typically lead to inefficiencies and could lead to cross-subsidization, which is especially harmful should it occur from poor to rich households.

In a case study for the Australian market, NERA [53] compares the marginal network cost caused by households with air-conditioning to the network tariffs they pay. They show that the marginal costs are 2.4 times higher. The reason is that air conditioning has a direct impact on the system’s capacity needs because its use is contemporaneous with aggregate peak demand. This implies that consumers with air-conditioning pay less than their marginal cost, and that they are cross-subsidized by consumers without it. Hence, the tariff is inefficient and places a burden on consumers that are likely to be poorer.

A similar argument exists against distributed generation in combination with net-metering [54]. Net metering assumes implicitly that the avoided network costs for distributed generation are equal to the marginal network cost for additional consumption. This is often not the case. Distributed generation may reduce energy losses in the network but it requires additional investment in smart systems to facilitate integration. Moreover, network capacity might not be reduced one for one, as peak demand still needs to be met by the grid at times with little decentralized production. Hence net-metering is inefficient. It also implies a cross-subsidy of households with distributed energy sources by those without them. The latter are often vulnerable households without sufficient finances to invest in decentralized generation, or without property to build upon. A combination of net metering along with a high penetration of distributed generation may lead to a volatile income stream for the network operator.

The supply and demand of DER are expected to become more elastic, both in the short run (operational decisions) and in the long run (investments in new equipment). This implies a higher demand elasticity for network usage. As demand becomes more elastic, setting tariffs above the marginal network usage cost will create larger efficiency losses, as illustrated in Figure 1. Hence it becomes vital to link tariffs to the correct marginal network costs, and to consider less distortive options than a tariff mark-up to cover the investment cost.

As tariffs are typically multi-part, aligning network tariffs with marginal network costs is not as straightforward as presented in Figure 1, which represents tariffs as a single one-dimensional number. The optimal multi-dimensional tariff structure will depend on own and cross-price demand elasticities of network usage to the different tariff components and will typically require numerical simulations for in-depth analysis. Brown and Faruqui discuss the different options for cost recovery in more detail [55].

3.4. Alternative Tariffs Structures

Assuming that demand response implementation will lead to higher demand and supply elasticities, grid tariffs need to become more closely aligned to the marginal network costs to improve efficiency.

Given that the network is a natural monopoly, setting tariffs equal to marginal cost will not cover the full cost of the network operator. Mark-ups will have to be set-upon the part of the demand that is the least elastic. This is likely to involve connection charges (€/connection) or additional capacity charges (€/kW).

A first improvement is to increase the capacity tariff (€/kW) while reducing the volume tariff (€/kWh). This is beneficial as marginal costs are more closely determined by capacity needs and less so by energy volume.

Schittekatte et al. model the consumer decisions in response to different tariff structures and show that a capacity charge (which does not differentiate between consumption and production) might lead to overinvestment in storage facilities by households [56]. By building storage they can lower their capacity needs and reduce their contribution to the fixed network costs. Those costs then have to be paid for by other network users. This cost-shifting provides a perverse investment incentive.

To prevent cost-shifting, net-metering is gradually being phased out in Europe, with separate tariffs being recommended for consumption and generation. In their white paper on regulation, ACER and CEER [57], state: “European Energy Regulators recommend that, in Article 15 of the Electricity Directive, the reference to, “cost reflective, transparent and non-discriminatory network charges, accounting separately for the electricity fed into the grid and the electricity consumed from the grid” for prosumers be further developed to exclude the possibility of net metering.”

Having separate consumption and generation tariffs also provides the network operator with more regulatory instruments to cover its network costs, which is more efficient [54].

In 2005, The Netherlands switched to a system where low-voltage consumers only pay a monthly connection charge. Those connection charges hold for all connections below a certain current draw (Ampere) and do not depend on actual consumption or production and therefore do not influence the production or consumption decision of prosumers. They also do not distort investment decisions, as there is no capacity tariff.

3.5. Smart Meters and Dynamic Tarrifs

Smart meters and smart grids allow for a richer tariff structure than was possible in the past and have lowered the cost of more innovative tariff designs. Direct information provision to customers and the automation of operational decisions in expert systems (e.g., smart thermostats) make these richer tariff structures more palpable for end-users.

The availability of smart meters allows setting tariffs on the basis of short-run marginal costs. This will be effective in alleviating congestion by reducing or time-shifting demand and or supply. This could be achieved by locational marginal prices (LMP) at the distribution level. LMPs are typically used at the transmission level, but not at the distribution level. Technological progress in communication and optimization software might make this technically feasible in the future at the distribution level [58,59]. However, it remains unclear at which voltage level this should be organized and whether LMPs would provide useful economic signals. We could imagine a CEC which owns an integrated network with local electricity storage and flexible DER to use some form of LMP/peak-load pricing, to optimize its local energy system. But, this might not be feasible for a DSO as a whole. The implementation of LMP might be different at the DSO level as well. Instead of an hourly bid-based model as is currently used for the TSO, a priority service auction could be established. In such an auction, consumers indicate priority levels for their energy consumption, and prices and volumes adjust based on the scarcity of network resources [60].

Along the same dimension also time-of-Use tariffs, where the transmission tariff depends on pre-determined time slots, or critical peak tariffs, where the tariffs are higher during a small number of hours indicated by the network operator, could be used. Those simplified tariffs will not adapt automatically to the level of (local) congestion, as LMP pricing, and reflect average network conditions. In Spain, consumers can up for time-of-use tariffs (ToU) for their network uses. Those tariffs are distinguish between peak and off-peak prices. Preliminary evidence [61] suggests that Spanish households adjust their consumption to those time-of-use prices. Borenstein shows significant long-run efficiency gains of real-time pricing compared to a flat price [62]. He also indicates that time-of-use pricing can only capture a small fraction of those gains, as prices correlate poorly with periods with the periods with critical peak demand, whereas critical peak pricing captures a significantly larger share of those gains.

As an alternative to dynamic network tariffs, a DSO could use simpler network tariffs and rely on ancillary service markets for flexibility to deal with local congestion. The DSO then procures local flexibility services from DERs either directly or through aggregators. In return for their network tariff, consumers receive a basic network service. They can opt-in for the more dynamic price setting of the ancillary service market. The good functioning of those flexibility markets requires clear baseline consumption and production levels. Those baselines are ideally based on network quantities that are contracted between network users and the network operator and will therefore require a network tariff which specifies a demand (or supply) profile with penalties for deviations from this profile. Alternatively, baseline levels are based on some averaged consumer profiles, but this is likely to lead to some form of gaming.

3.6. Distributive Concerns and the Death Spiral

Abolishing net-metering and shifting to capacity tariffs will reduce cross-subsidization from poor consumers to rich consumers with air-conditioning and PV installation and improve the fairness of the tariff structure. However, some tariff changes might harm vulnerable consumers as well. They may for instance not be able to invest in equipment that supplies flexibility and end up paying more than other consumers. Moreover, a fixed connection charge (€/connection) will, in relative terms, be a larger burden for poorer consumers. For a more in-depth discussion on fairness in distribution networks see [63].

In the long run, when the costs of storage and local generation are expected to drop further, then CECs might decide to partially or fully disconnect from the distribution network and operate on a stand-alone basis. The cost of the distribution network will then have to be covered by the remaining (possibly poorer) network users who will see their energy bills increase as a result. This could lead to a “death spiral” where more customers leave the distribution network (unlikely in northern Europe), network assets become stranded, the distribution network becomes obsolete and goes bankrupt, and only small island grids remain. This is not necessarily efficient: as consumers base their connection decisions upon a tariff that includes compensation for stranded assets, which are no longer economically viable, then they disconnect from the network too often.

4. Setting the Correct Incentives for Distribution Operators

The integration of DER and the digitalization of smart networks poses challenges for the DSO. These new challenges are likely to require a new regulatory framework but could also imply that the market structure and allocation of responsibilities have to change. The next subsections list a number of challenges and possible remedies.

4.1. Challenges

4.1.1. Capital vs. Operational Costs

Distribution system operators face a trade-off between further expanding their network to deal with larger and more volatile demand, or to use their existing network more efficiently and actively manage the flows on the network, for instance by procuring storage or flexibility form network users.

4.1.2. New Tasks Require New Skills

At the distribution level, many new tasks and goals have to be achieved. DERs must be integrated, localized congestion needs to be managed, ancillary services have to be procured and new market players (e.g., energy communities, aggregators) need to be accommodated. Secure data infrastructure and communication platforms need to put in place. Those tasks may go beyond the skills of a small independent DSO, and it seems likely that large economies of scale will need to be exploited in order to operate efficiently.

4.1.3. Preventing Cross-Subsidies

A regulated DSO that is vertically integrated with a retailer may be able to cross-subsidize its retail arm. This could lower the capital costs of this retailer and distort competition. If the future cost structure becomes more complex, it might become harder to allocate costs to specific functions and prevent cross-subsidization. Reference [64] discusses when cross-subsidies can be considered anti-competitive.

4.1.4. Prevent Discrimination

The future DSOs will have to procure ancillary services from market participants. Hence, the DSO will become an active market participant in the energy market. In order to guarantee a level playing field for all market participants, a vertically integrated DSO should not favor its own affiliate over its competitors.

4.1.5. Innovation

The distributed energy system is still in its infancy. Technology standards and interoperability requirements are still being developed, the market design has not yet crystallized and regulation (both national and EU-wide) is being developed as we go along. Distribution companies are intermediaries between many players, the enablers of the energy transition. For them to play this role, they need to have the freedom and incentives to innovate and take risks.

4.1.6. Limited Empirical Evidence

Worldwide there are no good large- scale examples on how the electricity sector should be organized (structure, regulation, technology) in order to integrate DER efficiently and equitably. The new European market design and regulation in this area remain untested.

4.2. Possible Remedies

4.2.1. No EU Harmonization, but Encourage Learning

We do not see a clear rationale for harmonizing regulation Europe-wide yet. Instead, it is important that we learn from experimenting with different forms of regulation in the Member States. Hence there is a value in variety. However, we might want to harmonize reporting requirements to improve mutual learning. Also, with respect to DSO tariffs, we do not favor harmonization as the marginal costs for distribution networks differ across countries and different preferences exist across countries with respect to fairness.

4.2.2. Regulation Open to Innovation

Member States should be encouraged to provide temporary deviations from existing regulation for small scale experiments (regulatory sandboxes). To promote innovation, subsidies may be required especially if there are regulatory spill-overs between DSOs, as we would expect too little investment in innovation otherwise.

4.2.3. Regulation and Market Structure are Interwoven

The regulatory framework will ideally depend on the market structure that is chosen. Below we highlight three possible options. Those options are also shown in

Table 1. Three options for organizing DSOs activities, and their benefits and drawbacks. Cells with the same color are operated by the same entity either directly or by a subsidiary.

		Option a	Option b	Option c
Tasks/actors	Retail	Utility Co.—Retailer	Utility Co.—Retailer	Utility Co.—Retailer
	Operating local grid	Independent DSO	Utility Co.—DSO	Utility Co.—DSO
	Organizing local energy market Procuring Local ancillary services	Independent DSO	TSO	Utility Co.—DSO
Pro and Cons	Pro	• high powered incentives • innovation by DSO • coordination investment & operation	• coordination DSO-TSO • knowledge sharing TSO • no restructuring costs	• no restructuring cost • knowledge sharing DSO-utility
	Cons	• costly unbundling of retail • DSO mergers for scale economies	• no coordination investment/operation costs	• limited incentives for innovation • limited coordination investment cost and operation • slow regulatory adjustments • risk of market power abuse

. Reference [23] reviews international proposals for organizing network services. It shows not only a wide variety of how responsibilities can be allocated between DSOs and TSO, but also the option for creating a new entity that is responsible for market coordination.

4.2.4. Option (a) High Powered Incentive Regulation of a Large Unbundled DSO

One option to deal with some of the challenges above is to give a DSO a lot of flexibility but provide it with strong incentive regulation. Ref. [25] (Section 3.1) highlights the benefits of flexibility in regulation in combination with incentive regulation.

An example of such regulation could be a price cap with a quality bonus and an efficiency improvement component. Under incentive regulation, the national regulatory authority (NRA) will not approve each individual decision by the DSO but will set monetary incentives. Price cap regulation has been used in the UK and the Netherlands, under an RPI-X regime.

Once the DSO has the right incentives it could determine for instance its own tariff structure and procure ancillary services under long-term contracts. In order for DSOs to make the trade-off between capital costs (new network capacity) and operational costs (procuring flexibility) the DSO could be regulated on the basis of total expenditures (TOTEX). In the Netherlands, DSOs are benchmarked against each other on the basis of TOTEX. By benchmarking companies, the regulator creates virtual competition between the regulated companies.

DSOs will have to cooperate with TSOs in minimizing production costs. This requires an incentive scheme where the profit of the DSO depends on the performance of the TSO (and vice-versa). This type of regulation depends on the theory of incentives in teams [26]. See also [65] for a discussion of a potential contractual framework that allows DSOs to have the primary economic responsibility for system balancing.

This type of high-powered incentive regulation might not work whenever the DSO is part of a vertically integrated utility. This could be due to the fact that the DSO’s profit will only be a small fraction of the integrated utility, and the regulatory incentives may in some cases conflict with the overall incentives of the company. Ref. [66] provides empirical evidence that a combination of incentive regulation and unbundling is required to improve investment decisions. Furthermore, as the DSO makes a lot of discretionary decisions under high-powered incentive regulation, it could discriminate against is competitors. The Netherlands, therefore, require full ownership unbundling of DSOs. Most member states require only legal or functional unbundling [67].

Ownership unbundling of the DSO will imply that the new DSO loses some know-how which was available in the vertically integrated firm. A small (ownership) unbundled DSO might therefore not be up to the tasks, and several smaller DSOs may have to merge. Unbundling will therefore drastically change market structures and its implementation is likely to take considerable time. For instance, in The Netherlands, the unbundling of DSOs took more than 10 years.

4.2.5. Option (b) Bring Some Market Activities of the DSO to the TSO Level

An alternative option is to shift some tasks currently performed at the distribution level to the TSO. The TSO could for instance manage local congestion on the distribution network and procure local ancillary services.

The advantages of this set-up are that synergies between the DSO and TSO level can easily be achieved and that the know-how on setting-up and managing markets is already present at the TSO level. One of the disadvantages is that if distribution assets are still owned by the DSO then the coordination of local investment decisions and operational decisions becomes harder.

4.2.6. Option (c) Comprehensive Regulation of Vertically Integrated Network Operators

In some Member States, DSOs are part of vertically integrated utilities. Vertical integration could help coordination between the different levels of the value chain, especially distribution and retail. For instance, the retail arm of an integrated utility could co-own storage facilities and share some of the risks with DER. Such an integrated utility is also likely to have sufficient economies of scale (and scope) and know-how to set-up local markets.

The downside is that vertical integration could generate concerns about the potential limitations of effective competition. NRAs control will therefore have to be more comprehensive and leave less flexibility to the DSOs than under incentive regulation (Option a). The procurement of local ancillary services will have to follow strict rules, guaranteeing a level playing field. Softer unbundling requirements (such as management unbundling, information-sharing requirements, technical protocols) may also need to be strengthened to make this option workable.

Reference [68] discusses the economic trade-offs of unbundling. Unbundling improves competition between market actors but might reduce coordination between retailer/generator and network operator.

5. Discussion and Conclusions

The CEP rewrites the market rules of the European electricity market and provides new opportunities for DERs. The EC introduces a new market-based paradigm where the residential energy consumer can participate in any organized energy market, directly or through an intermediary, either as a passive price taker or an active trader. The CEP recognizes two new forms of intermediaries: the independent aggregator and CECs. Those obtain similar rights as end-users with respect to market access.

This paradigm shift is rather large, and although the Commission does not impose one particular implementation, it sets minimal requirements as highlighted in Section 2. Consumers have the right to obtain a smart meter and be subject to dynamic prices, and the compensation paid to other market parties by independent aggregators is subject to a number of conditions.

By choosing for a market-based solution, the EC leaves the development of business models and the creation of new market places to market players. We believe this is the correct way forward, as there are very few large-scale examples of well-functioning DR programs yet, and a top-down implementation is therefore not appropriate. A crucial requirement for market forces to develop DR is that network tariffs are not distortive. The profitability of DR business models should not depend on arbitraging tariff imperfections or shifting fixed costs to other network users.

In this paper we argue that network tariffs should therefore become more cost-reflective. This implies that the capacity component has to increase, net-metering to be abolished, and alternative tariffication schemes such as critical peak prices for network capacity need to be developed. We also indicate that network tariffs could contain optional components where consumers can provide additional flexibility to manage local congestion.

The EC recognizes the crucial role that DSOs will play in catalyzing the development of new market places. DSOs have to enable DER to participate in organized energy markets and become procurers of local resources in their own right. We propose three different regulatory scenarios to support those new DSO roles. (a) a horizontal merger of unbundled DSOs under incentive regulation, (b) a DSO as a subsidiary of an integrated utility under cost plus regulation, (c) a transfer of market activities from the DSO to the TSO.

Under the subsidiarity principle, the implementation of the directives is left to member states. This allows member states to adjust regulations to their particular historical situation. Allowing for variety in regulation at the member state level has the additional benefit of creating room for innovation in regulatory practice, by creating regulatory competition [69].

We can imagine that Germany with its many small DSOs opts for shifting some of those responsibilities to the TSO-level (scenario c.). The Netherlands, which already has fully unbundled DSOs, could after a period of experimentation, allow for DSO mergers to guarantee sufficient economies of scale to organize smart energy markets (scenario a). Italy and France, with few large DSOs could decide for cost-plus regulation, where the regulator designs a single country-wide market mechanism. (scenario b).

Technological barriers for the development of DR are surmountable. The underlying technologies exist and are being codified into technological standards. Whether DR will grow beyond demonstration projects and become an economic success will depend on the successful development of new business models that are attractive to all parties involved. The activation of households might be an important hurdle, given their current low interest in switching suppliers notwithstanding significant monetary gains. The monetary benefits of activating DR in field experiments are smaller than that of switching retailers, and often other considerations (e.g., environment) are more important [23]. We therefore believe that economic margins in DR programs are likely to be modest. Randomized field experiments (RCTs) might give us some further insights into consumer behavior, but they will not replace empirical results from large scale DR markets.