Analyzing Demand Response in a Dynamic Capacity Expansion Model for the European Power Market

Abstract

One of the challenges in the transition towards a zero-emission power system in Europe will be to achieve an efficient and reliable operation with a high share of intermittent generation. The objective of this paper is to analyse the role that Demand Response (DR) potentially can play in a cost-efficient development until 2050. The benefits of DR consist of integrating renewable source generation and reducing peak load consumption, leading to a reduction in generation, transmission, and storage capacity investments. The capabilities of DR are implemented in the European Model for Power Investments with high shares of Renewable Energy (EMPIRE), which is an electricity sector model for long-term capacity and transmission expansion. The model uses a multi-horizon stochastic approach including operational uncertainty with hourly resolution and multiple investment periods in the long-term. DR is modelled through several classes of shiftable and curtailable loads in residential, commercial, and industrial sectors, including flexibility periods, operational costs, losses, and endogenous DR investments, for 31 European countries. Results of the case study shows that DR capacity partially substitutes flexible supply-side capacity from peak gas plants and battery storage, through enabling more solar PV generation. A European DR capacity at 91 GW in 2050 reduces the peak plant capacities by 11% and storage capacity by 86%.

1. Introduction

The transition of the European power system towards zero emissions in 2050 requires an increase in the share of intermittent renewable sources in the energy mix. Decarbonizing other sectors in the economy leads to an increase in power demand [1]. A challenge in this system will be to achieve efficient and reliable operation. Demand Side Management (DSM) [2] can play an important role in balancing the market, based on active consumers, smart metering and ICT infrastructure, new market rules, aggregators, and storage. Here DSM includes Demand Response (DR), energy efficiency, and other measures at the demand side that can play a role in facilitating this balance. DR comprises technologies that enable the temporary reduction of electricity consumption by the consumer, including the possibility to shift load in time, change load profiles, and curtail load [3,4,5]. Short-term Demand Response (DR) is an emerging technology that will introduce demand-side flexibility. The benefit of DR is to provide flexible loads, leading to a reduction in peak generation, transmission, and storage capacity investments. The objective of this work is to analyse what role DR can play in a cost-efficient transition to a European low-emission power system. This article presents a DR module within the European Model for Power Investments with high shares of Renewable Energy, EMPIRE [6]. The EMPIRE model is then used to study how flexible loads support the integration of Intermittent Renewable Energy Sources (IRES) and affect other flexibility technologies such as storage or transmission when integrated with other technologies. This DR extension models several classes of shiftable and curtailable loads in residential, commercial, and industrial sectors for 31 European countries.

There are three important flexibility dimensions to consider: ramping, power, and energy [7]. DR has flexibility characteristics allowing peak shaving, increase of the reliability of the system by reducing outages, integration of variable renewable resources, diversification of resources, enhancement of efficient system operation, provision of reserve capacity, and giving more decision power to customers [2]. The capacity of reducing $C{O}_2$ emissions through load-shifting and ICT energy efficiency measures has been estimated between 0.23% and 3.3% [8]. Other technologies might compete with or supplement DR in provision of flexibility. Pumped-storage hydro power, compressed air storage, Closed Cycle Gas Turbines (CCGT), and battery storage systems can resolve high-demand contingencies. Lund et al. [9] review flexible measures in energy systems and concludes that DR can provide very short, short, and intermediate flexibility in the range of 1 min to 3 days. EMPIRE focus on the energy balance between supply and demand when allocations are made under perfect competition and typically with 1 h as the finest time resolution. This will capture the effects identified by Lund et al. [9] that suggest that DR, storage, flexible power plants, and interconnectors can alleviate the power ramps and flexibility requirements in time spans from 1 to 12 h and IRES. Still, this energy-only approach may underestimate the value of DR as it may also play a role in ancillary services markets or regulation markets with shorter time horizons. This is normally not captured in this type of long-term model. There are challenges also in intra-hour DR optimal operation such as non-disruptive energy arbitrage in intra-hour energy markets for cyclic thermostatic loads such as air conditioners and refrigerators [10]. Models for DR commoditization at energy and intra-hour reserve markets have been proposed [11], and applied to an office building air conditioning system [12].

The main contribution of this article is the modelling and implementation of DR operation and capacity expansion extension in a long-term planning model with short-term uncertainty and dynamics. To our knowledge this is a capability not existing in the literature. Furthermore, this allows us to analyse the consequences of DR on the technology mix recognizing the intermittent nature of generation and load. By dynamic long-term decisions, we mean that the model can represent decisions in a sequence of discrete time investment periods (typically 9 periods in 5-year steps). By dynamic short-term operations we mean that the model includes for each investment period several sequences of discrete time periods with hourly representation (typically 168–336 h in sequence). By short-term uncertainty we mean that the model allows the representation of stochastic scenarios over these operational time periods for variables such as load, wind, solar, and hydropower. In fact, this model is formulated as a two-stage stochastic program [13].

Methodologically, EMPIRE uses multi-horizon stochastic programming designed to jointly optimize several time scales [14]. The problem of representing the short-term scale has been studied in Poncelet et al. [15], Pfenninger et al. [16], Nahmmacher et al. [17], Seljom [18]. There exist multiple models co-optimizing generation investments and system operation: EMPIRE, ReEDS, THEA, EMPS and TIMES [19,20,21]. According to a recent review of capacity expansion models [22], EMPIRE is at the power market modelling forefront in terms of the span of short- and long-term horizon (full year’s representative hours and decades respectively) and co-optimizing generation, transmission and storage. In addition, the short-term time-slices are build based on a moment matching method (for a deep review of capacity expansion models the reader is referred to [22,23,24]).

The next section gives a short overview of the DR potential in Europe and the state-of-the art in representation of DR in energy system models. Following that, Section 3 presents the DR model extension of EMPIRE. Section 4 analyses the effects of DR on the European power systems, before conclusions.

2. Related Literature

By 2015, the countries with open DR trading in the wholesale electricity market were: UK, Ireland, France, Switzerland, and Belgium. The countries with partial opening to explicit DR were Norway, Sweden, The Netherlands and Austria [25]. As of 2019, new partially opened markets for aggregators exist in Germany and Denmark. New commercially opened markets are Switzerland and Finland [26]. Despite the early implementation of DR programs in countries such as France [2], policy-makers have been reluctant to fully support DR programs, in favor of finding the right mix of solutions for the ambitious environmental goals, and focusing on liberalizing markets [27]. The most successful programs have been with large industrial consumers, by means of costumer incentive-based measures such as Direct Load Control (DLC) and interruptible/curtailable programs. Time of Use Tariffs (TOU) and Real Time Pricing (RTP) have been also available for residential and commercial customers, although to a lower extent due to a lack of price signals and support to DR programs. It is expected that residential DR will be greatly developed with the planned roll-out of smart metering and spread of heat pumps [28]. Industry consumers that may contribute are steel-making industries, paper, pulp, and chloralkali process, among others. In the services sector, DR could be used by storage heating, retail cooling systems, and ventilation systems for example. In the residential sector the appliances with reasonable potential are washing machines, tumble dryers, and dishing machines, as well as air conditioning systems.

According to Baker [29] and ENTSO-e [30] the maximum DR power assumed by each country barely reaches 5% of the peak demand in some of the most developed areas. For instance, this is 1.3 GW, 1 GW, and 0.7 GW for France, Spain, and Great Britain, respectively. The countries with the highest potential are Belgium, Germany, Spain, France, Great Britain, Ireland, The Netherlands, Poland, Slovenia, and Sweden. The 2025 DR forecast is around 6% for the countries with the highest potential. The European countries potential has been assessed recently in terms of spatial, seasonal and end-consumer representation [31]. At the European level DR shedding potential has been estimated at an average of 93 GW [31], which represents 18% of overall peak demand.

In the literature on DR in electricity systems there is a lack of articles on long-term investments with IRES uncertainty [32]. A linear optimization model with seven important DR processes was used to specifically estimate the DR impacts in Germany [33]. The potential of these DR loads was assessed at a regional level. A study by Schill and Zerrahn [34] represents 6 types of DR relying on the same costs assumptions as Gils [33]. In Kies et al. [35], a simplified deterministic European power system model with DSM as storage is presented and sheds light on the needs for demand flexibility under different transmission and IRES shares scenarios for Europe in 2030. This model includes only one investment period. Zerrahn and Schill [5] suggest a model solving the problem of recovery [4], with time free structure and handling excessive DR unit activation which is applicable to several numerical models. Also Zerrahn and Schill [36] present a green-field model, DIETER, to study the optimal storage capacity and flexibility options in 2050. They improve shortcomings of existing models, with the above mentioned DR model, plus reserve capacity and multi-term variability. However, the model cannot assess the system transformation since it is only for one year. In a companion paper [34], the results of a case study for Germany are shown, representing its power system as a copperplate (single node). They use 2013 as base year for IRES in-feed time series to estimate Germany’s 2050 generation and storage portfolio.

In a broad European perspective, Papadaskalopoulos et al. [37] study the impacts of Flexible Industrial Demand (FID) on the generation and transmission mix with a capacity expansion model and a stochastic unit dispatch model. The study investigates DR potential in 2030 and does not include flexibility scheduling costs. Their conclusion is that FID saves capacity investments in generation, transmission, and distribution grids to the order of billion euros. In Müller and Möst [38] the time availability of DR is investigated. They look at the potential of DR for integrating IRES generated electricity at 60% and 80% in Germany by 2035 and 2050. First they estimate the theoretical DR potentials for residential, industrial, and commercial sectors taking into account weather conditions and seasonality with a single base line year, but disregarding DR operational costs. Then, by using a deterministic linear electricity market model, ELTRAMOD, the impacts of flexible loads on renewable sources, peak plants, and storage are found to be relevant. Although the curtailment of IRES generation decreases by 35–77%, DR cannot integrate long timespan sustained IRES oversupply due to its short-term limitations. Therefore, in this approach the system uptake evolution of DR is overlooked. De Jonghe et al. [39] propose a complementarity programming approach, a method using electricity elasticities and a piece-wise linear integration of consumer value to integrate DR in a capacity expansion model. In Lohmann and Rebennack [40], a short-term DR model with a long-term investment scope is described using inter-hour electricity demand elasticities.

When it comes to DR short-term models, scheduling and market participation play a fundamental part, as well as how to model the different DR technologies and modes. In [41] two flexibility management methods for a multi-energy carrier building are studied. The stochastic model represents a prosumer optimizing its energy management, including flexible loads: shiftable volume, shiftable profile, curtailable and interruptible loads, assuming no efficiency losses. García-Garre et al. [42] present an algorithm to coordinate a prosumer’s household solar PV and residential DR. Their results show that self-consumption rate raises 25% and payback lowers 20%, but without considering system effects. In [43] a bidding methodology based on stochastic optimization for aggregators with several prosumers (equipped with lossless DR) is proposed. Uncertainty in wholesale spot market prices and loads are handled with a scenario generation method. The bidding strategy is obtained in a two-stage stochastic mixed integer linear program.

A similar problem setting is presented in Sáez-Gallego et al. [44] where a retailer buys energy in the day-ahead market for a pool of price-responsive consumers. They provide an analytic solution in the case that the retailer is not risk averse and a stochastic programming model for optimal bidding under risk aversion. In [45] a multi-market bidding strategy for an aggregator managing four industries is studied using a three-stage stochastic optimization approach. In Iria et al. [46] a two-stage stochastic optimization model is proposed for an aggregator bidding in the day-ahead and secondary reserve market, showing that the stochastic strategy is beneficial compared to a deterministic-based strategy. Casals et al. [47] model an aggregator using obsolete electric vehicle (EV) batteries that have a second life when used by a flexible consumer to provide capacity reserves to the grid. Vallés et al. [48] give an empirical methodology to exhaustively characterize residential consumers’ flexibility in response to economic incentives. Their approach uses quantile regression to assess the probabilistic responsiveness of residential consumers, ending up with two measures called flexibility at risk and conditional flexibility at risk.

The main contribution of this article is handling in the same long-term planning framework the short-term operational aspects of DR operation and capacity expansion. As far as we know this has never been included in models with short-term dynamics, long-term dynamics, and short-term uncertainty.

3. The EMPIRE Model with Demand-Responsive Loads

The European Model for Power Investment with Renewable Energy (EMPIRE) is a stochastic capacity expansion model designed to find optimal investments in generation, storage and cross-border transmission over a long-term planning horizon [49]. In the following we will present the EMPIRE model and its new DR extension. EMPIRE makes use of a multi-horizon approach [14] to avoid the curse of dimensionality. It combines short-term uncertainty in system operation linked to long-term investment periods. This allows the inclusion of several short-term stochastic scenarios created from uncertainty in load, solar, and wind production historical data in each country. Investments are available in several technologies for power generation, storage, and transmission. The power system is modelled as an energy transport network with 31 nodes (European countries). The main underlying assumptions of the model are: (1) Perfect competition markets in each country and no strategic behavior, (2) Demand and supply are aggregated at country level and there is no unit commitment, (3) The electrical distances within each country-node are not considered. Only distances between countries are considered to be an element in the infrastructure cost. The DC power flow is therefore not modelled, rather import/export capacities between countries are represented by Net Transport Capacities (NTC) [6]. (4) There are 26 different technologies included in the EMPIRE model to invest in. Some technologies that are not yet developed may be represented by learning curves for existing technologies with costs decreasing over time and increasing efficiency.

The objective of this model extension is to represent several types of flexible loads operating in an energy-only market. First, a flexible load, indexed by f, is defined as an hourly profile whose components $y_{fh}$ can be changed by upward or downward regulation for some hours h. Then a shiftable volume load is defined as a flexible load such that any load regulation keeps the same total energy (the sum of its components). A curtailable load is a flexible load whose components can be decreased but not increased. An interruptible load is a curtailable load which can either be left unmodified or decreased to all zero components. We assume DR endogenous investments are only driven by optimality considerations, disregarding costs and policy uncertainty, which is handled on a forthcoming paper.

3.1. Nomenclature DR Module

Symbol	Description
Indices and sets
$f\in \mathcal{F}=\mathcal{S}\cup \mathcal{C}$	Index and set of flexible loads
$f\in {\mathcal{F}}_n$	Set of flexible loads that belong to node n
$f\in \mathcal{S}$	Set of shiftable loads
$f\in \mathcal{C}$	Set of curtailable loads
$i\in \mathcal{I}$	Set of long-term periods where investments take place
$\omega \in {\mathsf{\Omega }}_i$	Set of operational scenarios in long-term period
$h\in \mathcal{H}$	Set of hours in each operational scenario
Variables
$x_{fi}^{\mathrm{DR}}$	DR capacity investment of group f in period i
$x_{gi}^{gen}$	Capacity investment of generator type g in period i
${\upsilon }_{fi}^{\mathrm{DR}}$	Aggregated DR capacity of group f
${\upsilon }_{gi}^{\mathrm{gen}}$	Aggregated capacity of generator type g
$y_{fi\omega h}^{\mathrm{DR}}$	Actual DR load of group f
$y_{fi\omega h}^{\mathrm{DRCAP}}$	Potential downward regulation of group f
$y_{fi\omega h}^{\mathrm{REG}}$	DR load deviation
${\lambda }_{fi\omega ph}^{\mathrm{REG}}$	Special Order Set 2 DR regulation variables
Parameters
$c_{fi}^{\mathrm{DR}}$	DR investment cost
$q_{fp}^{\mathrm{REG}}$	DR operational cost
${\alpha }_{fih}$	DR load profile
$X_{fi}^{\mathrm{DR}}$	Maximum DR capacity investment
$U_{fi\omega h}^{\mathrm{DR}}$	Maximum upward regulation of group f
${\mathsf{{\rm Y}}}_{fi}^{\mathrm{DR}}$	Maximum DR capacity of group f
$t_f^{\mathrm{shift}}$	Maximum load time shift of group f
${\eta }_f^{\mathrm{DR}}$	Efficiency of flexible load f
$Q_{fi}^{\mathrm{REG}}$	DR baseline operational costs
$m_{\mathrm{REG}}$	DR operational cost scaling factor
${\mu }_{fp}^{\mathrm{REG}}$	Special Ordered Set of type 2 intervals

3.2. Mathematical Model

The EMPIRE model is a two-stage stochastic program [49]. For each long-term period (typically consisting of 5 years), $i\in \mathcal{I}$, it is possible to invest in new generation, storage, or cross-border transmission capacity in each node. Linked to each long-term investment period i there are several operational scenarios $\omega \in {\mathsf{\Omega }}_i$ with hours, $h\in \mathcal{H}$, representing the uncertainty in load, solar power, wind power, and hydro power. The expectation $\mathbb{E}$ is taken over all the long-term periods $i=1,\dots ,I$ and over all operational scenarios $\omega \in {\mathsf{\Omega }}_i$ with hours, $h\in \mathcal{H}$. This is shown in Figure 1 with the $\mathbf{x}$ variables representing investments in periods $i\in \mathcal{I}$ and the $\mathbf{y}$ variables representing operational decisions over all operational scenarios $\omega \in {\mathsf{\Omega }}_i$. The operational costs, OPEX, and DR-OPEX, consist of the costs of generating power and modifying loads. Their full definition can be found in Equation (28). A model’s schematic representation can be found below (for clarity of exposition indices have been omitted):

min CAPEX$\left({\mathbf{x}}^{inv}\right)$ + $\mathbb{E}$ OPEX$\left({\mathbf{y}}^{op}\right)$ + DR-CAPEX$\left({\mathbf{x}}^{\mathrm{DRinv}}\right)$ + $\mathbb{E}$ DR-OPEX$\left({\mathbf{y}}^{\mathrm{DR}},{\mathbf{y}}^{\mathrm{DRCAP}},{\mathbf{y}}^{\mathrm{REG}}\right)$
s.t.	Initial Capacity $+{\mathbf{x}}^{inv}+{\mathbf{x}}^{\mathrm{DRinv}}\le$ Capacity Limit
	(for all scenarios $\omega$)
	Supply = Uncertain load + DR Regulation
	Power Supply ≤ Capacity(${\mathbf{x}}^{inv}$)
	IRES Supply ≤ Uncertain availability
	Energy Supply ≤ Energy Limit
	Emissions ≤ Emissions Cap
	Storage Balance = 0
	DR Balance = 0
	DR Regulation ≤ DR Capacity(${\mathbf{x}}^{\mathrm{DRinv}}$).

Generation and the power flow through transmission lines is constrained by capacity limits and ramping constraints. The DR module introduces new second-stage variables $y^{\mathrm{DR}},y^{\mathrm{DRCAP}},y^{\mathrm{REG}}$ for the flexible load types, $f\in \mathcal{F}$, as well as first-stage multi-period investment variables in DR technologies $x_{fi}^{\mathrm{DR}}$ (i.e., endogenous expansion capacities). First-stage variables ${\mathbf{x}}^{inv}$ represent capacity expansion on generation, transmission, and storage. Second-stage variables ${\mathbf{y}}^{op}$ represent operation of generators, interconnectors, and storages.

In the following we introduce the notation for flexible loads starting by the balance Equation (1). The module includes new DR operational variables called load deviation, $y_{fi\omega h}^{\mathrm{REG}}$ for each flexible load $f\in \mathcal{F}$, period $i\in \mathcal{I}$, scenario $\omega \in {\mathsf{\Omega }}_i$ and hour $h\in \mathcal{H}$. It represents the up-regulation (positive) and down regulation (negative) of flexible loads. It also includes the load loss of activating flexible loads $y_{fi\omega h}^{\mathrm{LOSS}}$ since we assume there exist DR rebound effects and operational inefficiencies, in the same way that storage and transmission have their own efficiencies ${\eta }_b^{\mathrm{dischrg}},{\eta }_a^{\mathrm{tran}}$, where $b\in \mathcal{B}$ is a storage unit and $a\in \mathcal{A}$ is an arc of the network. These efficiency factors are assumed to be constant in any grid operating state. We also assume ramp constraints, conventional generation, storage, and transmission limits to be constant.

The new second-stage variables are co-optimized with the network operational variables as generation $y_{gi\omega h}^{\mathrm{gen}}$, with $g\in \mathcal{G}$ a generator, storage $y_{bi\omega h}^{\mathrm{dischrg}}$, $y_{bgi\omega h}^{\mathrm{chrg}}$, transmission $y_{ai\omega h}^{\mathrm{flow}}$ and lost load $y_{ni\omega h}^{\mathrm{ll}}$.

$$\begin{array}{c}{\displaystyle \underset{\mathrm{Generation}}{{\displaystyle \underbrace{\sum _{g\in {\mathcal{G}}_n}{y}_{gi\omega h}^{\mathrm{gen}}}}}}+\underset{\mathrm{Storage}\mathrm{handling}}{{\displaystyle \underbrace{\sum _{b\in {\mathcal{B}}_n}{\eta }_b^{\mathrm{dischrg}}{y}_{bi\omega h}^{\mathrm{dischrg}}-y_{bi\omega h}^{\mathrm{chrg}}}}}+\underset{\mathrm{Net}\mathrm{import}}{{\displaystyle \underbrace{\sum _{a\in {\mathcal{A}}_n^{\mathrm{in}}}{\eta }_a^{\mathrm{tran}}{y}_{ai\omega h}^{\mathrm{flow}}-\sum _{a\in {\mathcal{A}}_n^{\mathrm{out}}}{y}_{ai\omega h}^{\mathrm{flow}}}}}\hfill \\ ={\xi }_{ni\omega h}^{\mathrm{load}}+\underset{\mathrm{Load}\mathrm{Deviation}}{\underbrace{{\displaystyle \sum _{f\in {\mathcal{F}}_n}}{y}_{fi\omega h}^{\mathrm{REG}}+y_{fi\omega h}^{\mathrm{LOSS}}}}-y_{ni\omega h}^{\mathrm{ll}}n\in \mathcal{N},h\in \mathcal{H},\omega \in {\mathsf{\Omega }}_i,i\in \mathcal{I}\hfill \end{array}$$

(1)

Stochasticity in the balance Equation (1) appears in the country load ${\xi }_{ni\omega h}^{\mathrm{load}}$, flexible loads $\mathcal{F}$ and through additional constraints (2) for the generation of variable renewable sources:

$$y_{gi\omega h}^{\mathrm{gen}}\le {\xi }_{gi\omega h}^{\mathrm{gen}}{\upsilon }_{gi}^{\mathrm{gen}},g\in \mathcal{G},h\in \mathcal{H},\omega \in {\mathsf{\Omega }}_i,i\in \mathcal{I}$$

(2)

where ${\xi }_{gi\omega h}^{\mathrm{gen}}$ is stochastic for wind, solar, and hydro power, and scenario independent for the rest of technologies, while ${\upsilon }_{gi}^{\mathrm{gen}}$, is a first-stage variable representing capacity of generator g. Constraints (2) link first and second-stage decisions.

The country load ${\xi }_{ni\omega h}^{\mathrm{load}}$ is assumed to include a share of inflexible loads and a share of flexible loads. To disaggregate the flexible share into each of the DR group loads, Equation (3) links country load, investment and operational variables in the same way that (2) links power supply operation and capacity. The available DR capacity depends on both the installed DR capacity, the actual time period and on the level of the stochastic country load. Variables $y_{fi\omega h}^{\mathrm{DRCAP}}$ are called potential load and are the available hourly load that potentially can be used by flexible load group f in hour h.

$$y_{fi\omega h}^{\mathrm{DRCAP}}={\alpha }_{fih}·\frac{{\xi }_{ni\omega h}^{\mathrm{load}}}{{\widehat{\xi }}_{ni\omega }}·{\upsilon }_{fi}^{\mathrm{DR}}f\in {\mathcal{F}}_n$$

(3)

This capacity is calculated using two scaling factors. First, the deterministic ${\alpha }_{fih}$ constants take values in $[0,1]$ and represent the share of flexible load in each DR group in a given hour h considering weekday, weekend, and seasonal patterns. They capture the hourly availability factor for flexibility of each DR group. The second scaling factor is stochastic and depends on the scenario $\omega$. For each node n, long-term period i and hour h it is based on the ratio between the actual load, ${\xi }_{ni\omega h}^{\mathrm{load}}$, and the yearly peak load ${\widehat{\xi }}_{ni\omega }$ for the same long-term period. Those two scaling factors are multiplied with the installed DR capacity, ${\upsilon }_{fi}^{\mathrm{DR}}$, to find the DR potential in each hour and scenario.

To track the amount of shifted and curtailed loads for each class, an auxiliary variable, $y_{fi\omega h}^{\mathrm{DR}}$, called resulting load is defined in (4). It is a load (i.e., positive values) representing the actual country aggregated load value of flexible group f.

$$\begin{array}{cc}\hfill y_{fi\omega h}^{\mathrm{DR}}& =y_{fi\omega h}^{\mathrm{DRCAP}}+y_{fi\omega h}^{\mathrm{REG}},\omega \in {\mathsf{\Omega }}_i,i\in \mathcal{I},f\in {\mathcal{F}}_n,n\in \mathcal{N}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill y_{fi\omega h}^{\mathrm{DR}}& \ge 0\hfill \end{array}$$

(5)

By definition, the resulting load in hour h is zero when $y_{fi\omega h}^{\mathrm{REG}}=-y_{fi\omega h}^{\mathrm{DRCAP}}$. The load is unregulated when $y_{fi\omega h}^{\mathrm{REG}}=0$, upward regulated when $y_{fi\omega h}^{\mathrm{REG}}>0$ and downward regulated when $y_{fi\omega h}^{\mathrm{REG}}<0$ (the regulation range for the load deviation is not necessarily symmetric with respect to zero. An upper bound is defined for $y_{fi\omega h}^{\mathrm{DR}}$ later (Appendix B), which in consequence bounds the load deviation).

The DR energy loss is the extra energy needed to shift or curtail load. The DR efficiency of load f, ${\eta }_f^{\mathrm{DR}}$, is defined as the ratio between the load deviation and load deviation plus the energy loss. It follows from this definition that the energy loss is

$$y_{fi\omega h}^{\mathrm{LOSS}}={\beta }_f\left|y_{fi\omega h}^{\mathrm{REG}}\right|\left(\frac{1-{\eta }_f^{\mathrm{DR}}}{{\eta }_f^{\mathrm{DR}}}\right)h\in \mathcal{H},\omega \in {\mathsf{\Omega }}_i,i\in \mathcal{I},f\in {\mathcal{F}}_n,n\in \mathcal{N}$$

(6)

where we model the energy loss assuming it equally distributed between the hours with load shift regulation. This sets the value 0.5 for shiftable loads to avoid double counting, that is

$${\beta }_f:=\left\{\begin{array}{cc}0.5\hfill & f\in \mathcal{S}\hfill \\ 1\hfill & f\in \mathcal{C}\hfill \end{array}\right.$$

(7)

The deviations from the potential load are performed under a supply cost function, included in the objective function.

$$z_{fi}=q_{fi}\left(y_{fi\omega h}^{\mathrm{REG}}\right)$$

(8)

The following additional constraint is required to limit the up-regulation of a flexible load

$$y_{fi\omega h}^{\mathrm{DR}}\le U_{fi\omega h}^{\mathrm{DR}}$$

(9)

The bounds $U_{fi\omega h}^{\mathrm{DR}}$ are used to allow extra upward regulation during off-peak hours proportionally higher than in peak hours (asymmetric regulation). In other words, it allows load valley filling and restricts upward regulation precisely in peak hours. The upper bounds are year, scenario and flexible group specific and mathematically defined in Appendix C.

3.2.1. Shiftable Volume Load

A shiftable volume load in a time window $(T_{f1},T_{fH})$, indexed by $f\in \mathcal{S}$, can be replaced by a new load provided that their total energy is equal. In other words, the new energy consumption is the same as the original. This is represented as constraint (10) named DR-balance [41]. A shiftable volume load satisfies Equations (3)–(11). For each hour h the demand variables $y_{fi\omega h}^{\mathrm{DR}}$ representing the shifted load, can have a larger or lower value than the original $y_{fi\omega h}^{\mathrm{DRCAP}}$. The time window sub-intervals ${\left\{W_{fj}\right\}}_{j\in \mathcal{J}}$ define the time span during which the load can be shifted and their size is limited by the maximum load-specific time shift parameter $t_f^{\mathrm{shift}}$, $|W_{fj}|<t_f^{\mathrm{shift}}$. They must be intervals forming a partition of interval $(T_{f1},T_{fN})$.

$$\begin{array}{cc}\hfill \sum _{h\in W_{fj}}{y}_{fi\omega h}^{\mathrm{DR}}& =\sum _{h\in W_{fj}}{y}_{fi\omega h}^{\mathrm{DRCAP}}j\in \mathcal{J}\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill y_{fi\omega h}^{\mathrm{DR}}& =y_{fi\omega h}^{\mathrm{DRCAP}}h\in \mathcal{H}\backslash (T_{f1},T_{fN})\hfill \end{array}$$

(11)

3.2.2. Curtailable Load

Curtailable loads$\mathcal{C}$ can be adjusted from zero to the original value in a predefined time window $(T_{f1},T_{fH})$, which means that they have to satisfy Equations (3)–(9) and the constraints (12)–(14).

$$\begin{array}{cc}\hfill y_{fi\omega h}^{\mathrm{DR}}& \le y_{fi\omega h}^{\mathrm{DRCAP}},h\in (T_{f1},T_{fH})\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill y_{fi\omega h}^{\mathrm{DR}}& =y_{fi\omega h}^{\mathrm{DRCAP}},h\in \mathcal{H}\backslash (T_{f1},T_{fH})\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill y_{fi\omega h}^{\mathrm{DR}}& \ge 0,\omega \in {\mathsf{\Omega }}_i,i\in \mathcal{I},f\in \mathcal{C}\hfill \end{array}$$

(14)

Please note that shiftable resulting loads are allowed to be higher than the potential load, but curtailable loads are constrained by the potential load in (12). Because both shiftable resulting load and curtailable resulting loads are required to be non-negative, the downward regulation can be as low as $-y_{fi\omega h}^{\mathrm{DRCAP}}$.

3.2.3. Interruptible Load

Interruptible loads switching on or off are represented by the binary variable $\delta$ in a predefined time window $(T_{f1},T_{fH})$ as in Equation (15). They satisfy constraints (3)–(9), (15) and (16).

$$\begin{array}{cc}\hfill y_{fi\omega h}^{\mathrm{DR}}& =\delta ·y_{fi\omega h}^{\mathrm{DRCAP}},h\in (T_{f1},T_{fH})\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill \delta & \in \{0,1\},\omega \in {\mathsf{\Omega }}_i,i\in \mathcal{I},f\in \mathcal{C}\hfill \end{array}$$

(16)

3.3. DR Capacity

In the first-stage problem, DR investments and capacities are bounded. The upper bound capacity ${\mathsf{{\rm Y}}}_{fi}^{\mathrm{DR}}$ is set exogenously. The maximum investment in DR capacity in year i for load $f\in \mathcal{F}$ is denoted by $X_{fi}^{DR}$. The relation between aggregated available capacity ${\upsilon }_{fi}^{\mathrm{DR}}$ of load f in period i and investments $x_{fi}^{DR}$ depends on the operational life of the investment $DRL$.

$$\begin{array}{cc}\hfill x_{fi}^{DR}& \le X_{fi}^{DR}\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill {\upsilon }_{fi}^{\mathrm{DR}}& ={\upsilon }_{f0}^{\mathrm{DR}}·\chi (DRL-i)+\sum _{\begin{array}{c}j=0,\\ j<i\end{array}}^{DRL}{x}_{fi-j}^{DR},\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill {\upsilon }_{fi}^{\mathrm{DR}}& \le {\mathsf{{\rm Y}}}_{fi}^{DR}i\in \mathcal{I},f\in \mathcal{S}\cup \mathcal{C}\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill \chi \left(x\right)& =\left\{\begin{array}{cc}0\hfill & x\le 0\hfill \\ 1\hfill & x>0\hfill \end{array}\right.\hfill \end{array}$$

(20)

3.4. The New Objective Function

The operation of flexible loads is executed at a fixed a priori cost. If a flexible load diverges at a given hour from the value of the original load, then that change is penalized by a cost (8). Therefore, the load deviation $y^{\mathrm{REG}}$ is penalized either if it is positive or negative. In this model the costs are given by a convex piece-wise linear symmetric function, parametrized as follows. Abscissae $y_{fi\omega h}^{\mathrm{REG}}$ are parametrized by variables ${\lambda }_{fi\omega ph}^{\mathrm{REG}},p\in \mathcal{P}$, that form a Special Ordered Set (SOS2) through constants ${\mu }_{fp}^{\mathrm{REG}}$ as shown in Equations (21)–(23), which are defined to be symmetric around the origin. Since the cost function is built to be convex there is no need to enforce the constraint that two consecutive $\lambda$ to be non-zero. Ordinates $q_{fp+1}^{\mathrm{REG}}$, i.e., marginal costs, which are symmetric for load increase and decrease, are increased proportionally by the ratio $m_{\mathrm{REG}}$ on each piece (see Figure 2). More specifically, the first interval has costs of operation given by input data, denoted by $Q_{fi}^{\mathrm{REG}}$ in the cost function. The following intervals have increased marginal costs by the factor $m_{\mathrm{REG}}$ (24). In this way it is more and more costly to produce a change in the load profile (Please note that curtailable loads, since they can only be decreased, do not need variables ${\lambda }_{fi\omega ph}^{\mathrm{REG}}$ on the positive axis, or equivalently, $q_{fp}^{\mathrm{DR}}=0,p\in {\mathcal{P}}^{+}$ ).

$$\begin{array}{cc}\hfill y_{fi\omega h}^{\mathrm{REG}}& =\sum _{p\in \mathcal{P}}{\mu }_{fp}^{\mathrm{REG}}{\lambda }_{fi\omega ph}^{\mathrm{REG}}\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill {\mu }_{f(-p)}^{\mathrm{REG}}& =-{\mu }_{fp}^{\mathrm{REG}},y_{0f}^{\mathrm{REG}}=0,p=1\dots ,\left|\mathcal{P}\right|/2\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill \sum _{p\in \mathcal{P}}{\lambda }_{fi\omega ph}^{\mathrm{REG}}& =1\forall \omega ,i,f\in \mathcal{S}\cup \mathcal{C}\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill q_{fp+1}^{\mathrm{REG}}& =q_{fp}^{\mathrm{REG}}+{\beta }_f{m}_{\mathrm{REG}}^p{Q}_{fi}^{\mathrm{DR}}·\left({\mu }_{fp+1}^{\mathrm{REG}}-{\mu }_{fp}^{\mathrm{REG}}\right)\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill q_{f(-p)}^{\mathrm{REG}}& =q_{fp}^{\mathrm{REG}}p=0,\dots ,\left|\mathcal{P}\right|/2,f\in \mathcal{S}\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill q_{f0}^{\mathrm{REG}}& =0\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill {\beta }_f& =\left\{\begin{array}{cc}0.5\hfill & f\in \mathcal{S}\hfill \\ 1\hfill & f\in \mathcal{C}\hfill \end{array}\right.\hfill \end{array}$$

(27)

The objective function is extended with two new terms: the investments in DR capacity and the cost of operation of flexible loads. The marginal costs $q_{fp}^{\mathrm{REG}}$ are calculated through Equations (24)–(27). The investment cost per unit of power for group f is denoted by $c_{fi}^{\mathrm{DR}}$. The rest of the investment costs are denoted by $c_{ei}^{\mathrm{tech}}$ where the index e is an index of the set of generators, arcs or storages (both for power storage and energy storage). The short run marginal cost of generator g in period i is represented by $q_{gi}$. The parameter u is the number of years between each investment period i (usually 5 years)

$$\begin{array}{ll}minz& ={\displaystyle \sum _{i\in \mathcal{I}}}{(1+r)}^{-u(i-1)}\times \\ & \underset{\mathrm{Investment}\mathrm{cost}\mathrm{for}\mathrm{generation},\mathrm{transmission},\mathrm{DR}\mathrm{and}\mathrm{storage}\mathrm{capacity},\mathrm{period}\mathrm{i}}{\underbrace{\left\{{\displaystyle \sum _{g\in \mathcal{G}}}{c}_{gi}^{\mathrm{gen}}{x}_{gi}^{\mathrm{gen}}+{\displaystyle \sum _{a\in \mathcal{A}}}{c}_{ai}^{\mathrm{tran}}{x}_{ai}^{\mathrm{tran}}+{\displaystyle \sum _{f\in \mathcal{F}}}{c}_{fi}^{\mathrm{DR}}{x}_{fi}^{\mathrm{DR}}+{\displaystyle \sum _{b\in \mathcal{B}}}{c}_{bi}^{\mathrm{storPW}}{x}_{bi}^{\mathrm{storPW}}+c_{bi}^{\mathrm{storEN}}{x}_{bi}^{\mathrm{storEN}}\right.}}\\ & +\vartheta {\displaystyle \sum _{\omega \in {\mathsf{\Omega }}_i}}{\pi }_{\omega }{\displaystyle \sum _{s\in \mathcal{S}}}{\alpha }_s{\displaystyle \sum _{h\in {\mathcal{H}}_s}}\underset{\begin{array}{c}\mathrm{System}\mathrm{operation}\mathrm{cost}(\mathrm{all}\mathrm{nodes}n),\\ \mathrm{generation}+\mathrm{cost}\mathrm{of}\mathrm{DR}\mathrm{operation}+\mathrm{value}\mathrm{of}\mathrm{lost}\mathrm{load},\\ \mathrm{period}i\mathrm{scenario}\omega ,\mathrm{season}s,\mathrm{hour}h\end{array}}{\underbrace{\left.{\displaystyle \sum _{n\in \mathcal{N}}}\left[{\displaystyle \sum _{g\in {\mathcal{G}}_n}}\left(q_{gi}^{\mathrm{gen}}{y}_{gi\omega h}^{\mathrm{gen}}\right)+{\displaystyle \sum _{f\in {\mathcal{F}}_n,p\in \mathcal{P}}}{q}_{fp}^{\mathrm{REG}}{\lambda }_{fi\omega ph}^{\mathrm{REG}}+q_{ni}^{\mathrm{ll}}{y}_{ni\omega h}^{\mathrm{ll}}\right]\right\}}}\end{array}$$

(28)

and $\vartheta$ = ${\displaystyle \sum _{j=0}^{u-1}}{(1+r)}^{-j}=\frac{{(1+r)}^u-1}{r{(1+r)}^{u-1}}$ is a recovery factor to count the present value of each year within the investment period.

4. Demand Response Baseline Case

This section presents a study of the European power system and demonstrate the effects of DR. First follows an overview of the data used.

4.1. Demand Response Data

The DR potential upper bound, ${\mathsf{{\rm Y}}}_{fi}^{DR}$, comes from the estimated DR capacity in Gils [31]. This is also the main source for the values of the other parameters used in the DR module. In addition, Gils [50] provides hourly and seasonal variations for DR loads, and Gils [33] contains both investment and operation prices for each DR technology. Initial DR potentials come from Baker [29] where current European flexible potentials are assessed. Here the countries with highest DR potential are Belgium, Germany, Spain, France, Great Britain, Ireland, The Netherlands, Poland, Slovenia, and Sweden (This potential is measured by the ratio between the DR installed capacity and the country’s peak demand).

The DR technologies that represent flexible demand in the model are aggregated in the following 7 groups (see also

Table A1. DR technology costs for the 7 groups considered in the model and other technology costs for comparison. Sectors: 1. Residential, 2. Industrial, 3. Commercial, DR source: Gils [33].

Technology	Sector	Investment Cost (k€/MW)	Fixed OM (€/MW) pr. yr.	Variable OM (€/MWh)	Efficiency ${\mathit{\eta }}_{\mathit{f}}^{\mathbf{DR}}$	Max. Time Shift (h)
1. Heating & AC	All	250	7500	10	0.97	2
2. HVAC	2, 3	10	300	5	0.97	2
3. Cooling & Water	2, 3	5	150	20	0.98	6
4. Process Shift	2	0	0	150	0.99	24
5. Washing appliances	1	30	900	50	1.00	6
6. Heat Storage	1, 3	20	600	10	0.98	12
7. Process Shedding	2	0	0	1000	1.00	-
Gas OCGT		400	1950	0,45	-
Battery Storage (Zn)		588			0.75
Battery Storage (Li-ion)		400			0.88
Pumped-Storage Hydro		1000			0.80

): Heating and air conditioning (AC) in all sectors are flexible since they can be controlled according to the network and weather conditions. Heating Ventilation and AC (HVAC) are specific of large industries and commercial centers. Cooling and water includes all industrial and commercial appliances aimed to manage large food cooling systems or water supply systems. The process shift group includes the industrial processes pulp, paper, recycling paper, and cement. All of them allow a shift production hourly. Washing appliances comprises household appliances. Heat storage consists of appliances and installations that allow thermal storage. Finally, process shedding are industrial processes that can be partially stopped: aluminum, copper, zinc, chlorine and steel [31] (

Table A2. DR potentials in GW. Adapted from Gils [33].

Country	Class 1	Class 2	Class 3	Class 4	Class 5	Class 6	Class 7	Total
Europe	23.2	22.7	7.8	9.5	63.4	156.1	8.8	291.5
France	3.0	3.6	1.3	1.6	8.1	26.0	0.8	44.3
Germany	4.2	3.1	1.1	1.4	13.1	17.8	1.7	42.5
Great Brit.	2.7	2.3	0.8	0.5	9.1	21.5	0.3	37.1
Italy	2.7	2.6	0.8	0.9	6.8	16.2	1.3	31.4
Spain	2.2	2.8	0.8	0.8	4.1	9.6	1.2	21.5
Poland	1.2	1.0	0.4	0.3	2.8	8.5	0.3	14.5
Sweden	0.5	0.8	0.3	0.8	1.6	8.7	0.2	12.8
Norway	0.2	0.7	0.2	0.3	0.8	6.8	0.5	9.6
The Netherlands	0.7	0.8	0.3	0.2	3.0	3.6	0.1	8.8
Finland	0.2	0.4	0.2	1.0	0.8	5.4	0.2	8.1
Romania	0.6	0.2	0.1	0.2	1.1	4.0	0.4	6.5
Belgium	0.5	0.5	0.2	0.2	1.7	2.8	0.3	6.3
Greece	0.6	0.6	0.2	0.2	0.9	3.2	0.5	6.2
Austria	0.4	0.3	0.1	0.2	1.3	2.5	0.1	4.9
Czech R	0.4	0.3	0.1	0.1	0.9	2.5	0.1	4.4
Switzerland	0.4	0.4	0.1	0.1	1.2	2.0	0.1	4.2
Hungary	0.4	0.3	0.1	0.1	0.7	2.4	0.1	4.1
Portugal	0.4	0.4	0.1	0.2	1.1	1.8	0.1	4.1
Denmark	0.3	0.2	0.1	0.1	1.1	1.6	0.0	3.4
Slovakia	0.2	0.2	0.1	0.1	0.5	1.6	0.1	2.7
Bulgaria	0.3	0.2	0.1	0.1	0.4	1.4	0.1	2.5
Ireland	0.2	0.2	0.1	0.1	0.6	1.3	0.0	2.4
Serbia	0.2	0.1	0.0	0.1	0.4	1.1	0.0	2.0
Croatia	0.2	0.1	0.0	0.1	0.3	0.6	0.0	1.4
Lithuania	0.1	0.1	0.0	0.0	0.3	0.8	0.0	1.3
Slovenia	0.1	0.1	0.0	0.0	0.3	0.6	0.1	1.1
Latvia	0.1	0.1	0.0	0.0	0.1	0.7	0.0	0.9
Bosnia H	0.1	0.0	0.0	0.0	0.1	0.5	0.1	0.8
Estonia	0.1	0.1	0.0	0.0	0.1	0.4	0.0	0.7
Luxemb.	0.0	0.0	0.0	0.0	0.1	0.1	0.2	0.5
Macedonia	0.0	0.0	0.0	0.0	0.1	0.3	0.0	0.5

shows the 2050 potentials).

The DR investment costs range from 0 to 250 k€/MW. The heating and air conditioning group has the highest investment cost because it requires installing remotely controlled equipment. Industrial DR investment costs are non-existent because DR is an inherent feature of industry operation, at different degrees according to each industrial sector. On average the DR investment costs are 45 k€/MW. The lower investment costs plus fixed O&M costs on DR technologies makes them competitive against lowest-cost generators (OCGT) and storage. However, variable O&M costs and capacity availability have also an impact on the optimal solution. The rest of generation technologies costs can be found in [49,51]. The group with the largest load-shifting time is the industrial shifting with 24 h, characteristic of a medium-term planning process, followed by heat storage with 12 h.

All the DR groups are shiftable volume loads, except for industrial process shedding which is a curtailable load type. Interruptible loads are not represented, as the problem size in the European model would make it intractable with flexible loads that require binary variables. The transport sector could not be included in the study case as load profiles for charging EVs were not available. The total energy demand by EVs is included in the study through the yearly demand input data. The electricity demand trends are based on the European commission 2016 reference case [1]. DR costs are homogeneous across countries and investment periods. The DR groups characteristics are presented in Table A1.

4.2. Short-Term Dynamics

Country load series are obtained from [52,53]. Wind and solar series are provided by the Renewables Ninja dataset [54,55,56], based on reanalysis data MERRA-2 by NASA [57] (Both data from load and IRES have been prepossessed to correct data measure errors and synchronize historical data in the period from 2010 to 2016).

The scenario generation routine based on moment matching creates the scenario tree representing the short-term uncertainty by sampling from a 7 year database consisting of hourly onshore wind, offshore wind, solar, and load data (The hydropower series are based on a synthetic approach that takes into consideration the countries maximum production [49]). It is based on [21] and finds a scenario tree that matches the mean, variance, skewness, and kurtosis of the historical data. Each scenario consists of 4 seasons with 168 consecutive hours each for load, wind, solar, and hydro, sampled from the same point in time across all countries. In addition, there are 24 extra hours for two extreme operating conditions: (a) Day with single highest peak net load and (b) Day with lowest global IRES energy generation. That makes 672 h per scenario, in 10 scenarios per short-term period over 9 long-term periods, making up 60,480 representative hours in total for the model horizon. The simulations performed with these input data are stable with respect to the scenario trees.

To represent the operational conditions of several years using representative hours, the first step is to cluster the data into four seasons. Within each season, a sample is a segment of d consecutive days with 24 h each. In the underlying data for a year, a season of 3 months has exactly 91 days or 2186 h. This gives $91/d$ groups of d days. If Y is the number of years in the sample space, then in total $91Y/d$ groups configure that season’s sample space. Each short-term scenario has 4 seasons (linked to an investment period) represented by 4 sampled seasons with d consecutive days, thereby keeping the chronological order for all $h=1,\dots ,24d$ in each season. A set of S scenarios makes a scenario tree. The scenario trees that best approximate the statistical properties of the random variables are used as input data. The scenario trees are sampled independently. This approach guarantees that the autocorrelation and cross-correlations of the population time series is kept for the representative days within a season.

4.3. Baseline Case Analysis

In the baseline case the upper bound for the DR is set exactly at the DR potential of each DR group in each country (Table A2).

There are considerable DR capacity increases in most countries during the investment period (Figure 3). The DR-to-peak ratio (DR capacity divided by peak demand) and the DR to IRES ratio (DR capacity divided by total IRES generation capacity) give insight on the DR uptake. In the final period (2050) the total DR-to-peak ratio amounts to 34.5% of European winter season peak demand, while the initial DR-to-peak ratio is 0.8%. Still, there are variations between countries, as shown in

Table 1. 10 highest DR capacities in 2050 and initial DR capacity in 2015 together with other ratios.

Country	DR Capacity (GW)	Initial DR Capacity (GW)	DR to Peak Ratio (%)	DR to IRES Ratio (%)	IRES Capacity (GW)	Total Capacity (GW)
Italy	21.7	0.0	31.0	11.2	194.2	219.6
Spain	15.6	1.1	29.6	4.8	322.3	343.7
France	8.5	1.3	6.8	5.0	168.8	242.3
Germany	7.6	0.4	6.5	8.0	95.3	221.6
Greece	4.6	0.0	38.6	9.0	51.0	58.2
Great Britain	4.5	0.7	5.4	5.9	76.7	117.2
Romania	3.0	0.0	25.2	7.4	40.8	56.2
Hungary	3.0	0.0	40.8	5.4	54.5	55.1
Portugal	2.6	0.0	27.1	11.7	22.6	28.5
Sweden	2.2	0.1	6.6	6.5	33.2	50.0
Europe	91.0	4.2	14.5	5.8	1575.1	2059.4

, with the lowest ratio in the United Kingdom (5.4%) and the highest in Hungary (40.8%). When it comes to the global DR to IRES ratio, the variation goes from 0.4% to 5.8% during the optimization horizon. The top 10 countries show a moderate DR to IRES ratio ranging from 4.8% to 11.7%. The top-10 countries have a 6.9% DR to IRES ratio and accumulate 81% of European installed flexible loads.

4.3.1. DR Operation

The first DR impact is on the hourly load profile. In all countries and all long-term periods, the results show that there is load upward regulation when the inter-hour price difference is higher than the cost of flexible load arbitrage. The net load profile is defined as load minus IRES production (Figure 4). When the IRES production varies it creates peaks and valleys in the net load. The valleys, are used for upward regulation and there is downward regulation during the peaks. The effect is especially stressed in countries with high solar capacity such as Spain (Figure 4). A 48 h load sample shows DR events like peak shavings in the order of 10 GW in Spain and 3 GW in France in the load profile during winter. DR regulation events depend strongly on the daily solar peak production creating substantial electricity price differentials, as opposed to wind power which is more stable throughout. Another factor influencing DR is the power mix of each specific country. Systems dominated by dispatchable power plants have less price volatility.

4.3.2. Analysis of DR and System Capacities

In 2050, the global DR activity amounts to 95.8 TWh, which is 2.4% of the yearly net generation and 22% of total available flexible energy. Across the long-term planning horizon a total of 295 TWh have been changed by DR loads, while 1274 TWh keeps the original load pattern. In this part the system’s installed capacity and energy mix are analysed (Figure 5).

The major investment in DR capacity occurs in the sixth period (27 GW). The DR groups in descending order of capacity are (6) heat storage (41 GW), (2) HVAC (23 GW), (4) process shift (10 GW), (7) process shedding (8 GW), (3) cooling and water (8 GW), (5) washing appliances (1 GW) and (1) heating and AC with (0.3 GW). The reasons that groups 2, 4 and 6 have the largest capacities are diverse: (a) heat storage has the 2nd largest shifting time Table A1, (b) HVAC has the lowest variable cost and (c) industry process shift has the largest time window even though its variable cost is the second largest (Since the investment cost of groups industry process shift and process shedding are zero their capacity is adjusted a posteriori to their actual maximum operational values in each country). The three largest total investments are led by the same groups heat storage, HVAC and process shift with 38 GW, 22 GW, and 10 GW, respectively. There are no investments in groups heating and AC, and washing appliances. The former has high investment cost. The latter has cost characteristics disadvantageous in comparison to HVAC (Table A1). Investments in groups cooling and shedding represent only 18% of all DR investments, leaving the main share, 82%, for groups 2, 4 and 6. For country average, the capacity of groups 2 and 3 reaches full potential in period 4, while the groups 1, 5 and 6 do not reach the capacity’s upper theoretical potential at the end of the horizon.

In the final period, 7 out of 10 countries with high installed IRES (onshore wind, offshore wind and solar) are among the 10 highest regarding DR installed capacities. The 10 countries with the highest DR capacity in 2050 have a total of 80% of European DR capacity, 67% of IRES installed capacity and 68% of total installed capacity.

The renewable that receives the most investments globally is solar, with an accumulated increase from 2010 to 2050 of 748 GW, followed by onshore wind with 718 GW. The main solar investment happens in 2035 with 234 GW. This corresponds to an annual increase rate of 17% with respect to solar capacity in 2030. Considering all the renewable technologies, there are 9 countries with more than 90% IRES share in 2050 (of which only Spain and Hungary are also in the previous top-10). Europe’s IRES share amounts to 76%.

Regarding technologies that can compete with DR by providing generation flexibility to the system, ramping capacity, and peak supply, it is interesting to see the changes in the following gas technologies. Existing stock gas turbines, Open Cycle Gas Turbines (OCGT), Closed Cycled Gas Turbines (CCGT) and gas with Carbon Capture and Storage (CCS), whose high ramping rates, can supply power during residual demand peak periods. This group of gas turbines has an increase of 35% caused mainly by gas CCGT power and investments in CCS from 2040. At that point, backup capacity from CCS substitutes the already abated nuclear power.

In a second group, regulated hydro power, hydro pump storage, and battery storage can store energy for periods with scarcity of supply. Their installed power capacity has a slight increase of 21% between 2010 and 2050. Regulated hydro power and hydro pumped storage have moderate increases at around 16%. Battery storage becomes cost-effective exactly when the solar capacity starts the large investments by 2030–2040.

Transmission capacity sums up to 501 GW by 2050, with the major investments happening when solar capacity reaches system dominance. This represents a significant increase of 700% interconnector capacity. The most reinforced transmission corridors are Spain–France, France–Great Britain, Poland–Germany, Poland–Lithuania, Poland–Czech Republic and Italy–France with total capacities in 2050 of 83 GW, 52 GW, 30 GW, 13 GW, 13 GW, and 12 GW respectively. The countries with highest aggregated transmission capacity in 2050 are France (179 GW), Germany (104 GW) and Spain (89 GW).

The emission constraints imposing a linearly decreasing emission cap from 2010 to 2050 (90% reduction with respect to 2010s emissions in EU aggregated) is binding in all periods. The carbon price (Given as the shadow price of the emission constraint) decreases from 41 €/EUA in 2020 to 32 €/EUA in 2030. From that point carbon price increases steadily to 86 €/EUA in 2050. In the results, there are many countries that decrease their emissions by 100% in 2050: Norway, Sweden, Finland and Austria (although they might be dependent on imports). In total there are 14 countries with reductions of more than 98%.

Average electricity cost increases from 42.0 €/MWh by 2020 to 53.2 €/MWh by 2050. The increase is caused mainly by increased investment costs, despite that non-polluting technologies have decreasing costs. Part of the increase is caused by high costs of CCS technologies, which are necessary to cover the demand. Investments in CCS start in year 2035 with 4 GW total installed, then a 10-fold increase in 2040, reaching at the final period a capacity of 85 GW in total.

The results above do not consider the use of DR in ancillary services or for local flexibility markets. This would most likely require a finer time resolution and geographical resolution in the model. In moments with limited DR capacity, part of it would be reserved for other uses than the ones represented in EMPIRE. The value of flexibility may be even higher in those applications.

4.4. No DR Case

The model outcomes with disabled DR investments and operation are compared to the baseline case, and the four main differences can be seen in

Table 2. Main differences between cases DR0 and DRB in 2050.

Metric	DRB	DR0	Difference %
Electricity Generation (TWh)	4019.30	4023.30	−0.10
IRES capacity (GW)	1575.10	1537.70	2.37
Solar PV (GW)	773.90	744.90	3.75
IRES generation (TWh)	2726.60	2677.70	1.79
IRES curtailment (TWh)	118.80	109.50	7.83
Total Installed Capacity (GW)	2059.40	2053.40	0.29
Transmission Capacity (GW)	501.10	499.40	0.34
Emissions (MtCO2) 2010–2050	6399.00	6401.00	−0.03
Peak Plants (GW)	293.00	324.00	−10.58
Electricity Price (/MWh)	53.20	54.50	−2.44
Storage Capacity (GW)	59.80	111.40	−86.29

. The difference in IRES installed capacity is caused by a higher solar capacity in the baseline (DRB) than in the no DR case (DR0), which is a consequence of the availability of shifting load to high solar production periods. Flexible loads allow taking advantage from solar production, which has 75% lower investment cost than onshore wind. This leads to the second main difference: a 7.8% higher curtailment of IRES in DRB than in DR0, which is explained by the fact that in certain days the solar production is high but not enough to make a significant price difference that can be economically efficient for more load-shifting. That electricity surplus that alternatively could be used in other sectors is not included in the model.

Despite the curtailment difference in 2050, the IRES annual generation is approximately the same until 2035, where DRB shows more production from IRES and less from peak plants (by 2050 in DRB there is only 1.8% more production). If we look into wind and solar separately, DRB has more generation of both from 2030 to 2050 except only for 2040 (when a high investment in solar capacity causes a change in the sign of production).

The third difference is that flexible loads allow significant savings on peak plants in DRB because the net demand peaks are reduced. Finally, the fourth difference appears in power storage capacity, with 86% more capacity in DR0 than in DRB. The battery energy capacity is 18 times higher in DR0. In this case, it is clear that the availability of flexible loads reduces the needs of investments in storage infrastructure. However, the DR loads cannot substitute all the storage power and energy capacity needed.

When it comes to generation from the peak power plants, there are low differences in 2020 and 2025 when solar still has low shares in the energy mix (Figure 6). In 2035 the electricity generated from peak plants is higher in DR0 than in DRB. From next period a trend switch starts: in DRB there is more generation from CCGT and less from CCS than in DR0. Therefore, only in peak plants, the DRB’s energy mix has higher emission intensity (which is compensated by a major IRES electricity production). The main 2050 generation differences comes from CCS, onshore wind and solar with −68 TWh, +17 TWh and +32 TWh, respectively.

The overall average electricity price without DR technologies follows the same evolution as in DRB but are 2% higher from 2035 due to a lower IRES integration and the need for gas CCS. The CCS capacities in DR0 are two times higher than 2035 DRB ones, and decrease until 2050 when they set at 16%.

The differences in energy generation reflect the differences on capacity portfolio (and vice versa). Total generation investments are in fact very similar in both cases (2059 GW). Aggregating storage power and transmission capacity to the mix results in DR0 requiring in fact 50 GW more than DRB. These 50 GW are compensated in DRB by the virtual infrastructure (or ICT capacity) of flexible consumers, with 91 GW.

It is interesting to analyse the capacity mix in 2035 because by then there is a huge 25 GW DR investment, solar PV rise and current fossil fuel plants are about to be decommissioned. In DRB there is more onshore wind and solar capacity, while in DR0 there are more peak plants (Figure 7). That is the moment when both cases start to diverge. From 2045 to 2050, DRB’s capacity portfolio is based on solar and DR flexible loads, while the DR0 case has to make use of gas CCS, onshore wind, and solar production through storage (The portfolio of lignite and coal plants is the same in both cases until 2050 with minor differences in generation).

4.5. Sensitivity Analysis

4.5.1. Capacity Sensitivity

A sensitivity analysis is performed on the total DR capacity of each country. The upper limit is specified relative to the theoretical DR potential in DRB by using the following scaling factors: 0.5, 0.75, 1.25 and 1.5 compared to the DR capacity in DRB (Table A2), for cases DR1 to DR4, respectively. In the model this is implemented through constraints (19).

Table 3. Relative differences of each sensitivity case in percentage with respect to the reference case DRB (%).

Metric	DR0	DR1	DR2	DR3	DR4
DR Capacity	−100.0	−41.8	−17.2	27.0	46.8
CAPEX	1.3	0.1	0.0	0.0	−0.1
OPEX	2.0	0.7	0.4	−0.4	−0.7
Net Generation	0.1	0.0	0.0	0.0	0.0
iRES cap	−2.4	−1.4	−0.6	0.7	1.4
Solar cap	−3.7	−2.9	−1.3	1.6	3.1
iRES generation	−1.8	−1.0	−0.5	0.5	0.9
Curtailed generation	−7.8	−2.1	−1.3	1.1	2.1
Total capacity	−0.3	−0.8	−0.3	0.4	0.8
Transmission Capacity	−0.3	−0.6	−0.2	0.1	0.1
Emissions 2010-2050	0.0	0.0	0.0	0.0	0.0
Storage power capacity	86.3	13.7	7.0	−3.5	−5.4
Storage energy capacity	3.8	0.7	0.3	−0.1	−0.2
Average electricity cost	2.4	1.1	0.6	−0.4	−0.8
Peak plants	10.7	2.3	1.0	−0.9	−1.8
of which Gas CCS	16.2	5.9	2.5	−2.6	−4.9

shows the percentage variation with respect to the reference case DRB. We see that relaxing the DR total capacity constraint (DR3 and DR4) allows more DR capacity into the system, more solar PV, more IRES curtailment, less peak plant capacity globally and less gas CCS. The sequence of cases shows around 20% change in DR capacity while solar capacity changes by 1% and storage power capacity by 2–6%.

The groups with highest DR capacity in all the cases are groups 6 (heat storage), 2 (HVAC) and 5 (washing appliances), except in DR4 where group 3 takes group 5’s third position. In particular, in DR4 their capacities are 234 GW, 30 GW, and 11.7 GW respectively, which added to the rest of the group totals 315 GW.

4.5.2. Costs Sensitivity

To analyse how costs affect the uptake of DR capacity, sensitivity cases DRC1-DRC5 scales both the investment and operational costs of all the DR groups with the following factors: 0.25, 0.5, 2, 4, and 8. In

Table 4. DR costs sensitivity’s percentage variation analysis respect to DRB (%).

Metric	DRC1	DRC2	DRC3	DRC4	DRC5
DR Capacity	205.4	124.1	−39.8	−76.9	−76.9
CAPEX	−1.6	−0.6	0.5	0.6	0.7
OPEX	−10.7	−5.9	2.0	2.6	−6.4
Net Generation	−1.6	−0.5	0.1	0.1	0.1
iRES cap	11.9	10.2	−3.7	−3.8	−3.6
Solar cap	29.7	23.2	−6.8	−7.2	−6.8
iRES generation	6.3	5.6	−2.8	−2.8	−2.7
Curtailed generation	11.2	17.9	−7.4	−7.1	−7.1
Total capacity	6.2	5.7	−1.7	−1.4	−1.2
Intercon. cap	6.2	3.3	0.8	0.5	0.5
Emissions	0.0	0.0	0.0	0.0	0.0
Storage power cap	−9.0	−10.2	20.1	22.9	37.5
Storage energy cap	−0.2	−0.2	0.8	0.9	1.6
Average electricity cost	−7.5	−5.6	2.8	3.2	3.0
Peak plants	−22.7	−15.1	7.7	10.9	10.8
Of which Gas CCS	−52.9	−40.1	20.2	20.2	19.6

the main differences between cases are represented in percentage. As expected the amount of DR capacity decreases with increasing costs. There is less IRES, solar capacity and generation when less DR capacity is available. IRES curtailment is higher in DRC1 and DRC2 than in DRB because they have higher solar capacity. High costs of DR make solar capacity less efficient as can be observed in cases DRC3 to DRC5. The reason is that operational costs in DRC1 and DRC2 make it efficient to shift more energy to the IRES production hours. In DRC4 and DRC5 the curtailment is lower than in DRB since there is less IRES hourly surplus during the normal and low demand seasons. Storage capacity differences are significant (−9% to 38% in DRC1–DRC5), with storage and DR capacity as substitute technologies. High differences in storage capacity arise in 2035 and onwards in cases DRC3–DRC5 in comparison to DRB (Figure 8).

Although total installed capacity decreases with increasing DR costs (due to solar capacity), the capacity of peak plants (+11%) and in particular gas CCS (+20%) increase. Electricity prices are down to -8% cheaper in DRC1 and stay stable at +3% in DRC3 to DRC5.

5. Conclusions

This paper describes how to implement a DR module in a capacity expansion model for the European power markets. The case study includes endogenous DR investments co-optimized with intermittent renewable sources, other generation technologies, storage, and transmission. Flexible demand is aggregated in groups of end consumers that share similar characteristics, investment, and operative costs. Two types of flexible loads have been used in the case study, shiftable volume loads and curtailable loads. These two types of DR can be modelled with linear variables, whereas interruptible loads require binary variables.

DR’s development in Europe has been assessed for seven different end-consumer groups. The results show that the main contributor to load-shifting measured by energy volume is heat storage end uses, followed by industrial process shifting. These are the groups that offer the most flexible timeframes for DR. Investments in DR capacities happen because flexible loads provide a more efficient way to handle intermittent solar energy than alternatives. We see that the countries with the highest IRES capacities also have the highest DR investments. The model applies load-shifting during periods of high solar production and residual load sudden variations.

The adoption of demand-responsive loads has impacts on the power system. Particularly relevant is the fact that gas power plants supplying flexible energy during peak events are partially substituted by the flexibility offered by DR loads. On one hand, DR introduces more degrees of freedom into the model. It allows more IRES integration into the system and fewer capital investments in peak plants. On the other hand, investments in polluting plants are affordable in gas OCGT with low investment costs. In that sense, DR alone is not a technology that would allow a significant reduction of $C{O}_2$ emissions.

Solar PV and onshore wind capacities follow similar paths because the former is more efficient in southern countries while the latter is efficient in the Nordic ones and the British Islands. By 2050 their installed capacities will be 774 GW and 802 GW, respectively. Countries with the highest installed DR capacities are the ones with the highest solar capacity.

The impacts to the power system are not extreme in the sense of turning around the capacity mix. DR capacities act as a support technology by integrating 2% more IRES, saving 11% of peak capacity and 86% of storage capacity, and therefore facilitating the transition towards a system with high shares of renewable sources.