Transformation of Demand-Response Aggregator Operations in Future US Electricity Markets: A Review of Technologies and Open Research Areas with Game Theory

Abstract

The decarbonization of electricity generation by 2030 and the realization of a net-zero economy by 2050 are central to the United States’ climate strategy. However, large-scale renewable integration introduces operational challenges, including extreme ramping, unsafe dispatch, and price volatility. This review investigates how demand–response (DR) aggregators and distributed loads can support these climate goals while addressing critical operational challenges. We hypothesize that current DR aggregator frameworks fall short in the areas of distributed load operational flexibility, scalability with the number of distributed loads (prosumers), prosumer privacy preservation, DR aggregator and prosumer competition, and uncertainty management, limiting their potential to enable large-scale prosumer participation. Using a systematic review methodology, we evaluate existing DR aggregator and prosumer frameworks through the proposed FCUPS criteria—flexibility, competition, uncertainty quantification, privacy, and scalability. The main results highlight significant gaps in current frameworks: limited support for decentralized operations; inadequate privacy protections for prosumers; and insufficient capabilities for managing competition, uncertainty, and flexibility at scale. We conclude by identifying open research directions, including the need for game-theoretic and machine learning approaches that ensure privacy, scalability, and robust market participation. Addressing these gaps is essential to shape future research agendas and to enable DR aggregators to contribute meaningfully to US climate targets.

1. Introduction

According to NASA’s Goddard Institute for Space Studies (GISS) [1], the average global temperature on Earth has increased by about 0.8° Celsius (1.4° Fahrenheit) since 1880. Two-thirds of the warming has occurred since 1975, while the consequences have been catastrophic for food and water security [2], the population living within 100 km of the coast, and international peace and security due to the competition for resources. The aforementioned climate change concerns in combination with concerns about the dependency on decreasing oil resources and political games (i.e., 1973 embargo by Saudi Arabia against the USA, which led to a 300% oil price increase) have directed the USA’s attention away from conventional fuels. Specifically, the Environmental Protection Agency (EPA) proposed the Clean Power Plan (CPP) policy in 2014, which suggests reducing carbon dioxide (CO₂) by 32% in the years 2005–2030 and reducing coal plants’ CO₂ emissions, increasing renewable integration, and investing in energy-saving programs [3].

Although multiple measures have been taken to reduce CO₂ emissions and achieve the CPP policy goals, the climate change crisis continues to persist due to an increase in energy generation emissions in the 2020s in order to cover the increasing global energy demand. Therefore, the CPP policy goals were revised to the more aggressive 2030 generation decarbonization and 2050 net-zero economy goals. Specifically, the International Energy Agency (IEA) set four pillars for action in the 2020s [4], which, if implemented, will be sufficient to keep the global temperature increase under 1.7 °C by 2100, i.e., aligning with the net-zero emissions scenario. These pillars concern [5] deforestation reduction by 2030 and land use decarbonization; the reduction in non-CO₂ emissions; the use of carbon capture, utilization, and storage (CCUS); and the decarbonization of electricity generation by 2030. IEA has summarized the importance of revising existing market instruments, mechanisms, and regulations to achieve the 2030 and 2050 goals [6], knowing that, by 2050, the global electricity demand is expected to more than double compared to 2020. These revisions include the following:

• Deploying significantly more renewable resources and completing the demand coverage with low-carbon resources, such as nuclear and hydrogen, is important [7]. Battery storage and DR are not considered as additional generation capacity but rather important enablers if sufficiently incentivized due to the introduced flexibility [8].
• Incentivizing clean technology market players with effective investment frameworks, carbon pricing, and other decarbonization instruments.
• Improving market price signals to better represent the time and geographical value of the energy.
• Ensuring that hedging with long-term contracts does not harm the short-term market dispatch.
• Offering decarbonization financial instruments that help reduce the long recoupment periods of wind and solar to attract new private investment.
• Incorporating distributed energy resources via aggregators to increase flexibility and resilience in systems with high-penetration renewable resources and defer system upgrade costs. Digitization via metering devices is necessary to increase visibility and ensure fair compensation with adequate customer privacy.
• Facilitating battery storage investments with taxation and network tariffs that appropriately recognize its role as a flexibility provider, ensuring that it is not charged twice as a consumer and as a generator, and compensating for its fast response and flexibility services.
• Rewarding capacities that can respond in periods of high system stress to reduce price volatility.
• Protecting retail customers against market risks, natural gas price peaks, and extreme weather phenomena.
• Planning system expansion to support low-carbon resources (including DR).

As a result of the CPP, 2030, and 2050 policies, the US energy mix has changed significantly in the last decade, with oil and coal resources progressively being replaced by renewable and natural gas-based generation (less polluting than oil and coal; backs up intermittent renewable sources) [9,10]. However, over-investment in renewable resources has introduced manufacturing pollution (renewable devices), natural-gas-related pollution from backing up renewable sources [9,10,11,12], and operational issues. In California (CAISO), the duck curve [13,14] indicates that the increasing trend in solar panel investment can be threatening to the system’s reliability and efficiency for two reasons. First, renewable over-generation can compromise the system’s safe dispatch due to uncertainty in generated power, lack of control, increased marginal prices (aging network congestion [15]), potential blackouts (2001 California crisis [16]), and negative marginal prices (renewable over-supply [17,18,19]). Second, ramping, i.e., sudden renewable production changes, can cause dispatching difficulties since conventional generators’ ramping is limited and they have voltage support issues [20] due to the lack of sufficient reactive power in solar and most wind farms [20].

Similar problems have been observed or are expected to happen in other systems, such as in Irish systems [20], ERCOT, MISO, and NYISO [21]. With the aforementioned operational and investment issues related to renewable resources and battery storage in the CAISO and other US markets, the incorporation of the distributed load, i.e., commercial and residential buildings with a complex net load consisting of demand, local generation, and storage, in DR has been increasingly gaining interest for CO₂ emission reduction and grid operation support (by the Federal Energy Regulatory Commission, also known as FERC) [22]. As distributed loads may be a mix of different loads, battery storage, and generation, their contribution to CO₂ reduction, as well as grid flexibility, will be vital to the 2030 and 2050 goals. However, their large-scale integration via aggregators is difficult to implement due to the load mix complexity behind the meter, aggregator operations’ scalability, prosumer privacy, and uncertainty quantification. Therefore, the need for aggregator operations with the aforementioned characteristics is immediate.

Related Works: Although previous review papers have provided overviews of demand–response (DR) programs [23,24,25,26,27], DR frameworks [23,26,28], DR benefits for systems penetrated by renewable resources [24,29,30], ancillary services offered by DR [27,28,31], customer response to DR programs [24], DR pilot studies [26], demand-side resources [23] and methods [31], potential savings from energy management systems [32], blockchain technologies [33,34,35,36], or local trading methods [37], a critical view on the role that distributed loads will play in future US electricity markets is missing not only in terms of energy savings, reliability, and flexibility but also in terms of the scalability of aggregator operations and the privacy of the prosumer in a future fully digitized smart grid. This review paper provides a holistic investigation of existing DR aggregator and prosumer technologies and frameworks, and proposes future directions for research that are compatible with five identified criteria. Limited related work exists in terms of critical review papers for game-theoretic frameworks on DR aggregators and distributed loads. For example, Ref. [38] covers a breadth of game-theoretic frameworks without proposing a unified evaluation framework to guide researchers on how to facilitate the large-scale and safe integration of distributed loads for the 2030 and 2050 goals. Our paper introduces and uses five evaluation criteria, including scalability, which evaluates the computational complexity of existing frameworks; this is something that is missing from [38], which primarily focuses on modeling. Other review papers in game theory with a scope broader than that of the DR aggregator exist [39,40] but are not directly relevant.

The contributions of this review are threefold. First, a thorough overview and evaluation of existing energy efficiency solutions and technologies for the distributed load are provided, including energy management systems and market-based DR via aggregators. The overview includes the operational and privacy challenges of the distributed load due to its complex nature and the existing state of its integration in different markets via DR aggregators. Second, a critical analysis of the existing DR aggregator and prosumer framework literature in terms of five important gaps is carried out: flexibility, competition, uncertainty, privacy, and scalability (FCUPS framework). These evaluation criteria support the safe and large-scale integration of distributed loads into future US electricity markets and facilitate the 2030 and 2050 goals. Third, future work directions are provided for researchers to develop FCUPS-compatible frameworks that allow for the safe and large-scale integration of distributed loads via DR aggregators in future US electricity markets.

The rest of this paper is organized according to Figure 1. Section 2 analyzes the potential for energy savings and grid reliability improvement from distributed loads in the US and provides an overview of energy-efficient technologies, with a focus on DR aggregators. In Section 3, available frameworks for the DR aggregator’s operation are investigated in terms of flexibility, reliability, uncertainty, scalability, and the consideration of prosumer’s privacy, and missing framework capabilities are identified. This section summarizes open research questions for researchers and market policymakers that will transform the US electricity markets and help achieve the 2030 and 2050 goals.

2. Transformation of Future US Electricity Markets by Distributed Load and Demand-Response Aggregators

2.1. Overview of Energy Efficient Technologies for Distributed Load

In the USA, the second most polluting country in terms of CO₂ after China (Figure 2a), the breakdown of electricity usage by sector shows great potential for energy and CO₂ savings in residential and commercial buildings (distributed loads), which are responsible for 32% of the total CO₂ emissions in the USA (Figure 2b). Several technologies for energy savings and grid support from the distributed load exist today, such as participation in building energy efficiency programs [41], which have increased rapidly over the past 10 years by performing structural upgrades for indoor heating, ventilation, and air conditioning (HVAC) (up to 55% energy can be saved [42]); and investing in smart energy management systems, efficient lighting, sensors [43], and other controls [41]. By investing in these technologies, the USA can save up to 16% of CO₂ emissions by 2050 compared to the 2005 reference. Additionally, another 62% of CO₂ savings is possible with renewable technologies applied on buildings [44,45].

Table 1. Summary of energy-efficient technologies for the distributed load. Energy/CO₂ savings are shown as a percentage of the building’s total demand, calculated by multiplying the savings at the device level by the device’s participation in the total building’s demand. HVAC is estimated to consume about 50% of the total building’s demand [48].

Technology	Energy/CO2 Savings	Equipment/Service	Paper
Efficient equipment	Varies	Appliances and devices	[49,50]
Efficient lighting	Varies	Multiple technologies	[51]
Efficient HVAC	4.5%	Energy Star HVAC	[48]
Sensors	55%	Occupancy detection for HVAC	[43]
	20%	Sensing for light control	[52]
Energy management	Up to 23%	Smart control per device	[32]
Efficient structures	76.8%	Polystyrene insulation	[53]
	50–70%	Passive solar design	[54]
	3%	Double skin facade (DSF)	[55]
	60%	DSF with solar and motor-blinds	[56]
	Varies	Green walls, shape, atrium architecture	[57,58,59]
Renewable integration	62%	Renewable energy supply	[44]
Utility-saving programs	30–40% of controllable load	Direct load control (DLC)	[60]
	3.09%	Conservation voltage reduction (CVR)	[61,62,63]
	64%	Irreducible/curtailable programs	[64]
Energy market	3% in 2004	Peak load DR	[64]
	42%	99% time of usage/1% critical peak pricing (CPP)	[65]
	12–33%	Real-time pricing	[66]

summarizes energy-efficient solutions in buildings.

The large-scale integration of such energy-efficient technologies can provide energy savings and grid reliability towards the 2030 and 2050 goals. However, such integration introduces several operational challenges due to the distributed load’s complex nature, which is analyzed next.

2.2. Complexity of Distributed Load Mix in Future US Electricity Markets

Prosumers (distributed load) may already have (or will have in the future) diverse load characteristics and, therefore, may require different contracts or pricing schemes (Figure 1 in [67]), some of which have been tested in pilot programs [31]. The prosumer’s load synthesis is better explained by Figure 3. Five load sub-types are identified:

• Storable load (i.e., electric cars, batteries);
• Shiftable load (i.e., dishwasher);
• Curtailable load (i.e., lighting);
• Self-generation (i.e., solar panels);
• Baseload, which is not curtailable and therefore not appropriate for DR [68].

The customer may choose if they wish to control their load independently via an energy management system or if they wish to transfer the control of their mixed load to the DR aggregator [69]. However, with the increasing penetration of renewable resources and storage in the load mix, the role of the DR aggregator has become more challenging, not only because of the millions of constraints but also due to the data coupling from a prosumer’s meter. The customer’s load mix shown in Figure 3 is measured as one net demand, and hence, existing forecasting methods for individual load sub-types do not apply [70].

Figure 3. Distributed load mixture [71]. CHP stands for combined heat and power, and IT stands for information technology.

Figure 3. Distributed load mixture [71]. CHP stands for combined heat and power, and IT stands for information technology.

The challenge of managing thousands/millions of prosumers with different load types, device constraints, demand and generation forecasting, and coupled behind-the-meter data creates the need for decentralized DR aggregator operations and for prosumers to schedule their own operations via an energy management system under the coordination of the DR aggregator while having more control over their own data privacy from local sensors and meters (open research questions on distributed load privacy, DR aggregators, and distributed load scalability are provided in Section 3.7). This necessity will be investigated in Section 3, where gaps in the literature and research questions will be addressed properly for the future work of the interested researcher and policymaker. Meanwhile, the next section will introduce energy management systems for distributed loads and emphasize their role in energy efficiency and grid reliability.

2.3. Energy Management Systems for Distributed Load Complexity Management

Energy management systems can manage demands in distributed loads based on optimization algorithms; energy production forecasting from distributed resources, and appliances; and energy storage control [72]. For a distributed load that participates in the wholesale DR market through a DR aggregator [73,74], such energy management systems can schedule appliance and device operations to match with the new demand bid submitted into the market. In other cases, the energy management system responds directly to price signals from the local utility [75]. The related terminology includes Home Energy Management Systems (HEMSs) for residential buildings and Building Energy Management Systems (BEMSs) for commercial buildings. These systems optimize the total operation cost, and in some cases, they also maximize user satisfaction. For examples of implementations, see

Table 2. Examples of sensor-based energy management systems for the distributed load.

System Type	Sensor Inputs	Devices Controlled	Paper
HEMS	Appliance status, renewable energy and energy storage status, electric car battery status	Schedulable appliances, renewable energy storage devices, electric car	[76]
HEMS	Occupancy sensor, PIR, camera	Lights, HVAC, security	[77]
HEMS	Energy prices, user behavior, renewable status, weather, indoor temperature, occupancy, CO2 sensor, fuel cost	HVAC	[78]
HEMS	User behavior, user feedback, indoor temperature, luminosity, humidity, appliance demand, motion sensor, camera	Savings recommendations, lights, HVAC, appliances, devices	[79]
HEMS/BEMS	Energy prices	Heat pump	[80]
BEMS	Indoor temperature	Boiler	[81]
BEMS	Calendars, computer activity, employee badge scanning, Wi-Fi	Lights, HVAC	[82]

.

Energy management systems utilize sensors that receive environmental, biomedical, or device usage signals and an associated controller that makes decisions to modify a device’s demand so that an optimal result is achieved, such as energy efficiency, occupant’s thermal comfort, occupant’s visual comfort, or indoor air quality [83].

Table 3. Overview of sensor-based device control for energy savings at the distributed load.

Sensor Type	Inference	Device Controlled	Paper
Motion sensor	Occupancy levels	Lights	[84]
Photo sensor	Light levels	Lights	[85]
Chair sensor	Occupancy levels	Building systems	[86]
CO2 sensor	Occupancy levels	Lights, HVAC	[87]
Pressure mats	Occupancy levels	LED lights	[88]
Sound sensor	Occupancy levels	Lights, HVAC	[87]
Camera sensor	Occupancy levels	Lights, HVAC, appliances, controls	[89]
Wi-Fi sensor	Occupancy levels	HVAC	[90]
Smart phones	Occupants’ information	HVAC	[91]
Indoor temperature sensor	Occupants’ information	HVAC	[91]
Indoor humidity sensor	Occupancy levels	Lights, HVAC, appliances, controls	[89]
Skin temperature sensor	Thermal sensation	HVAC	[92]
Heart rate sensor	Thermal sensation	HVAC	[92]
Wearable sensor	Occupancy levels	Demand control	[93]
Thermo-fluidic sensor	Occupancy levels	HVAC	[94]

summarizes sensing technologies and applications in building device control for energy savings. Energy management systems may also consider sensor input to schedule the day-ahead demand or to make demand modifications in real time. Specifically, when bidding into the day-ahead market (DAM), an initial demand schedule may be derived for the prosumer, but due to multiple uncertainties, this demand may slightly vary in real time when the day-ahead demand is realized. Therefore, real-time sensor inputs, such as the occupancy state of a room or building, may become real-time suggestions made in energy management systems that modify the operation of appliances, such as lights and HVAC [43], in order to avoid large deviations from the day-ahead demand bid and associated market costs and penalties; this is carried out if the real-time pricing (spot market) is higher than the day-ahead demand for some hour of the day and the prosumer demand has increased for that hour.

The current literature in energy management systems for the distributed load does not consider integration with the prosumer–aggregator game via its pricing signals, as will be better explained in Section 3, and therefore, each prosumer’s day-ahead demand bid is not tested for feasibility according to the prosumer’s schedule, i.e., whether the demand bid is feasible and optimal for the prosumer’s load mix (Section 2.2), leading to potential spot-market-imposed fines due to the day-ahead versus real-time demand discrepancies. This observation leads to the open research questions outlined in Section 3.7, which are related to the distributed load’s HEMS/BEMS flexibility; its competition with the DR aggregator and other prosumers; uncertainty around the day-ahead demand considered by HEMS/BEMS; prosumer’s preferences or personally identifiable (PI) information collected from the distributed load’s HEMS/BEMS; and scalability when multiple device constraints, load types, and forecasting signals are present within HEMS/BEMS or when multiple distributed loads are integrated under the same DR aggregator via their HEMS/BEMS. The next section will introduce market-based DR programs via DR aggregators for the distributed load and emphasize the benefits of market participation for CO₂ emission reduction and grid reliability support.

The next section will explain the current operating mechanism of the DR aggregator, the plethora of information it requires from the distributed load, which may include personally identifiable (PI) data, and the challenges associated with future DR aggregator operations in US electricity markets.

2.4. The Intricate Role of Demand-Response Aggregators in Large-Scale Distributed Load Integration with Safety

DR plays a strategic role in improving the safety and reliability of power systems from a demand-side management perspective [67]. According to [30,69], DR can carry out the following:

• Reduce CO₂ emissions;
• Postpone or prevent the construction of new power plants;
• Improve the power system’s reliability and security (ramping and reshaping demand [24,95]).

DR is a type of demand-side management [65,96] and includes market-based DR programs that are available via DR aggregators, which allow the distributed load to access higher profits than utility-based DR programs (see

Table 4. Some DR aggregators in the USA [28,69]. Selected content from two references is merged.

Company (Capacity)	Customer Portfolio	Strategy	Business Model	Paper
EnerNOC (MA) (∼1000 MW)	Large customer (<1 GW): industrial, commercial	DR design, sell DR to ISOs	Automation, metering, and communication with direct control	[101]
Cpower (MD) (∼2000 MW)	Large customer (<1 GW)	Energy management metering	Strategic energy asset management	[102]
Comverge (GA) (∼500 MW)	Residential	Smart thermostat and web portal	Installation and control, sell DR to utilities and ISOs	[103]
Energy Connect (CA)	Large customer (<1 GW): industrial, commercial, government	Automation, metering and communication	Energy automation service provider	[104]
Energy Curtailment	Large customer (<1 GW): industrial, commercial	Sell DR to utilities	Metering	[105]
Specialist ECS (NY) (∼1000 MW)	Industrial, commercial, services	Sell DR to utilities and ISOs	Software and analytics	—
North America Power Partners NAPP (NJ) (∼500 MW)	Industrial, commercial	Sell DR to utilities and ISOs	Web-based platform for monitoring, self-scheduling	[106]

for a list of US DR aggregators). These programs are very efficient for energy savings, monetary rewards, grid reliability, and overall market performance [95]. DR loads are increasingly participating in different markets (

Table 5. DR participation across different ISOs in the US in 2010 [107] and in 2022 [108]. The available DR is used as a capacity resource and for DR emergencies, ancillary services, and bidding energy.

ISO	Total DR Load		DR as % of Peak Demand
	2010	2022	2010	2022
CAISO	2135 MW	3900 MW	4.5%	36%
ERCOT	1484 MW	4354.5 MW	2.3%	5.9%
ISO-NE	2116 MW	533.7 MW	7.8%	2.3%
MISO	8663 MW	12,197 MW	8%	10.2%
NYISO	2498 MW	1345.5 MW	7.5%	4.4%
PJM	13,306 MW	9914 MW	10.5%	6.8%
SPP	1500 MW	176.2 MW	3.3%	0.3%

), and based on PJM, their main role is that of a capacity resource [97]. The main disadvantage of market-based DR is that although investments in market-based DR are growing (slide 34 in [98]), the total payments are not significantly changing, leading to payment reductions per player over time (diminishing marginal returns). According to a Brattle study [99], “the nationwide benefits of load flexibility could exceed $15 billion per year by 2030 and are driven by avoided investment in generation capacity, reduced energy costs, geographically-targeted transmission and distribution investment deferral, and the provision of ancillary services”. However, it is important to consider that the implementation of DR comes with some overhead cost (administrative, incentive, and enabling technologies like telemetry, hardware, software, communication systems, etc.). Another study performed by FERC for the California grid estimates the total benefits from DR across the 3 years of 2012–2014 to be slightly over USD 50 million, and the costs are estimated to be slightly over USD 40 million [100]. The same study emphasizes that if the period is extended, the cost-effectiveness would improve for DR due to the fact that some of these costs are one-time costs, i.e., equipment cost (Figure 6-1 in [100]). A summary of retailers or DR aggregators offering DR services is provided in Table 4.

DR aggregators can offer demand modifications and grid services, such as shifting or curtailing loads, reducing unbalanced costs, participating in balanced markets, and providing ancillary services [31]: for example, providing voltage support and frequency support as one entity [109]. Existing market entities, such as retailers, balanced responsible parties, and third-party companies, may act as aggregators [69]. For a better understanding of the DR aggregator’s interaction with other market players, please see Figure 1 in [23] and Figure 8 in [69]. A DR aggregator is cleared by the energy market similarly to an energy producer or generator, since reducing demand is equivalent to producing energy for the energy market’s clearing mechanism. A DR aggregator will need to be capable of performing the following operations in order to safely integrate large distributed loads into the power grid and future US electricity markets:

• Analyze the flexibility and capabilities of the customers’ devices [110].
• Schedule demands ahead of time [111].
• Install communication infrastructure, necessary controls, and meters [112,113].
• Perform necessary predictions [114] (system load forecasting [115,116,117], electricity price forecasting [118,119], and wind and solar production forecasting [120,121,122]).
• Develop protocols and contracts for customer participation [71].
• Develop the software for managing all of the above.

The DR aggregator may be a retailer or have a close connection to a retailer [71,123]. Retailers perform load forecasting, assess fluctuations in customer demand, and provide new pricing schemes [124], services that are similar to the DR aggregator [125]. On the other hand, the DR aggregator communicates with system operators, such as the distribution system operator (DSO), if it exists, and the ISO. The DSO can provide access to the DR aggregator with customers’ demand data [126], and the ISO is the buyer of services from transmission and distribution resources [70,127].

Case studies have been performed before to evaluate the effectiveness of DR, including Table 5. In a University of Pennsylvania case study [128], the building’s “DR-Advisor” tool showed that with a data-driven DR tool, 17% better curtailment performance can be achieved compared to rule-based strategies (380 kW load curtailment saving over USD 45,000 in one event). In another study by the US Department of Energy [129], 13 different DR types were characterized across sectors according to the frequency and duration of response metrics, enabling costs, etc., and it was found that the cooling load types in residential and commercial prosumers had the least ability to take advantage of energy arbitrage opportunities across US states in the Western Interconnection, while data centers and water heating showed moderate capability, with municipal pumping experiencing the highest ability. CAISO in another voluntary DR study [130] defined a metric for evaluating the response time and ramping capability of DR across 200 K residential buildings and found the period of responses to vary within −18% and 3% from the event’s time, while the response varied with income and other demographics. Another study by OSTI, Department of Energy [131], showed that aggregate DR and battery optimization can reduce the load peak by 2.9% (or 26 MW) during the summer of 2017 in Austin, TX. Lastly, another Department of Energy study [132] framed DR as a co-equal flexibility resource alongside storage. The same study provided materials for future case studies as they emphasized the need for modeling innovation to accurately represent aggregated DR behavior via DR aggregators, while they called attention to regulatory and data hurdles that also plague DR participation and DR valuation. The same study mentions that inconsistent DR participation rules, aggregation limits, inadequate metering/communication, and unclear compensation mechanisms are key obstacles to DR valuation in general.

The traditional operation of DR aggregators provides them with access to information about each distributed load with a load mix, as explained in Section 2.2, which includes knowledge of the prosumer’s devices and models; the prosumer’s demand patterns, behaviors, and preferences (potentially PI data); the prosumer’s work and house occupancy schedule; individual device demand forecasting for devices without models; generation forecasting; operations of battery storage or electric vehicles; etc. All this information is not only overwhelming to the operations of the DR aggregator, which has to coordinate and optimally operate thousands or millions of prosumers, but also a danger to the customer’s privacy because the data collected from local sensors is often PI, as one can see in Table 3. All the aforementioned observations lead to research questions regarding privacy in future US electricity markets (Section 3.7), where privacy and cybersecurity will be an issue of central importance, in addition to the issues related to prosumer’s flexibility and uncertainties, the scalability of DR aggregator operations, and competition with the prosumers or amongst the prosumers. Section 2.5 will analyze prosumer privacy further.

2.5. Privacy Concerns for Distributed Load Integration in Future US Electricity Markets

The modernization of smart grids is occurring with the collection of data from demand meters, sensors, etc., despite the slow introduction of data privacy regulations in smart grids, and this modernization has raised physical safety, privacy, and cybersecurity concerns for the prosumer and other grid and market entities. The contribution of distributed resources to electricity generation decarbonization will not only be further enabled by smart grid digitization via smart meters (Section 2.3) and data exchange structures that increase the visibility of such resources to system operators but also raise customer privacy concerns [6].

The critical role of cybersecurity in the effective operation of the smart grid is documented in legislation and by the Department of Energy [133]. Section 1301 of the Energy Independence and Security Act in 2007 (P.L. 110–140) [134] states that “it is the policy of the United States to support the modernization of the Nation’s electricity transmission and distribution system to maintain a reliable and secure electricity infrastructure that can meet future demand growth and achieve each of the following, which together characterize a smart grid”:

• “Achieve increased use of digital information and control technology to improve the reliability, security, and efficiency of the electric grid”;
• “Achieve dynamic optimization of grid operations and resources, with cybersecurity”.

According to the US National Institute for Standards and Technology (NIST) [135,136], common adversaries to Information Systems, including the smart grid, are nation states; hackers, terrorists, and cyberterrorists; organized crime; industrial competitors; disgruntled employees; and careless or poorly trained employees. Due to the privacy and cybersecurity concerns raised by the Department of Energy and NIST, efforts toward preventing physical and cyberphysical attacks on smart grid entities, including the distributed load, and privacy and cybersecurity preservation are important. The privacy abuse caused by the DR aggregator’s unconstrained access to prosumer models, behaviors, and data calls for attention and a potential revision to the DR aggregator’s accessibility that is permitted by the market operator, and a potential revision of such privileges is also needed if alternative operations can be suggested for DR market participation that hide as much sensitive information and/or PI data of the prosumer as possible. The assumption behind this statement is that limited information and data access can minimize knowledge extraction from the phished data and its fusion with other customer data sources. An open question related to prosumer privacy is identified here and summarized in Section 3.7. The next section refers to the market bidding process of a DR aggregator, which is part of its market operations.

2.6. Market Bidding Process and Challenges for Demand–Response Aggregators

Market bidding takes place on an online platform offered by the ISO, and the bidding strategy is often a complicated mathematical tool designed by the DR aggregator. The DR aggregator can participate in the following:

• The emergency DR market.
• The economic DR market, which consists of the following:

  -

  The capacity market (pre-capacity bidding).

  -

  The energy market (DAM and spot market).

  -

  The ancillary service market (day-ahead scheduling reserve, synchronous reserve, flexibility reserve, etc.) [137].

Although DR aggregators can sign bilateral contracts with retailers, generators, and system operators, bidding in the energy market directly in order to maximize profit is the most common activity of DR aggregators [138,139]. Settlements are generally determined differently by each market type and the flexibility that market type may offer. In PJM, the DR aggregator is compensated for the reduced demand in full emergency and full pre-emergency markets [69]. The same applies to the emergency energy market, which, however, is voluntarily joined by the prosumer. The DR aggregator is faced with a number of uncertainties when it comes to market bidding, such as market price uncertainty, prosumer demand uncertainty, prosumer flexibility, and competition with other market players; these are major open research topics that, if answered, will enable more efficient DR aggregator operations in future US electricity markets (Section 3.7).

Section 3 will introduce the existing literature frameworks for DR aggregator operations and their interactions with the distributed load and marker, and this is accompanied by critical thinking on how DR aggregator operations will have to change in future US electricity markets in order to support the 2030 and 2050 goals, a topic that the existing literature falls short of covering.

3. Demand–Response Aggregator Frameworks in Future US Electricity Markets

3.1. Competition in the Prosumer–Aggregator Frameworks

An overview of aggregator or prosumer frameworks categorized by methods, objectives, and devices compatible with that framework are presented in

Table 6. Overview of DR aggregator frameworks for the DAM, IDM, and reserve markets (aFRR, FCR, and mFRR). The star ✓^★ denotes an imbalance type of cost for the aggregator. The optimization of either the aggregator’s or the prosumer’s objective indicates a significant lack of competition consideration when modeling aggregator-prosumer frameworks.

Market Type	Method	Objective: Min Agg./Pros. Cost	Devices	Prosumer Reward	Paper
DAM	Linear Optimization	✓/-	Appliances	-	[111]
	MILP	✓/-	BESS, appliances	Time-varying	[140]
	MILP	✓/-	Appliances	Optimal flat rate	[141]
	MILP	-/✓	BESS, appliances	TOU, CPP, RTP	[142]
	MILP	-/✓	Appliances	TOU, time-varying	[143]
	Bi-level Optimization	✓/-	EV	-	[144]
	Bi-level Stochastic Opt.	✓/-	EV	-	[144]
	Stochastic Robust Opt.	✓/-	EV	-	[145]
	Stochastic Robust Opt.	-/✓	Appliances	RTP	[146]
	Robust Optimization	✓/-	EV	-	[147]
	Robust Optimization	✓/-	BESS, thermal loads	-	[148]
	Optimization	✓/-	Electric heaters	Flat rate, extra	[149]
	Algorithm	-/✓	BESS, appliances	RTP	[150]
	Algorithm	✓/-	EV	-	[151]
DAM, IM	Linear Optimization	✓/-	EV	-	[152]
	Linear Optimization	-/✓	Storage heaters	RTP, fixed pay	[153]
	2-Stage Stochastic Opt.	✓/-	EV, TCL, appliances	-	[137]
	2-Stage Stochastic Opt.	✓/-	EV	Flat Rate	[154]
	2-Stage Stochastic Opt.	✓/-	EV	Yes	[155]
	2-Stage Stochastic Opt.	✓/-	BESS	Flat rate, extra	[156]
	Bi-level Stochastic Opt.	✓/-	EV	Optimal flat rate	[157]
	Stochastic Optimization	✓/-	Unspecified	Optimal time-varying	[158]
IDM	Linear Optimization	✓/-	TCL	-	[159]
	MPC	✓★/-	TCL	-	[160]
DAM, IDM	Probabilistic Opt.	✓/-	BESS, appliances	-	[161]
	MPC, Simulation	✓★/-	Space heaters	-	[162]
DAM, IDM, aFRR	2-Stage Stochastic Opt.	✓/-	EV	-	[163]
FCR	Simulation	✓/-	Heat pumps	-	[164]
mFRR	Heuristic	-/✓	TCL	-	[165]
aFRR	Agent-based	-/✓	EV	Yes	[166]
	Multi-Objective Opt.	✓/-	EV	-	[167]
DAM, aFRR	Optimization	✓/-	EV	-	[168,169]
	Optimization	✓/-	BESS	-	[170]
	Optimization	✓/-	EV	-	[171]
	Optimization	✓/-	EV	Degradation	[172]
	Optimization	✓/-	EV	Flat rate	[173]
	Quadratic Optimization	✓/-	EV	-	[174]
	2-Stage Stochastic Opt.	✓/-	EV	-	[175]
	2-Stage Stochastic Opt.	✓/-	EV, TCL	-	[176]
	Stochastic Optimization	✓/-	EV, BESS	-	[177]
	MPC	✓/-	EV, TCL	-	[178]
	MPC	✓/-	Office HVAC	-	[179]
DAM, aFRR, IM	2-Stage Stochastic Opt.	✓/-	EV	-	[180]
	2-Stage Stochastic Opt.	✓/-	EV, TCL, appliances	-	[181]
	2-Stage Stochastic Opt.	✓/-	EV	Optimal fixed	[182]
Congestion DAM	MILP	✓/-	Appliances	RTP, time-varying	[183]
Cong. DAM/DSO tariffs	Linear Optimization	✓/-	EV, heat pumps	Flat rate	[184]
	Linear Optimization	✓/-	EV or appliances	-	[185,186]

for the day-ahead and imbalance markets (IM) and for the intra-day (IDM), reserve, and congestion markets equivalently. These frameworks consider specific devices only and optimize either the aggregator’s or the prosumer’s costs, but not both simultaneously; therefore, the underlying competition between the prosumers and the aggregator is neglected (open research question about competition in Section 3.7). This gap around competition is one of the five evaluation criteria summarized in Section 3.7, and it can be overcome with game-theoretic methods, which are analyzed later in this section, where all players are optimized while device models may or may not be considered. In the second case, some energy management systems from the prosumer’s side are assumed.

The literature often models the demand side (prosumer) separately from the supply side (aggregator). For example, Ref. [187] focuses only on the prosumer side, ignoring the presence of an aggregator and the competition between them. These approaches lead to suboptimal solutions because the players (prosumers or aggregators) have a wrong or no perception of the underlying competition. Game theory can model existing market competition and, therefore, derive optimal strategies that maximize the payoffs of all selfish players of the energy trading game. In the case of the prosumer-aggregator, the aggregator firm has the freedom to set up its own competition model for its prosumers, as long as it is able to market it properly and convince the prosumers to register. The market operator (independent system operator, also known as ISO) requires metering infrastructure to be in place so that it is able to monitor actions that are unfair to the market, such as hiding true capacities, bidding significantly lower quantities than the nominal capacity, etc.

The most common types of competition with respect to homogeneous products in markets are Stackelberg, Cournot, and Bertrand. Stackelberg games have a leader–follower structure, where the leader firm (firm with the most market power) plays first with an advantage and the followers observe the leader’s action and respond appropriately. The optimal strategic response (highest payoff for all players) to the leader’s Stackelberg action is a Stackelberg-type response from the followers. Stackelberg describes competition based on quantity [188]. A key advantage of Stackelberg models is that they often yield closed-form strategies, especially in full-information settings, making them analytically tractable and attractive for mechanism designs. However, their main limitation is the strong assumption of perfect information.

In Cournot competition, players play simultaneously by competing for quantity (players with market power can affect the market price). Players can no longer observe others’ actions, and this leads to different equilibria compared to the Stackelberg. Note that the leader in a Stackelberg competition can achieve at least the same or higher payoff as in the Cournot [189], which means that if the leader firm is in charge of the mechanism’s design, it will choose a Stackelberg type of competition. Cournot equilibria can be more realistic than Stackelberg in electricity markets where aggregators or participants act independently and do not know the actions of others in advance. However, Cournot models often yield less efficient market outcomes and may be harder to solve analytically, especially with constraints.

Bertrand competition describes the competition between multiple firms with respect to price. An ideal example of Bertrand competition could be the competition between multiple aggregators for the same prosumer, who would be offered multiple prices simultaneously for the same commodity (homogeneous product). In this case, Bertrand competition would result in marginal costs for the prosumer, which is a significant advantage. However, many system operators currently prohibit prosumers from registering with multiple DR aggregators, a fact that makes this competition type inappropriate. An aggregator may choose how to design the prosumer-aggregator mechanism; therefore, a Stackelberg game would be preferred because it can provide closed-form strategies for game analysis in the full-information setting (see

Table 7. Aggregator applications formulated as Stackelberg games (also known as bi-level programs). Application areas include different aggregator or prosumer types, as well as different market types where such games are observed between aggregators and prosumers.

Description of Bi-Level Program	Service/Market	Approach	Paper
Large consumers, significant wind	Energy, reserves	Stochastic, MPEC	[190]
Electric vehicle aggregator	Flexible ramping	Robust optimization	[191]
DAM and RTM prosumer aggregator	Energy, reserve, flexible ramping	Robust, risk-averse	[192]
Batteries and flexible load	Energy, flexible ramping gen.	Co-optimization, MILP	[193]
Battery swap aggregator	Economic, peak shaving	RL, MILP	[194]
Large-scale electric vehicles	Frequency regulation	Piece-wise linear	[195]
Strategy evaluation of DER aggregator	Economic, wholesale and local	Stochastic, MPEC	[196]
Large-scale biogas and DR	Energy, DR	Non-linear, MPEC	[197]
Ground source heat pump aggregator	Peak shaving, valley filling, DR	Optimal control	[198]
Specialized load aggregator	Balancing, frequency, reserves	Custom algorithm	[199]
Combined heat and power	Energy, balance	Stochastic, big-M, MPEC	[200]
Mix load aggregator	Energy, flexibility, local	Robust, cutting planes	[201]
DSO and battery-generator aggregator	DR	$ϵ$-constraint, KKT	[202]
Residential consumer aggregator	DR	Stochastic, MPEC	[203]
Bidirectional solved uni-directionally	DR	Convex optimization	[204]
Battery-generator aggregator	DR	MINLP	[205]

for various aggregator applications formulated as Stackelberg/bi-level optimization programs). No estimation or learning would be required for this analysis. In most designs, no direct competition is assumed between aggregators, but rather through the market.

The existing literature considers either the aggregator’s or the prosumer’s cost optimization, but not both. Simultaneous cost optimization would require an underlying competition assumption and would have to be modeled with game theory (open research question about competition in Section 3.7). Such a problem would be very complex and potentially computationally challenging in a centralized manner, especially if device model constraints and energy/demand forecasts are considered for the distributed load (see distributed load mix in Section 2.2). However, few efforts in the game-theoretical direction appear in the literature and are analyzed next.

3.2. Deterministic Game Theoretical Prosumer–Aggregator Frameworks

To enjoy the benefits of market-based DR, market presence in modeling is substantial for DR aggregator frameworks. However, some frameworks like [206,207] consider a non-profitable utility/aggregator (lack of competition) without market access. Both papers assume the deterministic modeling of aggregators and prosumers, ignoring the effect of several uncertainties. At the same time, Ref. [206] proposes a framework with some level of flexibility since prosumer battery models are considered, while Ref. [207] proposes an auction-like process run by a utility with multiple aggregators splitting the rewards but without any consideration of prosumer flexibility. However, Ref. [207] is a good starting point for the derivation of closed-form game-theoretic strategies in DR aggregator frameworks. Prosumer privacy or framework scalability are out of the scope of these studies.

Modeled competition between the DR aggregator and the prosumer can be found in other works like [208], which considers market access but not uncertainty or any particular prosumer flexibility options (no HEMS/BEMS constraints). Prosumer privacy and framework scalability with the number of prosumers and constraints are out of the scope of this work. However, this work is another excellent example of Stackelberg bottom-up pure strategy extraction, with full mathematical proofs provided. Some other Stackelberg competition frameworks with selfish players but without uncertainty include [73,74,209], where a degree of flexibility is assumed through prosumer time-coupling constraints on the total desired daily traded energy. Moreover, in the same works, the authors have considered prosumer privacy by developing a decentralized solution that does not require sharing any HEMS/BEMS details (objective, constraints, devices, preferences, settings, PI data, etc.) with the DR aggregator. The same decentralized solution allows for scalability with the number of prosumers, while the strategies and computational complexities are rigorously derived.

Other game-theoretic competition frameworks, like [210], show a different degree of flexibility by adding budget constraints for the prosumers and considering multiple utilities (no market). Uncertainty is not part of this study; however, a degree of privacy is obtained based on the prosumers’ sharing of optimal decisions, and a degree of scalability is attained from the distributed nature of the algorithm, but for convex games only. Another degree of flexibility is that of bidirectional energy flow between DR aggregators and prosumers. As many frameworks [207,208,210,211] do not allow this, we identify that some other works, like [73,74,204,209], allow for energy to flow bidirectionally between the prosumer, the DR aggregator, and the market, resulting in either selling energy to the market or buying energy from the market, achieving new levels of monetary rewards, as explained in [74,209]. Please note that in the case of [204], a bidirectional energy formulation is offered, but the proposed solution only solves the unidirectional energy flow problem. Moreover, Ref. [204] does not consider any uncertainty, prosumer privacy, or scalability of the solution as part of the scope of their research. A final example of framework flexibility is the prosumer’s behavioral heterogeneity, which includes different objectives and constraints, including differences in prosumer sector (residential, commercial, or industrial) [212], or allowing for elasticity to be evaluated for each prosumer separately [74,209].

Although this deterministic game-theoretic framework review on Stackelberg games may not be fully exhaustive, the key takeaway is the evaluation criteria used in this framework and all subsequent sections, which are one of the main contributions of this paper. Apart from DR applications, Stackelberg games have also been used for energy storage aggregators [213], cybersecurity [214], and road network checkpoints around the LAX airport (ARMOR program) [215]. Other types of games, such as Vickrey and Dutch auctions, have also been studied for DR applications [216] but less frequently, while Nash games are often used to describe the competition between DR aggregators [217], which, as mentioned in Section 3, is not realistic for most market operators.

3.3. Uncertainty in Prosumer–Aggregator Game-Theoretic Frameworks

A DR aggregator and prosumer framework can face several uncertainties depending on the level of implementation details. Some of the most common ones are clearing market price uncertainty, prosumer total and per-device demand uncertainty, and prosumer generation uncertainty [218].

In order to manage uncertainty in modeling, robust, stochastic, or RL-based approaches are usually proposed. In this paper [219], a robust optimization framework for DR aggregators in the DAM and RT markets was proposed, which takes into consideration prosumer demand uncertainty by forecasting the customer’s baseline load. Their method includes a single optimization problem (no competition) where the DR aggregator makes all the decisions for the prosumers who have battery devices and shiftable and reducible loads, therefore allowing a degree of flexibility in the prosumers’ device scheduling; however, this compromises prosumer privacy since the DR aggregator schedules all prosumer devices centrally, while solution scalability is not addressed. Robust optimization methods for DR aggregators have also been proposed within a game-theoretic context for the DAM [220], modeling the competition between DR aggregators as a Nash game but ignoring the competition with the prosumers. The main uncertainty considered is the market-clearing price, and hence, uncertainty sets are constructed. Since there are no explicit prosumer optimization problems, prosumer flexibility is not applicable, while privacy is not guaranteed if the DR aggregator is aware of some prosumer demand forecast. Scalability is not addressed theoretically or experimentally. Distributionally robust Stackelberg games have been proposed for the competition between DSO and DR aggregators, with some level of prosumer load type detail [221] provided specifically for HVAC and solar panels, which are part of the DR aggregator’s model (no privacy). Uncertainties include renewable supply and outdoor temperatures that are distributions within this framework. Utilizing an MILP reformulation, the authors turn this into a tractable problem for a 33-bus system, but no larger simulations were performed to evaluate the scalability further. Lastly, another distributionally robust approach [222] proposed a robust Stackelberg game to model the competition between a microgrid operator (conceptually close to a DR aggregator) that includes battery systems. The microgrid operator competes with the prosumers (some level of flexibility due to load breakdown and EV battery), while the overall uncertainties are around market prices, uncontrollable load demand, and solar generation from each prosumer. The program generated is a MINLP, which is simplified after some linearizations and solved iteratively to protect prosumer privacy.

Some examples in Bayesian or stochastic games for DR aggregators are found in the literature. These approaches assume that Bayesian or stochastic (distribution) uncertainties are present, with solutions optimized on average instead of the worst-case scenario (robust approaches), and they can be used when expected cost savings or performance is more important than guarding against extremes (risk level can be adjusted with CVaR). The DR aggregator game under demand uncertainty has been approached in [223] using Bayesian Stackelberg game theory, modeling only the game between multiple DR aggregators and ignoring the game between the prosumer and the DR aggregator (no competition). It is assumed that the DR aggregator has full control over the prosumer’s load, with the customer’s probability of opting out modeled (no privacy). An example of a stochastic game for the DR aggregator for the interested reader can be found in the context of microgrids participating in DR with DSO [224], solved by introducing the lower level KKT constraints to the upper level program, an approach that can lead to issues with privacy and scalability.

The most important aspect of decision-making under uncertainty in the aggregator’s problem is adaptivity. In energy trading, the players have to be able to adapt to changes in prosumers’ demand (highly uncertain at the residential level [117]), price elasticity, dis-utility, players’ strategy (i.e., dishonesty), and changes in the market side that affect players’ payoffs, i.e., introduction of a new player, change in a player’s strategy, etc. Reinforcement learning (RL) is a popular adaptive method where past knowledge is used (exploitation of learning), in combination with the exploration of actions to further maximize the players’ payoffs, and this has not yet been implemented for the DR aggregator. However, DR approaches related to consumers and utilities have been proposed as bi-level non-cooperative Stackelberg–Nash games [225]. In another RL approach, Ref. [226] proposes a Markov decision process (MDP) for aggregators to learn better pricing strategies. This approach is based on Q-learning, assuming competition between aggregators, but it lacks a game theoretic approach and does not consider prosumer modeling (no prosumer flexibility). It has been stated that decision-making under uncertainty for DR aggregators, especially under partially observed environments, is still an open question [227]. Game-theoretic approaches under uncertainty for the DR aggregator problem are limited, as mentioned by the two literature review papers [227,228]. Additionally, the problem of incomplete information regarding the prosumer’s energy-selling or energy-purchasing patterns was not addressed properly [229] and remained a roadblock until [230] proposed an RL mechanism, in which an EMS (with a role similar to a DR aggregator) coordinates the energy exchange between the utility and prosumers, modeling them in a competitive way via uncertainty quantification and solving it in a combined manner. The scalability of the solution was proven to be satisfactory with the number of prosumers, while heterogeneous prosumer behavior was also modeled. In another work, an RL-based DR strategy is designed for the DR aggregator to learn the best charging decisions for prosumer batteries based on market reward outcomes (no privacy and minimum prosumer flexibility modeled), utilizing deep Q-networks. Scalability was not addressed, but it is dependent on the state and action’s dimensionality. Lastly, a Q-learning framework in [231] allows customers to engage in a cooperative game with the utility, which performs both DR and economic dispatch (ED), while Stackelberg equilibria are found with the help of a pre-trained neural network. The latter provides evidence that learning Stackelberg game equilibria with RL is possible but does not specifically consider the DR aggregator problem. RL can provide model-free frameworks, i.e., frameworks without a specific utility function, opening paths for adaptivity in the prosumer–DR aggregator games while playing close to the equilibrium.

Before any learning can be applied, adaptivity must be ensured for the algorithm. Online metrics such as regret minimization [232] have been used in playing two-player zero-sum Nash games [233] in an adaptive fashion, where the difference between the total payoff under uncertainty and the total best payoff is minimized in a horizon of N games. In [234], players can play non-zero-sum Stackelberg games with uncertainty in the follower’s action, where the leader is trading off the estimated follower’s action with the true value revealed.

In summary, Bayesian approaches require data to construct distributions for the uncertain variable, online learning does not require any a priori knowledge and learns on the fly, and online adaptivity combined with RL methods can trade-off exploration and exploitation compared to robust methods. Decision-making frameworks under uncertainty for prosumer DR aggregators are still an open question due to various challenges and very limited market data access. They are high-dimensional control problems faced with the challenges of partial observability and randomness (open research question about uncertainty in Section 3.7).

3.4. Prosumer Privacy in the Prosumer–Aggregator Game

When it comes to the prosumer DR aggregator game frameworks available in the literature, not much has been carried out to provide such levels of privacy to prosumers. Specifically, in [235], prosumers only share local optimal solutions and their power flexibility in fulfilling the constraints of the proposed global optimization program for the ancillary service market. In other approaches of the day-ahead energy market, like [73,74,209], only optimal solutions from the prosumers are shared with the DR aggregator, which prevents the sharing of objectives, constraints (device models from HEMS/BEMS), preferences and settings, forecasts of demand or generation (including limits), etc., collected from sensors [117] or forecasted with data science methods [236] in accordance with the DR aggregator, protecting prosumer privacy. Another work such as Ref. [237] provides a two-stage framework for day-ahead and real-time market bidding, where only the prosumers’ schedules are shared with the aggregator, and the problem is solved distributively with the alternating direction method of multipliers (ADMM), while they exchange both their optimal solutions and their power flexibility. ADMM methods can be slow in convergence and require strong assumptions for primal variables to converge to an optimal solution, while the decomposability of the objective is necessary [238]. Since prosumer privacy has not been broadly studied with DR aggregator frameworks, privacy will become one of the open research questions in Section 3.7, and these frameworks are also necessary for future US electricity markets. Last but not least, the authors in [239] proposed a distributed, prosumer privacy-preserving algorithm to solve an MILP for a manual frequency restoration reserve (mFRR) service, allowing the aggregators and prosumers to participate in the balancing market. Please note that although prosumer blockchain privacy frameworks exist, these community coordination schemes follow a different logic [240,241,242].

3.5. Scalability in Demand–Response Aggregators’ Operations

Decentralized solutions for the prosumer–aggregator Stackelberg game are limited in the literature. In [73,74,209], a decentralized algorithm was designed for the DAM DR problem to solve the bi-level program, sharing only optimal solutions between prosumers and the DR aggregator. Other decentralized solutions were mentioned in Section 3.4 from a few papers [237,238,239].

The rest of this section will focus on decentralized solutions for general bi-level optimization programs (Stackelberg games) to facilitate the future decentralization of DR aggregator operations in the case of thousands/millions of prosumers with hundreds/thousands of constraints each. A proper game-theoretic framework setup and the proper utilization of one of the decentralized algorithms listed below may additionally allow for a solution to the prosumer privacy problem (constraints and other problem details remain hidden). The specifics of such research remain open questions, and if addressed, they may facilitate the large-scale deployment of DR aggregators in future US electricity markets, contributing significantly to the decarbonization and net-zero emission goals analyzed in Section 1.

In the decentralized algorithm proposed in [243], the lower-level program is reformulated into an unconstrained one with the penalty method, and then, the transformed bi-level program is solved iteratively by applying the penalty function method to the lower-level problems. It was also proven that a sequence of approximated solutions converges to the optimal solution, but privacy concerns for the lower-level programs are not taken into consideration in the solution method, nor is it mentioned whether the same convergence result applies to bi-level programs with multiple followers. In [244], the authors proposed a decentralized bi-level program algorithm that is solved via the collaboration of the lower-level programs. Specifically, all lower-level variables are updated simultaneously based on a gradient descent algorithm, exchanging information about their current variable updates and their objective derivative values at the current variable value (not a completely private algorithm), while the upper-level variable is updated based on lower-level program’s variable update information. After some iterations of lower-level program variable updates, the upper-level program’s variable value is updated based on the current lower-level variable values. The drawback of this method is that the number of iterations is pre-defined and the same for all lower-level programs, and the method may converge to a local optimum. Last but not least, in paper [245], the authors proposed a similar approach to this dissertation, which is a bi-level evolutionary algorithm, but they restricted the solution’s mapping to quadratic polynomial functions. The authors do not study the effect of the approximation and learning errors on the solution’s quality, but they do provide a very insightful visualization of a generic solution map. Moreover, the authors propose a way to find the best solution map approximation model by choosing the model that minimizes the empirical risk error on a given dataset.

To facilitate the large-scale deployment of DR aggregators in future US electricity markets and, hence, the DR benefits, a scalable and private algorithm that is able to solve deterministic or uncertain game-theoretic frameworks that take into consideration prosumer–aggregator competition is necessary. In this way, DR benefits can be extracted on a large scale in future US markets, the probability of electricity generation decarbonization is increased, and a net-zero-emission US economy can be realized.

3.6. Prosumer–Aggregator Market Bidding Frameworks

The problem of market bidding, often formulated as a separate game, considers the market-clearing mechanism with only the total market capacity cleared and the market-clearing prices as the available information. In electricity markets, firms (power producers, aggregators, etc.) compete for both quantity and price, and they all play simultaneously. However, bidding strategies are usually simplified. Examples include auction-like mechanisms [207], piece-wise, constant-quantity price-bidding mechanisms [213,246], a finite pool of past (quantity and price) successful actions, and sampling bids from a nominal bidding curve and running scheduling algorithms [247], among others [246]. Market bidding strategies have also been developed using RL [248,249,250]. In the other literature, players decide to form tacit collusion by withholding capacity [251] in order to minimize a response from another firm (avoid the opportunity to price cut an opponent) without explicitly stating so. These are modeled as auction games with repeated play where the firm that sets the initial price is the leading firm. In [252], RL is used, and the market mechanism is modeled as a double auction between energy sellers and consumers. In [252], microgrids with private information compete in the energy market using RL. Stochastic optimization approaches, such as MILP [123,139,253] and scenario-based stochastic programming [254,255], have been proposed before, in addition to robust optimization [256,257], theoretical information gap decisions [258,259], and genetic algorithms [260,261] according to [69].

3.7. Open Research Questions for Future Demand–Response Aggregators’ Operations

This paper introduced the environmental goals for 2030 and 2050 in the US and analyzed the role that DR aggregators will play in future US electricity markets, for which their mechanisms are expected to be revised by policymakers in order to better compensate clean technologies and grid reliability support providers. To facilitate the large-scale deployment of DR aggregators and registered distributed loads (prosumers) in future US electricity markets, DR aggregator operation frameworks will need to have the characteristics shown in Figure 4, the achievement of which remains an open research question, as observed and analyzed in Section 2 and Section 3. In short, we refer to this as the FCUPS evaluation scheme for any DR aggregator and prosumer framework, and we next justify how these operational changes in future US electricity markets will facilitate the 2030 and 2050 goals (Section 1):

• Flexibility (F) with the type of prosumers (residential, commercial, and industrial), their behavior (strategy, rationality, bidirectional flow of energy and information, etc.), their device constraints, their distributed load types, and their operational capabilities through their HEMS/BEMS: Ideally, a flexible DR aggregator prosumer framework would model, in detail, the prosumer’s distributed load mix (Figure 3), data pipelines, settings, and controls via an HEMS/BEMS to ensure the feasibility and optimality of the DR aggregator prosumer solution, utilizing technologies introduced in Section 2.3. Flexibility is important to the 2030 and 2050 goals in order to operate a grid with significant renewable resources, according to Section 1’s points 1 and 6.
• Competition (C) between the DR aggregator and prosumers, as well as between prosumers, to incentivize prosumer engagement: Competition of the DR aggregator with other market players to maximize profits. Ideally, the DR aggregator and prosumer framework should model the underlying competition type, considering the types of competition analyzed in Section 3.1. Competition consideration is important in order to ensure that the prosumer’s interests are also taken into consideration, considering prosumers as active market players with preferences and rewards, unlike numerous previous approaches (Table 6). In this way, the distributed load is incentivized to participate in market-based DR, which can facilitate the 2030 and 2050 goals according to Section 1’s points 1 and 2.
• Uncertainty (U) quantification in multiple areas, such as the demand and generation profiles of the prosumers and the market and market prices for the day-ahead and spot markets to reduce unexpected costs: Ideally, the DR aggregator and prosumer game would be modeled as a Bayesian, stochastic, or RL game to allow mixed strategies and uncertainty quantification to be included in decision-making while still considering competition modeling in the framework. Uncertainty consideration is important for better scheduling in electricity markets and helps prevent supply shortages, large demand and price variations, or unexpected costs inflicted on the DR aggregator and prosumer, resulting in more reliable grids and markets, as well as financial prosperity and risk management for market participants. The aforementioned facilitate the 2030 and 2050 goals according to Section 1’s points 3 and 9.
• Privacy (P) preservation of the prosumer with respect to any for-profit market entities, such as the DR aggregator, hardware manufacturer, software developers, etc.: Privacy preservation includes PI data, prosumer preferences for comfort, device models and the mode/frequency of operation, load types and operational choices, demand and generation forecasts, etc. Ideally, the prosumer’s strategy, PI data, settings, preferences, controls, objectives, and structural details of the HEMS/BEMS would not be shared with the DR aggregator, a goal that aligns with the Department of Energy’s Energy Independence and Security Act (Section 2.5). Decentralized solutions that do not require the DR aggregator to solve the prosumer’s problem, but rather only receive the optimal solution, are preferred for this purpose. Such approaches facilitate the 2030 and 2050 goals according to Section 1’s points 6 and 9.
• Scalability (S) with the number of prosumers, as well as with the number and type of prosumer constraints, despite the complex nature of the distributed load mix: Ideally, every prosumer would be represented by a detailed HEMS/BEMS system and be given controls to facilitate flexibility, but at the cost of computational complexity. Methods like the solution decentralization of DR aggregator and prosumer operations (i.e, avoid solving a large, single-level optimization problem of the DR aggregator and the prosumers’ KKT conditions), parallelization, and approximate algorithms can be introduced to manage these computational challenges. Scalability is important to the 2030 and 2050 goals because the large-scale integration of flexible generation and demand to the grid via DR can defer system expansion costs and alleviate the system’s stress according to Section 1’s points 6 and 8.

Figure 4. This diagram shows the key players in DR markets and indicates potential interactions between them with arrows. The different necessary frameworks are displayed on the left, and they are enabled by the discussed technologies in this paper. Finally, the operational challenges are depicted in blue, which pose the open research questions examined in this paper, and they are important for the 2030 electricity generation decarbonization and 2050 net-zero-economy goals.

Figure 4. This diagram shows the key players in DR markets and indicates potential interactions between them with arrows. The different necessary frameworks are displayed on the left, and they are enabled by the discussed technologies in this paper. Finally, the operational challenges are depicted in blue, which pose the open research questions examined in this paper, and they are important for the 2030 electricity generation decarbonization and 2050 net-zero-economy goals.

To allow for the aforementioned characteristics to co-exist in the same prosumer–aggregator framework, researchers and policymakers are encouraged to consider the literature gaps addressed in Section 3 and to find ways to accelerate the distributed load participation in the DR markets (data sharing policies, information provided by ISO/DSO to DR aggregators, carbon pricing, etc.). In future research, the aforementioned characteristics can be addressed by considering flexible constraints (inequalities instead of equalities, a range of possible demand scenarios, etc.); hierarchical or decentralized algorithms that solve the framework (including blockchains); game-theoretic frameworks that address prosumer–aggregator competition according to the mechanism designed by the DR aggregator; RL and multi-agent RL, MPC, and Bayesian games; robust, distributionally robust optimization, or stochastic optimization frameworks that can address uncertainty; and finally, privacy preservation operations that hide prosumer-specific information and PI data from the DR aggregator while still allowing all the above characteristics to exist (i.e., decentralized and private learning, also known as federated learning, differential privacy, homomorphic encryption, and secure multiparty computation). The challenging task of integrating all of the above characteristics in the same framework and algorithm will allow for the large-scale deployment and flexible operation of DR aggregators and distributed loads in future US electricity markets while preserving customer privacy, a concern of increasing amplitude to the Department of Energy.

Following the gap in prosumer DR aggregator game-theoretic frameworks, it is not only necessary that prosumers are able to implement their total demand schedule bid on their devices, but they should also be able ensure that demand uncertainties can be corrected in real time to avoid market fines. Energy-saving solutions at the distributed load level include sensor-based automation for smart device control, as well as energy management systems that optimize the building’s operation based on price signals by a utility or aggregator. Therefore, the direction of integrating the prosumer–aggregator game with the prosumer’s energy management system in the same algorithmic solution and software can facilitate flexibility, scalability, competition, uncertainty, and privacy. In other words, during the negotiation with the DR aggregator, each prosumer can run their day-ahead HEMS/BEMS and provide not only feasible but also optimal demand profile responses to the DR aggregator, which will minimize costly demand discrepancies. Day-ahead and spot market bidding operations can be combined in the same solution so that the cost from real-time demand discrepancies is minimized in the settlement process (multi-horizon frameworks). Such integration of algorithmic solutions and software could facilitate the large-scale deployment of DR aggregators and prosumers in future US electricity markets and help achieve environmental and grid reliability goals.

To enable the integration of energy management systems (HEMS/BEMS) and aggregator–prosumer market bidding games, aggregator optimization programs with prosumer device constraints would be transformed into bi-level programs with lower-level prosumer objectives and device constraints. Such a large-scale bi-level program would be very hard to solve for thousands of prosumers, and it would traditionally require all prosumer information to be available to the aggregator. However, with decentralized, scalable, flexible, uncertainty-quantifying, and privacy-preserving algorithms, such obstacles can be overcome. Some examples satisfying these criteria are federated multi-agent learning with MPC and game theory, or hierarchical distributionally robust optimization with privacy guarantees (coordination at higher levels using differentially private updates) and blockchain-enabled transactive energy markets. Lastly, explainable AI can be utilized to better interpret the main drivers behind demand, price forecasting, etc., with methods like SHAP, LIME, saliency maps, etc., most of which are appropriate for privacy-sensitive applications as well.

4. Discussion

This study highlights the critical role that existing technologies and emerging frameworks can play in achieving the 2030 and 2050 decarbonization and grid reliability goals of the US energy sector. Energy-efficient technologies, such as high-performance HVAC systems, smart lighting, building insulation, advanced sensors, and smart energy management systems (HEMS/BEMS), have already demonstrated measurable reductions in energy consumption and CO₂ emissions. For example, structural upgrades and sensor-driven HVAC systems can reduce building energy use by up to 55%, while smart controls and energy management can achieve an additional 23% in savings. These technologies provide a robust foundation for immediate and scalable deployment. However, while these technologies contribute significantly toward CO₂ reduction targets, their integration into large-scale distributed energy systems introduces operational challenges that current methods do not fully address. Existing centralized demand–response (DR) aggregator models often require extensive access to prosumer data, including personally identifiable information (PI), device-level demand profiles, and behavioral patterns. This access creates substantial privacy risks and scalability bottlenecks, particularly as the number of participating prosumers grows into the thousands or millions. Moreover, current DR frameworks frequently fall short in managing uncertainty, ensuring competitive fairness between prosumers and aggregators and enabling flexible market participation that aligns with individual prosumer constraints and preferences when evaluated through the proposed FCUPS criteria—flexibility, competition, uncertainty quantification, privacy preservation, and scalability. Centralized models also struggle to efficiently coordinate the diverse and dynamic load types that characterize modern prosumers: from residential solar and battery systems to smart appliances and electric vehicles. To address these gaps, this paper proposes future research directions centered on decentralized, privacy-preserving, and scalable solutions. Specifically, federated multi-agent learning, hierarchical distributionally robust optimization with privacy guarantees, and blockchain-enabled transactive energy markets are identified as promising avenues. These approaches can enable prosumers to retain local control over their data while contributing optimal demand bids to the market. Furthermore, integrating game-theoretic models that rigorously capture competition and cooperation dynamics, along with reinforcement learning for adaptive bidding under uncertainty, could greatly enhance market resilience and efficiency. By focusing on these advanced methods, future DR aggregator frameworks can overcome the limitations of current approaches, facilitating the large-scale adoption of distributed load participation and strengthening the grid’s ability to meet 2030 and 2050 targets for decarbonization, reliability, and a net-zero economy.

5. Conclusions

This review has provided novel insights into the critical role that demand–response (DR) aggregators and distributed load will play in the transformation of future US electricity markets. Unlike prior reviews, this work introduces the FCUPS evaluation framework—highlighting the necessary characteristics of flexibility, competition, uncertainty quantification, privacy preservation, and scalability—as a comprehensive lens through which DR aggregator operations must evolve. The FCUPS framework offers a unified, structured approach for identifying gaps in current DR aggregator and prosumer frameworks, which had not previously been addressed with such holistic rigor. By systematically analyzing operational challenges across flexibility, privacy, uncertainty, scalability, and competitive mechanisms, this paper opens clear research directions that can shape future agendas. These directions include the development of decentralized, privacy-preserving frameworks, the incorporation of advanced game-theoretic and machine learning models to capture competition and uncertainty, and scalable solutions capable of managing millions of prosumers with complex, heterogeneous load mixes. Furthermore, this review stresses the need for algorithms that balance operational efficiency with privacy and cybersecurity, addressing growing concerns from both consumers and regulatory agencies. The proposed open research questions are positioned to guide interdisciplinary collaborations between energy researchers, data scientists, policymakers, and system operators. By addressing these questions, future work can directly support the 2030 decarbonization and 2050 net-zero goals, helping to redesign electricity markets that not only reduce CO₂ emissions but also foster resilient, equitable, and efficient grid operations.