The Next-Generation U . S . Retail Electricity Market with Customers and Prosumers — A Bibliographical Survey

Due to the rapidly-changing technologies in the power industry, many new references addressing the frameworks and business models of the next-generation retail electricity market are entering the research community. In particular, considering new customers with considerable demand response awareness and so-called prosumers with localized power generation based on distributed energy resources (DERs), the next-generation retail electricity market infrastructure will be a level playing field for local energy transactions, strategic pricing scheme design, new business model design and building an innovative energy ecosystem. Consequently, there is an urgent need to keep track of international experiences and activities taking place in the field of the market mechanism design problem at the distribution level. This paper provides a comprehensive survey of recent technology developments and aims to inspire awareness of the further deregulation of the electricity market, especially in areas close to customers. We mainly bring attention to the more than 90 articles published during the past five years. The collected literature has been divided into different sections to discuss different aspects of the next-generation retail electricity market under the deregulated power industry.


Introduction
Although electricity market deregulation has been underway since the United Kingdom opened a power pool in April 1990 [1], competitive forces in the U.S. electricity market have been largely silent since the early-2000s California electricity crisis.Then, since the 2010s, many power sector reforms and new market mechanism designs have been under intense discussion again due to the emerging smart grid technologies plus some innovative information technology (IT) business models and an Internet-inspired commercial paradigm [2].However, most research on the electricity market still focuses on the wholesale market, particularly the bidding process and financial transmission rights [3,4].The development of the retail electricity market seldom borrows much experience from such bulk power transactions, though.Instead, it prefers to follow principles, like multi-options, peer-to-peer, sharing economy friendliness, negotiability, and so on, that are utilized successfully in the customer-centric IT industry.This characteristic is also the reason for popular proposals such as the energy Internet [5] and digital grid [6] in many references.
Around the world, many countries are also pushing the reform of the electricity power sector very positively.Chile pioneered in the 1980s the deregulation of the electric power industry [7].The European Union had taken steps to liberalize its electricity industries in the 1980s and, late in the 2000s, allowed all customers to choose their electricity suppliers [8].The restructuring and regulatory reforms in the PR China's power sector happened in the 2000s along with other Asian countries [9].The electricity retailing in Japan was fully deregulated with fierce competition in April 2016 [10].In today's U.S. retail electricity market, 14 states have already adequate retail competition with Texas, Illinois and Ohio having 100%, 60% and 50% of their residential customers receiving service from electricity suppliers [11].However, many customers still have very limited "energy choice" or direct participation in the existing retail electricity market.
The key to open innovation in the power sector has been believed to be the development of consumer-centric business models and well-designed demand side management (DSM) programs [12,13].Following these ideas, the recent work in [14] looks even further forward to more subtle modeling of customer behavior, with considerations of their willingness to participate and even emotional or irrational features.With these prevailing ideas in the research community, the next-generation retail electricity market infrastructure will be a level playing field, where all energy end-users and customers have equal opportunity to play the role of active participants rather than pure passive price-takers [15,16].Fortunately, the recent development of the functionalities of the energy service companies (EsCos) and the distribution system operator (DSO) has opened many new possibility for monitoring, coordinating and controlling short-term or real-time delivery of electricity at the distribution level [17].Especially with the further development of the concept of the DSO, deregulation of the electricity market has been spreading out from wholesale market design into retail market design, as shown in Figures 1 and 2. In the new paradigm for energy transactions, different customers or customer groups (e.g., energy communities) are free to choose their service provider, either a distribution company or utility company, including even pure energy retailers, periodically.

GenCo
GenCo  Moreover, in smart grids, more and more customers will be able to have local generation capability, i.e., distributed energy resources (DERs), along with various flexible controllable loads, such as thermostatically-controlled loads (TCLs), distributed energy storage devices (DESDs) and washing machines [18,19].Electric vehicles (EVs) and plug-in electric vehicles (PEVs) are also appealing as the most controllable loads because they can be curtailed for significant periods of time (e.g., several hours) without impact on end-use function [20,21].These kinds of customers are encouraged to actively participate in the retail market to provide demand response or localized power balance between energy surplus and energy deficit.Some existing survey papers focus mainly on the decision-making process of retailers in the wholesale market and somehow ignore the significant effect that various types of future energy end-users will have on the whole electricity market landscape [16].The entire energy business ecosystem will be re-formed if the most recent research trends and principles, such as transactive energy [22,23], transactive control [24] (Transactive energy and transactive control are explained further in Section 2.4), an energy sharing economy [25], and so on, are adopted.

Retail Electricity Market with Pure Consumers
In most scenarios, customers play a passive role as price-takers in retail electricity, purely serving as consumers of energy at different locations.Those who have the capability to generate power locally with the help of microgrids and are able to supply electricity to other customers are called prosumers at the distribution level.We will leave the discussion of the retail electricity market that includes prosumers for the next section.

DSO with Distribution Level Pricing
As a result of the distribution grid's increasing number of roles and functionalities, the deployment of a DSO is becoming a necessity to ensure efficient and reliable delivery of electricity to emerging proactive customers.Customers now have more willingness to control their energy use and transactions with the utility grid, as their energy preferences have evolved.In parallel, there is a potential need for an intermediate entity between the regional transmission operators (RTOs) or independent system operators (ISOs) and energy end-users due to the limited visibility and control over the meter resources (e.g., advanced metering infrastructure) at the customer side [26].A DSO in the future energy system and energy market design may be considered the evolution of a distribution management system, with, however, more functionality at different layers (Figure 3).In addition to the traditional mission to operate, maintain and develop an efficient electricity distribution system, the DSO possesses more functionality rather than only mimicking the ISO's pure responsibility of electricity pricing and independent market-clearance at the transmission level as a non-profit entity.In the wholesale market, many ISOs nationwide implement the locational marginal pricing (LMP) strategy either in the form of ex ante LMP, for example New York ISO (NYISO), or ex post LMP, for example ISO-New England (ISO-NE), PJM and Midcontinent-ISO (MISO) [27].Based on the fact that LMP has been widely adopted to compute electricity prices in the wholesale electricity market [28], some scholars have begun to downscale the LMP schema for distribution networks by proposing its counterpart, distribution locational marginal pricing (DLMP) [29], which can directly work for individual energy end-users without referring to a load serving entity (LSE) or other demand bidding aggregators.It has been applied to several scenarios, such as the congestion management problem and the electric vehicle charging problem [30].
However, as shown in Figure 3, the DSO may not only play the role of an ISO at the distribution level since there is a huge difference between a distribution network and a transmission network, such as three-phase imbalance, radial system topology, high ratio of power loss, numerous low-voltage buses, and so on.To some extent, DLMP is hardly effectively obtained through running alternating current optimal power flow (ACOPF) for a distribution system.A very recent three-phase ACOPF-based approach has been developed to define and calculate DLMP accurately [31].

Decision Making of Retailers
Retailers in the electricity market are supposed to purchase electricity in the wholesale market and sell electricity to their subscribed end-user customers through assigning appropriate tariffs, either in a temporal variance way or at a flat rate.Currently, the electricity retail company is usually operated as an entity that is independent of any generation or distribution company [16].The decision-making process involved in buying and selling strategies usually contains some volatile market risks that are similar to the ones in any other market, such as the stock market and oil market.Especially with the further deregulation of the electricity market, along with the development of DSM and the proliferation of DERs, retailers participating in both the wholesale market and the retail market should carefully design their buying-selling trade-off and electricity portfolio optimization [32].In the future, many innovative pricing schemes will be necessary, taking into account emerging factors such as the increasing penetration of renewable energy, wide deployment of storage devices, adoption of advanced information and communication technologies (ICT) and rising customer awareness of switching among electricity suppliers.These new challenges also require retailers to incorporate some typical operations into their decision-making processes, which include retail energy forecasting, portfolio evaluation and risk management.
Due to the page limit and many mature approaches that already exist for residential load forecasting [33,34] and portfolio evaluation [35,36], risk management will be the focus of discussion here, along with many recent advances in the research community.In a typical example such as [37], the author utilizes stochastic programming techniques to determine the day-ahead market bidding strategies for retailers with flexible demands to maximize their short-term profit, specifically including a case study based on Sweden's electricity market and consideration of the demand uncertainty of retail customers.In most studies of retail electricity market operation, with risk from either real-time price or demand uncertainty, conditional value-at-risk (CVaR) is widely used to consider risk management [38].CVaR is a risk assessment technique often used to reduce the probability that a portfolio will incur large losses, which is performed by taking a weighted average between the value at risk and losses exceeding the value at risk [39].

Price Scheme and Demand Response
On the customer side, energy end-users do respond to the long-term electricity contract and price schemes offered by the utilities; however, they are usually insensitive and uncomfortable with respect to the highly dynamic or real-time pricing, due to the lack of competence to immediately respond to the price signal or little awareness of instantaneous opportunity [40,41].However, electricity prices that describe marginal costs can vary substantially over time.Fixed rates may ignore continuous changes in the electricity system conditions.Setting prices that differ for certain periods is an approach to approximating the real-time price.If rates are set much in advance and fixed over periods of time, they miss the majority of the potential gain as measured by the variance index [42].Both time-of-use (TOU) and critical-peak pricing (CPP) play crucial roles in providing load flexibility and tariff design in the retail electricity market [43].Based on the similar idea of rationalizing energy consumption behavior for the whole system cost, a prediction-of-use (POU) tariff is proposed and believed to better reflect the predicted cost for a customer [44].The possible combination of POU with the more widely-known TOU tariffs is also considered, which allows customers to fully benefit from meaningfully managing their consumption, as well as from their contribution to the system's delivering energy-efficient solutions.
Using TOU, CPP or other price schemes as baselines, some additional incentive mechanisms are also proposed on top of them to reflect the demand response from customers with energy awareness, which are aware of the electricity price elasticity and reasonable energy saving.Energy tokens, coupons and eVouchers, similar to their literal meaning in daily commercial activities, are proposed in [17,45,46] to encourage voluntary energy demand adjustment based on the negotiation principle.These kinds of negotiation-based demand response programs can be categorized as incentive mechanisms [47] that provide an additional economic management tool for the power system and market efficiency.

Transactive Energy and Transactive Control
In order to combine power systems tightly with economic or market-based operation, the term "transactive energy" has begun to be used to refer to techniques for managing the generation, consumption or flow of electric power within an electric power system through the use of economic or market-based constructs, while considering grid reliability constraints [48].In fact, transactive energy (TE), one of many promising solutions to electrical grid restructuring issues, has gradually become a more and more concrete concept among many discussions [49].Some experts give it the official definition of "a set of economic and control mechanisms that allows the dynamic balance of supply and demand across the entire electrical infrastructure using value as a key operational parameter" [48].Specifically, transactive energy mainly focuses on the value or economic operation of a modernized electrical grid, primarily from an economic perspective.It emphasizes the innovative business models and new consumption patterns in electric markets, along with taking some social impact into consideration.Some researchers [50] also believe TE is a potential framework to close the gap between wholesale and retail markets.Most DERs and demand resources can be aggregated as virtual power plants (VPPs) to provide bulk power adjustment capacity in different markets.
In practice, analogous to the price reaction approach, a concept named transactive control [24] allows the operation of flexible devices to be optimized economically by a local intelligent controller (or agent) under the control of the end-user and follows the principles of TE.In a way, a society of intelligent devices is formed to allow for market bids to be sent by a particular group of devices (e.g., hot water buffer, dishwasher, air conditioner, etc.) [23].The local-level bidding process, or laminar market architecture, is extremely suitable for thermostatically-controlled loads [51].Furthermore, both transactive energy and transactive control concepts are becoming more and more widely adopted by many recent pilot projects [23,49].

Retail Electricity Market with Prosumers
In this section, the retail electricity market with prosumer participation in the local power supply will be discussed.Prosumers are defined as agents that both consume and produce energy [52].With the growth in the number of small-and medium-sized energy entities using solar photovoltaic panels, small wind turbines, vehicle-to-grid EV/PEVs, home storage energy systems, smart meters and other smart devices, prosuming offers the potential for consumers and smart device owners to re-evaluate the energy practices in their daily lives.As the number of prosumers increases, the retail electricity market of today is likely to undergo significant changes over the coming decades, not only offering possibilities for localized inter-network energy trading and balance, but also introducing many challenges and risks that need to be identified and managed.To develop strategies for the future, policymakers and planners need knowledge of how prosumers can be integrated effectively and efficiently into a competitive retail electricity market.Some promising potential market mechanisms, such as prosumer grid integration, peer-to-peer models, indirect customer-to-customer trading and prosumer community groups, along with their implementation approaches, are identified and discussed below.

Prosumer Grid Integration
Most prosumer integration problems can be incorporated by extending the conventional optimization model to solve the pure energy consumption and energy management problems.However, some characteristics of two-way power flow need to be carefully considered for various types of challenges and optimization constraints, such as inverse current fault detection, distribution topology estimation, power surplus balance, and so on.When leveraged by an energy storage system (ESS), including vehicle-to-grid (V2G) technology, distribution network operation with a high penetration of prosumers needs to make sure that prosumers' benefits are aligned with the regulator/DSO's concerns, thus satisfying the requirements of both sides.The authors in [53] propose a market-based control to solve this issue.The complexity in the environment and in the interactions among players prompts techniques derived from complex systems theory.The work in [54] analyzes the optimal planning and operation of aggregated DERs with participation in the electricity market.In most cases, the aggregator of a large amount of DERs can operate as a virtual power plant (VPP) [55], which is connected as part of the main grid and participates directly in the wholesale electricity market.Many similar ideas based on aggregation frameworks have been frequently employed in solving the prosumer grid integration problem.However, from an energy utilization and market efficiency point of view, localized integration at low voltage levels with direct delivery to end-users is still highly encouraged.More and more decentralized decision-making frameworks without the necessity of aggregation are welcome nowadays.
The integration of various DERs and EVs also provides a new chance for building an innovative business model and a new energy ecosystem.There is a plethora of research and development areas related to prosumer grid integration that can be exploited for new business opportunities, thus spawning another branch of the so-called "green economy" focused on turning smart energy usage into a profitable business [56].

Inter-Network Trading with Peer-To-Peer Models
The encouragement of localized energy trading within a distribution network at low voltage levels promotes an eBay-like market platform and peer-to-peer models.Additionally, a high penetration of distributed energy resources raises operational and market challenges such that existing incentives and tariff support cannot be sustained with penetration growth at the microgrid level.As a result, some competitive market mechanisms or peer-to-peer models are required at the local distribution level.In [57], a matching mechanism is proposed to allow individual generators and load units to meet to conduct a bilateral trade.Each unit interested in maximizing its benefit adopts its own bid strategy.Trade between a randomly matched generation and load unit is established if their bids are compatible, which does not require the units to share their private cost or value information.
Sometimes, this type of peer-to-peer energy sharing is described as "Energy AirBnB" for future electricity retail markets.Furthermore, some peer coalition might be allowed in the electricity retail market and work as a new business model for a very short term.Some possible strategic coalitions among independent electricity retailers or prosumers may happen under the designed distributed framework [58] to maximize profits, which implies that electricity retailers may solely compete with each other, while some of them may cooperate with others to form a coalition in the economic operation of future electricity retail markets.In [59], a scalable and modular system is proposed and demonstrated for energy trading between prosumers.Even a novel decentralized digital currency, named after NRGcoins, is proposed by the same group of researchers to encourage prosumers to locally trade their excess energy while payments are carried out using NRGcoins [60].
The driving force of such a peer-to-peer mode becoming welcome in the retail electricity market is mainly due to two facts.The first one is frequently discussed: that the rapid adoption of DERs enhances people's willingness to trade in a decentralized way.The centralized operation will put too much burden on the central controller when all the individual customers send the trading requests at the same time.The other fact seems not so explicit: that the rise of Internet-connected devices (e.g., Internet of Things) has led to a wide energy connection, which is also strengthened by the disappearance of the conceptual gap between energy as a physical supply service and energy as an information service.The behavior of trading energy among peers more or less carries some meaning of social interactions.It is also another important source for the proposed idea: the energy Internet.

Indirect Customer-To-Customer Trading
Although peer-to-peer models are very attractive for a highly decentralized energy supply, some customers or prosumers can find it difficult and time-consuming to search for suitable partners.They may feel more comfortable and find more convenience trading through an intermediate trader, like an agent or broker in real estate business.This role of intermediate trader particularly in a local retail electricity market allows them to keep additional energy transaction options besides only selling back or buying from utility companies.There are already several pilot projects and demonstration projects underway, verifying the possibility of monetized local energy exchange.For instance, since 2010, Pecan Street Inc. (Austin, TX, USA) has been collecting high-resolution data on how and when homes and small businesses in the United States use and, when PV is present, generate energy [61].Then, this temporal and spatial information of energy usage/generation can be used to test potential energy trading programs along with predicting the market capacity.On the other hand, in order to reduce the energy transaction cost and search friction in such an indirect customer-to-customer trading paradigm, a local energy market is proposed in [62] to accommodate localized energy trading and exchange for communities, buildings and campuses, which may own surplus local energy produced by on-site DERs.
In this framework, as shown in Figure 4, an important new role, named the energy broker (EB) and working as a middleman or trader in this localized retail electricity market, is introduced to get buyers and sellers together, serving as a solution to search friction [63].Both the buyers and sellers who would like to participate in this local energy market will provide bid/offer information of price-quantity pairs (price (PC) and amount (AM) in Figure 4) in each open time interval.The trader itself will also choose to maximize transaction efficiency or revenue with consideration for search cost in each open market time interval accordingly.The index of historical credit for energy transactions, sellers' commitment probabilities (SCPs), is also proposed for power allocation of different trading peers.In this way, the proposed market structure can be modeled based on search theory and an optimal stopping problem (OSP) [62].Some other similar works about this topic can also be found in [64,65].

Prosumer Community Groups
The prosumer community group is another typical prosumer paradigm that aims to provide common platforms for coordinating neighboring or local prosumers for exchanging energy and information within the local community or interfacing with outside energy entities as a whole.The authors in [25] argue that energy sharing among neighboring PV prosumers in the microgrid could be more economical than the independent operation of prosumers.They propose an energy-sharing model with price-based demand response (DR) for microgrids of peer-to-peer (P2P) PV prosumers to validate the benefit of forming prosumer communities.In [66], a new vision for local distribution systems is proposed, in which prosumers are encouraged to better balance their electricity usage in a local community through psychological balancing premiums.Even the social behaviors and some quantitative psychological characteristics of self-interested prosumers are considered in modeling the energy exchange and transactions.Price-responsive generation and demand of an individual prosumer are affected by his/her inherent characteristics and the individual's attitudes toward benefit and comfort, which evolve during social interactions.The authors in [67] also introduce a novel concept to manage prosumers in the form of goal-oriented virtual communities.They meanwhile discussed different aspects of the formation, growth and overall management of a prosumer-community.The main significant implication of this approach is that the prosumer-communities are able to facilitate the joining together of prosumers with similar interests.In this way, the quantity of energy to be auctioned to the smart grid can be increased accordingly, and furthermore, the prosumers' bargaining power is increased in the energy market.In a smart community, the benefits of DERs can also be considered in an energy management scheme, where a large number of residential units can participate and a shared facility controller (SFC) can be introduced [68].The SFC is defined as a public controller that exclusively controls electricity for those publicly sharing used equipment, devices and machines (e.g., water pumps, lifts, parking lot gates, lights, etc.) by the residents of the community.
Therefore, the SFC needs to afford all its energy cost either buying from the main grid or buying from the residential units with DERs due to its lack of electricity generation capability.

Methodology
The methodology used in the study of the retail electricity varies greatly according to particular application scenarios, including making market rules, predicting customer behavior, reducing system operation cost, and so on.On the other hand, all the methods applied in different projects also depend on how to describe the dynamics of the market mechanism in a quantitative way, namely system modeling.In this section, some common methodologies are discussed.However, those methodologies are usually not applied very independently and have the trend of being combined in a hybrid framework to make the system modeling more accurate and efficient.

Optimization, Distributed Optimization and Blockchain
Optimization methods are still dominant in most decision-making problems pertaining to system and market operation.Stochastic optimization, robust optimization, multi-objective optimization and mathematical programming have been widely adopted for research on the wholesale market for market-clearance, and most of this research takes into consideration various types of uncertainties resulting from variable demand or renewable energy supply [29,35,58].Retailers in the retail electricity market often refer to these optimization methods to guarantee their revenue through deterministic analysis.However, since there are numerous decision variables at the distribution level associated with frequent monitoring activities and a large number of customers, especially given more and more small-sized local generation units, global optimization has become rarely implemented due to its increasing computational complexity.Consequently, the state-of-the-art strategy has begun to shift to distributed optimization with necessary decomposition, such as the alternating direction method of multipliers (ADMM), consensus-based algorithms, proximal message passing (PMP), and so on [19,69].
Blockchain technology, borrowed from the IT industry, has also attracted much attention due to the prevailing distributed optimization implementation in practice.It has been suggested as promising and suitable for such a decentralized decision-making process [70].The authors in [71] present an architecture for peer-to-peer energy markets that can guarantee that operational constraints are respected and payments are fairly rendered, without relying on a centralized utility or microgrid aggregator.They demonstrate how to address trust, security and transparency issues by using blockchains and smart contracts, two emerging technologies that can facilitate decentralized coordination between non-trusting agents.While blockchains are receiving considerable interest as a platform for distributed computation and data management, this work may be the first one to examine their use to facilitate distributed optimization and control of DERs.Some other works also introduce the utilization of blockchains in local energy transactions between DERs, including a custom-designed blockchain mechanism designed to maintain a distributed database trusted by all DERs and to stipulate and store a smart contract that enforces proportional fairness [72].

Game Theoretic Method and Prospect Theory
In prosumer-centric energy trading, since most interconnected microgrids or DERs operate autonomously and have their own goals of optimizing performance and maximizing benefits through energy trading, the selfish nature of players participating in local energy transactions is inclined to be described by game theoretic methods.A Nash bargaining theory-based incentive mechanism is proposed and designed in [73] to encourage proactive energy trading and fair benefit sharing.It takes autonomous microgrids independent self-interested entities, without assuming that all the microgrids are coordinated by a common grid operator or controlled following a hierarchical structure.In [74], game-theoretic day-ahead energy scheduling in a residential distribution system is proposed, in which the distributed electricity prosumers may only compete with each other while some of them may cooperate with others to form a coalition.A similar noncooperative Stackelberg game between the residential units and the shared facility controller is proposed in [68] in order to explore how both entities can benefit, in terms of achieved utility and minimizing total cost, respectively, from trading energy with each other and with the grid.
It is noteworthy that the proposed game in [75] accounts for each prosumer's subjective decision when faced with the uncertainty of profits, induced by the random future price.In particular, the framing effect from the framework of prospect theory is used to account for each prosumer's valuation of its gains and losses with respect to an individual utility reference point.Prospect theory (PT) is mainly an interpretative theory that considers weighting effect to transform the objective probabilities into subjective probabilities, which was proposed to explain the fact that people usually over-weigh the low probability bad outcomes and under-weigh their favorite outcomes with high probabilities [75].PT is helpful for modifying conventional game-theoretic methods because it relaxes the assumption of rationality in most game frameworks by taking into account subjective irrational decision behavior [14,76].It is not at the same level as game theory, but possible to be combined into building utility functions in game models.Even so, most game-theoretic methods still possess too much simplification, making it hard to find the equilibrium solution, especially for large-scale systems with high computational complexity.

Agent-Based Simulation
Agent-based simulation (ABS) has been another popular tool, since at least the early 1990s, to model the dynamics of the electricity market, including both the wholesale market and the retail market [77].ABS is particularly suitable for large-scale systems involving various types of interacting system participants.These participants are usually assigned distinct roles, functionalities, behaviors and decisions, which depend on different objective design and interactions with other system participants [78].In an agent-based system, an agent can be as simple as a single variable (e.g., energy price-amount pair) within a computer program or as complex as an intelligent object, such as a human being (e.g., speculator), involving possibly an infinite number of states, decisions and actions/reactions.However, most ABS are mainly designed for the electricity wholesale market, neglecting transmission/distribution grid constraints [79,80].The difficulty of validating an ABS model's outcomes against empirical data is also one of the weaknesses of the ABS methodology.
In recent years, many agent-based systems have become popular again for electricity market simulation, due to the further development of reinforcement learning and the other computational resources available.The Power Trading Agent Competition (Power TAC) is an influential event and simulator that allows rich competitive simulation of future retail power markets and helps with understanding the dynamics of customer and retailer decision-making and the robustness of market designs.Power TAC models a liberalized retail electricity market, where competing business entities or brokers offer energy services to customers through tariff contracts [81].On the other hand, some researchers also mimic the wholesale market mechanism to study the behavior of a day-ahead retail electrical energy market with price-based demand response from air conditioning loads through a hierarchical multi-agent framework [82].Meanwhile, ABS is also frequently used as a validation tool for testing certain market rules for policy makers.For instance, an agent-based simulation of the liberalization of a retail electricity market has been developed to introduce competition into a sector historically characterized by the regional monopoly of retail electricity [83].It is worth mentioning that most existing ABS usually assign some learning capability to intelligent agents and often leverage the Q-learning algorithm from the machine learning field [82].

Machine Learning Techniques
Machine learning has become the status quo for most intelligent systems, including power systems and the electricity market.Utilizing machine learning techniques to detect distinct energy consumption patterns of customers and select high-quality customers for energy programs (e.g., demand response programs) is becoming more and more popular in addressing competitive utility companies and the future energy business ecosystem [84][85][86][87].
Together with various types of machine learning techniques, including successful application of supervised learning in demand response targeting [85] and unsupervised learning in individualized pricing design [87], reinforcement learning (RL) is also believed to have the potential to deal with the energy trading problem and guide energy entities to interact with the market environment.The most important feature distinguishing RL from other types of learning is that it uses training information that evaluates the actions taken rather than instructions by giving correct actions [88].This is very suitable for economic activities (e.g., energy transactions) that are based on the voluntary principle and associated with privacy issues regarding consumption information.On the other hand, the online nature of RL makes it possible to approximate the best decision-making strategy or optimal policies in ways that put more effort into learning to make good decisions for frequently-encountered states (e.g., high energy demand during the daytime), at the expense of less effort for infrequently-encountered states (e.g., peak load happening at night).The project in [89] demonstrates a data-driven control approach for demand response in real-life residential buildings, in which the objective is to optimally schedule the heating cycles of the domestic hot water (DHW) buffer.
However, most RL applications in power systems depend heavily on Q-learning or other tableau methods, which are based on look-up tables and afford low computational efficiency with increasingly big datasets.In recent research advancements, combining deep learning (DL) and RL to form a deep Q network (DQN) is suggested as a good approach for value function approximation and improving algorithm performance [90].It can also conquer many of the weaknesses (e.g., feedback delay, partially-observable environments, numerous enumerations) in the energy system decision-making process for the retail electricity market.
These methods, including the optimization model, game-theoretic model, agent-based simulation and machine learning, are usually correlated with each other and often combined together as hybrid frameworks.For instance, the game-theoretic method that finds the equilibrium point is easily transformed to an optimization problem that solves for equivalent optimal results [74].Agent-based simulation is mostly combined with machine learning techniques to facilitate the interaction dynamics among different agent entities [82].The machine learning technology also frequently uses optimization methods to train its model parameters and hyper-parameters [85].Last but not least, the summary of each individual modeling method is shown in Table 1.

Discussion and Policy Issues
Based on the various aforementioned studies of the retail electricity market in recent years, some trends can be easily observed that: (1) the system or market operation is more fine-grained from different perspectives, trying to balance credits' assignment and benefit sharing among many types of market entities, including suppliers, speculative retailers, utilities, service providers, customers and other new parties introduced by new business models; (2) more and more consideration is given for economic operation on top of pure system requirement satisfaction, and a certain degree of risk is acceptable given the improving uncertainty of the whole system; (3) customers are expected to be more active in this market-loop instead of passive participants, which are allowed to directly interact with other market participants and exercise negotiation power.
The study of the electricity market is more or less not a pure technique problem, especially considering the fairness rule (e.g., non-discrimination), data privacy and renewable energy subsidy policy in the retail electricity market close to the customer side.In North America, the U.S. electricity ownership structure is actually quite complex.The U.S. electric power industry consists of approximately 3300 publicly-owned, investor-owned and cooperative utilities; more than 1000 independent power generators; 3 regional synchronized power grids; 8 electric reliability councils; about 150 control-area operators; and thousands of separate engineering, economic, environmental and land use regulatory authorities [15].We provide a retrospect of the history of U.S. electricity deregulation in Table 2 based on our previous work in [15] and hope to remind that electricity deregulation should keep track of the development of emerging technologies, especially considering the manipulative market power brought by these technologies and new business models.Further deregulation of the retail electricity market definitely requires cooperation and technical support from the wholesale market, which is still under intense discussion across the industry and the academic community [91].

2008
FERC issued Order 719 to improve the competitiveness of the wholesale electricity markets in various ways, and to enhance the role of RTOs.

2012
FERC issued Order 768 to facilitate price transparency in markets for the sale and strengthen the Commission's ability monitor its retail markets for anti-competitive and manipulative behavior.
Although our discussion is mainly focused on the U.S. electricity market, it is worth mentioning that many countries in Europe and on other continents also meet similar challenges with the retail electricity market, such as electricity buy-back volatility, cross-subsidies, distribution cost allocation, and so on [92].Take for instance the residential U.K. electricity market: it was opened for the first time in 1999, introducing the choice of supplier, and about 40% of households changed supplier in the first four years.After three years, price caps were removed.The work in [93] reviews this process and assesses the competitiveness of the market by examining how the charges levied by suppliers depend on cost and factors for three different payment methods and consumption levels, whose experience may be helpful for U.S. retail electricity market development.However, the market deregulation process is not always so successful and full of various kinds of challenges that are far beyond our expectation.Some researchers have summarized two main negative phenomena that could reduce the impact of introducing competition in the retail electricity market: cognitive bias affecting consumers' decisions to switch and a technological paradigm reducing innovation opportunities in commercialization [94].These discussions can go on and on due to the many research perspectives involved in this field.Research on the electricity market is always hungry for more interdisciplinary study from other fields, such as economics, computer science and operational research.

Conclusions
To help researchers have an overall understanding of the recent research work on the retail electricity market, different sub-topics with/without prosumers and commonly-used methodologies are surveyed and discussed in this paper.The state-of-the-art, emerging new market functionalities (e.g., DSO's new role, incentive mechanisms, transactive energy, prosumer community groups) and recent innovative techniques (e.g., prospect theory, blockchain, reinforcement learning) have been discussed, covering the entire landscape of the retail electricity market.
In the survey of more than 90 papers published within the last five years that study the retail electricity market, the phenomenon can be observed that more and more intelligent system technology, like machine learning and the Internet of Things, is coming into play in this field.These new automation methods and autonomous systems or controllers allow customers to easily coordinate with each other and actively participate in the electricity market, instead of only passively accepting what they are provided.Another observation is that innovative business model design remains the key driving force behind the reform of traditional energy exchange and transactions.
We intentionally skip some common topics, such as load forecasting and demand response, covered by many existing survey papers, and mainly focus on the most recent developments in the area of innovative conceptual frameworks in the study of the retail electricity market.In the future, more localized energy market models under incubation will come into practice and revolutionize the whole energy ecosystem.

Figure 2 .
Figure 2. The further deregulation of the retail electricity market.

•Figure 3 .
Figure 3.The new role design of the DSO.ACOPF, alternating current optimal power flow.

Figure 4 .
Figure 4.The localized retail market with energy broker and search theory.SCP, sellers' commitment probability.
The deregulation of the wholesale electricity market.

Table 1 .
Solution methods for the new paradigm of the retail electricity market.

Table 2 .
History of U.S. electricity deregulation.intent of the Act in 1996 with Orders 888 and 889, with the stated objective to "remove impediments to competition in wholesale trade and to bring more efficient, lower cost power to the nation's electricity customers."2005CongresspassedtheEnergy Policy Act of 2005, a major energy law to repeal PUHCA and decrease limitations on utility companies' ability to merge or be owned by financial holding/non-utility companies.2007FERCissued Order 890, reforming the open-access regulations for electricity transmission, in order to strengthen non-discrimination services.