Prosumer Flexibility: A Comprehensive State-of-the-Art Review and Scientometric Analysis

Abstract

There is a growing need for increased flexibility in modern power systems. Traditionally, this flexibility has been provided by supply-side technologies. There has been an increase in the research surrounding flexibility services provided by demand-side actors and technologies, especially flexibility services provided by prosumers (those customers who both produce and consume electricity). This work gathers 1183 peer-reviewed journal articles concerning the topic and uses them to identify the current state of the art. This body of literature was analysed with two leading textual and scientometric analysis tools, SAS© Visual Text Analytics and VOSviewer, in order to provide a detailed understanding of the current state-of-the-art research on prosumer flexibility. Trends, key ideas, opportunities and challenges were identified and discussed.

1. Introduction

One of the more unique characteristics of power systems is that the consumption and production of electricity should occur at the same time. Energy storage systems (ESS), mostly pumped hydro systems and more recently battery ESSs have challenged this paradigm but for the vast majority of the time, the consumption and production of electricity must occur simultaneously. Both supply and demand for electricity fluctuate, but, until recently, it was the fluctuations in demand that were of primary concern [1,2]. Demand profiles change drastically over different time periods, from days to weeks to months. The supply of electricity was thus built around these demand fluctuations with different electricity-generating technologies used to meet different aspects of demand across varying time periods [3]. Baseload demand was largely met by nuclear and coal-fired power plants, while the peaks in demand were met by gas turbines or hydroelectric power plants. This paradigm was largely based on supply following variable demand. The power system required some flexibility to account for deviations in demand and this was largely met by increasing or decreasing the production of electricity from various power plants.

This paradigm has started to shift with the awareness that the power system needs to decarbonise and this has been largely done by the introduction of a large amount of variable renewable energy sources (vRES), mostly wind and solar photovoltaic (PV) [1,4]. The power produced from vRES cannot be dispatched and is subject to significant variation and uncertainty. This has increased the complexity of operating the power system and requires new sources of flexibility as the existing sources are no longer sufficient. The changes being experienced by the power system are shown in Figure 1. This figure shows how there are three drivers of change, which are the need to decarbonise the energy system by increasing the penetration of Renewable Energy Sources (RES), the need to reduce system costs and the need to increase the flexibility of the system. These drivers are met by four enablers of the energy transition which are novel technologies, new business models, innovative market design, and new methods of system optimisation and operation. The combination of drivers and enablers produces an ideal opportunity for prosumer flexibility to develop and become a key aspect of future power systems. In this paper, the definition of a prosumer follows from [5,6] and the concept is defined as follows: those customers who decide to invest in distributed energy resources (mostly solar PV) for a variety of reasons and now can satisfy a portion of their electricity demand and, in some hours, even produce more than they consume, possibly through storing some of the self-generated energy.

Apart from helping to match supply and demand, sources of flexibility can also assist in various ancillary services such as frequency and voltage profile control and helping to defer certain costly investments in network infrastructure [7].

Historically, flexibility sources have been provided by the supply side of the power system and this was done by changing the level of electricity produced (either increasing or decreasing), investing in more flexible technologies, regional integration of power systems, or curtailing RES production. Also, methods to increase the flexibility of the system from the demand side include demand response programs and energy efficiency initiatives, while grid-side measures to increase flexibility have included optimal network reconfiguration, incorporation of smart grid technologies, and line rating altering [3]. In addition, ESS can also provide flexibility and efficiently designed markets and regulations can help to incentivise the uptake of flexible technologies [8]. Flexibility resources on a large scale (industrial, commercial, and wholesale measures) have been widely used for grid management in the past. There has been an effort to increase the flexibility of the demand side of the power system as well. Recently, there has been discussion around the increased utilization of flexibility provided by prosumers, but these measures have largely failed to take root. These difficulties can be largely explained by technological shortcomings as well as issues related to developing markets for prosumer flexibility. In the preceding years, developments in technology have largely solved these issues and there is a concerted effort to introduce incentives and markets to increase the impact of prosumer flexibility [8,9].

In recent years, demand-side measures to increase flexibility have become more common and are now thought to be the least costly and most viable means of adding flexibility to the power system especially with large amounts of RES generation. This has meant a shift away from the traditional paradigm of supply following demand to a paradigm of demand being altered to meet the available supply of electricity. The potential of the demand side to contribute flexibility has been widely discussed and the potential for demand-side flexibility is thought to be significant. Methods to change the demand profile of consumers (through either monetary or non-monetary incentives) have been used. These methods could involve a permeant modification of consumption such as energy efficiency initiatives or more temporary measures such as time of use tariffs [8].

This shift has also meant that consumers are becoming more active in the energy system. They are transforming from passive agents to prosumers who can take an active role in future electricity systems. One of the major resources that has been used for modulating demand in terms of both magnitude and timing has been demand-side management (DSM) [10]. The most common strategies of DSM may include load shifting, valley filling, conservation efforts, planned load growth, and peak clipping. DSM is still largely operated by the utility, but it does involve higher customer participation. Increasing the level of customer participation is key in building future energy systems where customers are placed at the heart of the system. Prosumer flexibility can increase the level of participation and allow the energy system to be more customer-centric [11]. True customer participation will occur when the customer can decide how when and why they interact with the wider energy system. Customers should be allowed to utilise their resources in a manner that first and foremost benefits them.

To maximise the societal welfare delivered by an energy system, it will be important to maximise the welfare of all participants in the system, including prosumers. The key to maximising the benefits that prosumers can offer will be providing them with the necessary knowledge and decision-making framework [12,13].

In addition to providing consumers with the necessary technical knowledge, the political and regulatory issues relating to the market design and incentives for the energy system to embrace prosumers must be also resolved. This is most evident in the European Commission’s Clean Energy for all Europeans initiative which is designed to put the consumer at the heart of the energy system and leads to a decentralised energy system based largely on RES and allows customers to actively participate in the energy system using various assets and resources [14]. This Clean Energy package requires the member states of the European Union (EU) to begin the process of removing barriers that currently prevent demand-side resources from actively participating in wholesale and retail markets [14].

This has meant that there is considerable interest in quantifying the potential for flexible services to be provided by the demand side of the power system, with a clear focus on the provision of these services by prosumers. This paper serves to summarise the current literature surrounding prosumer flexibility and to apply a scientometric analysis to the body of literature to better understand the trends, key aspects, opportunities and challenges within the body of knowledge for this emerging and rapidly growing area of research.

Contributions

This paper provides a thorough review of the state-of-the-art literature surrounding prosumer flexibility. This is an important and growing topic and such a review provides a detailed understanding of the status quo in this field of research while also highlighting some important trends that are emerging in the research. An extensive body of literature was collected and, to this corpus, two state-of-the-art textual analysis tools were applied. The combination of these tools ensures that the body of literature is analysed fully and that robust conclusions can be drawn concerning the research on prosumer flexibility. Specifically, this paper provides the following novel contributions:

• Robust collection, analysis, and synthesis of state-of-the-art peer-reviewed literature surrounding the growing field of prosumer flexibility;
• The application of various textual analysis tools to a large body of literature to distil relevant insights;
• A discussion about the opportunities, challenges, and trends of prosumer flexibility.

2. Demand Side Flexibility

Demand-side flexibility (DSF) is essential for the cost-effective and efficient integration of RES into the energy system [15,16]. Increasing the flexibility of the system through demand-side flexibility can help to better manage the fluctuations not only in demand, but also increasingly fluctuating supply. DSF can help with the idea of the local balancing of the system which can help with the reliability and resilience of the energy system. There are concerns that increasing customer participation may not be desirable from a customer’s point of view if this increase in participation leads to increasingly complex decisions about their energy use at various times throughout the day [17,18]. This concern can be alleviated by the automation of DSM programs through smart grid technologies of Home Energy Management Systems [19].

While the potential for prosumer-provided flexibility has been widely discussed, a clear distinction needs to be made between the theoretically available potential for prosumer flexibility provision and the realizable amount of flexibility that prosumers may offer. This depends on the technologies considered in the analysis as well as the type of flexibility program studied. In addition, the amount of flexibility available largely depends on the behaviour of the customer and the analysis of this behaviour has been difficult to forecast and then modify [12,20]. Changing the behaviour of customers is a crucial problem to solve if the potential for DSF is to be realised [21]. In addition, it is difficult to reliably determine the energy demand and flexibility resources available to a certain prosumer at a given time. There is significant uncertainty about these issues and aggregators of flexibility services could play a significant role in resolving these concerns [22,23]. The aggregation of various types of devices (each with their flexibility profile) requires a detailed understanding and modelling of that specific device [24]. Some studies have raised concerns around the collection of a large number of private consumers’ data as well as issues relating to the communication overheads that such systems require [25]. Issues are also raised about the effect of a large number of prosumers carrying out coordinated actions within a small geographic area, especially when a large number of electric vehicles (EVs) is present in the distribution grid [26,27].

3. Methodology Followed

3.1. Textual Analysis

In order to present a thorough review of the current literature around prosumer flexibility, a detailed scientometric analysis was carried out. There has been a rapid increase in the amount of literature concerning prosumers and flexibility; however, a detailed analysis of this literature has not been presented. This paper uses a combination of two tools that use natural language processing, machine learning, and various linguistic rules and applies them to a large dataset containing over 1000 academically peer-reviewed articles concerning prosumer flexibility. The results of this analysis are used to inform the remainder of this paper as they provide a structure of the most relevant terms used in the literature surrounding prosumer flexibility. The methodology used in the scientometric and text analysis which was carried out to examine the data associated with the collected 1183 peer-reviewed journal articles is presented in Figure 2.

3.2. Tools and Datasets Used

A combination of two well-known tools was used in this analysis. The first tool used was SAS© Visual Text Analytics in SAS© Viya© (version 8.5, SAS Institute, Cary, NC, USA) and the second was VOSviewer (version 1.6.15, Centre for Science and Technology Studies, Leiden University, The Netherlands). Both tools are designed to extract information from a large textual database and these two tools were chosen to evaluate the differences that might occur and to produce a complete assessment of the body of literature. The statistical and linguistic methods of SAS© Visual Text Analytics were used to extract information about the authors, keywords and dates of publication of the various papers [28].

VOSviewer is software designed to carry out a scientometric analysis of bibliometric networks. The text mining functionality of VOSviewer was used to construct and determine the co-occurrence networks of the abstracts of the various papers, where co-occurrence is defined as the network of links between terms in the keyword lists of the studied papers [29]. VOSviewer was developed by Nees Jan van Eck and Ludo Waltman at the Centre for Science and Technology Studies at Leiden University [29].

In total, 1183 distinct papers were analysed. The papers were sourced from the ScienceDirect and IEEE Xplore portals with the keywords ‘prosumer’ and ‘flexibility’ in November 2019. The existing literature concerning flexibility was surveyed and a number of reviews focused on flexibility services, especially from the grid side, were found [1,3]. Thus, the decision was made to focus on a specific aspect of flexibility provision, with a clear focus on the demand side provision of these services. Existing literature reviews were found on the following search terms: demand-side flexibility [30], residential flexibility [31] and local flexibility markets [32]. It was seen that there has not been a thorough review of the flexibility provided by prosumers and thus this research sought to chart the rise of the active participation of prosumers in electrical networks by examining the current state of the art of research concerning this topic.

The searches were limited to journal articles and thus the papers analysed in this study are peer-reviewed and of a high standard. The first step of the analysis was an initial screening to ensure that the papers concerned the electricity system and dealt with flexible services provision by active customers within distribution networks. Of these 1183 papers, 186 papers were removed at this stage as they were not relevant to the topic of prosumer flexibility.

The metadata of each paper was then extracted from the relevant database, either ScienceDirect or IEEE Xplore. The data extracted included the title, authors, keywords, date of publication and the full paper abstract. The information was then put into a database that was fed into either SAS Viya or VOSviewer depending on the type of analysis carried out.

During the analysis, a minimum threshold of occurrences of a specific term was used and the threshold used was a minimum of 10 occurrences of the term. This corresponds to the term appearing in approximately 1% of the 997 articles studied. A total of 26 terms met this requirement. A relevance score was calculated by VOSviewer to highlight the most relevant terms. As a default option, the 60% most relevant terms were included in the network plots. The 60% most relevant terms were then verified manually and added to the network plots.

4. Scientometric Analysis Results

The results of the scientometric analysis are divided into four subcategories, year, author, keyword, and abstract. Each of these subsections provides some relevant information relating to the body of literature surrounding prosumer flexibility.

4.1. Prosumer Flexibility Over Time

From the available papers, the frequency of papers dealing with prosumer flexibility published during the years between 2008 and 2020 are shown in Figure 3 below. The first use of the term prosumer appeared in [33] in their discussion surrounding uncertainties in the design and operation of distributed energy resources (DERs).

As can be expected, there are very few papers published dealing with prosumer flexibility up until 2016, where there is a noticeable increase in publications. There were 134 papers published in 2016 that dealt with prosumer flexibility, which was slightly less than all of the papers published from 2008–2015.

4.2. Who is Researching Prosumer Flexibility?

The next section of the text corpus to be analysed was the number of papers published by various authors. From the 997 papers, a total of 3477 distinct authors were identified. The frequency of authorship is shown in Figure 4 below, which used SAS Viya textual analysis.

From the textual analysis, Christian Breyer has the most articles dealing with prosumer flexibility with a total number of 28 followed by Dimitrii Bogdanov, with 20 articles. Both of these authors collaborate closely with each other and are from Lappeenranta University in Finland.

4.3. Abstract Analysis

The full abstracts of all 997 papers were studied using VOSviewer to provide a detailed scientometric analysis of the papers [29]. VOSviewer allows for two types of counting methods for words within abstracts. These two methods are full counting and binary counting. Full counting means that each occurrence of a word within an abstract are counted, whereas with binary counting, the number of times that a word occurs within an abstract is not counted, all that is counted is if that word appears within the abstract. Full counting is used in this study as it can provide a more detailed description of the frequency of words used in an analysis.

Two images are presented, dealing with the analysis of the abstracts using the full counting methodology. These images are the network and density plots from the VOSviewer tool. The network plot shows the interrelation between certain terms and this is shown in Figure 5. The different colours represent the clustering of the terms with each other. The size of the circle is related to the frequency of the word within the abstracts and the links show the words that occur most frequently with the chosen word.

As can be expected from an abstract that aims to give a brief overview of the work, including the motivation behind the research and a summary of the results, the words identified by VOSviewer are quite generic, with the most frequent words being cost, research, and problem. Business model, load, and power system are the next most frequently used words that are more closely related to the energy system. The words are clustered into groups of words that most often appear together and, in this research, two major clusters emerge: the ‘Red’ cluster, which is more closely associated with power systems, for example, ‘electricity’, ‘microgrid’, and ‘battery’, and the ‘Green’ cluster, which is more associated with the general planning of academic research and contains words such as ‘research’, ‘innovation’, and ‘information’.

VOSviewer allows the use of a so-called thesaurus file, which allows the user to clean the data by specifying a two-column table, one column with the title ‘label’ and the other titled ‘replace by’. The user can specify words in the ’label’ column and have them replaced by terms in the ‘replace by’ column for the abstract analysis [29]. In this paper, this method was done to a small degree but care was taken to not alter too many terms as would materially affect the results and not provide a complete overview of the current state of the art of prosumer flexibility research.

The density plot represents the number of times that a word is used within the abstracts and is shown in Figure 6. For the full counting methodology, the three most common words were research, cost, and problem.

4.4. Keyword Analysis

The final section of the analysis was done on the keywords of each of the 997 journal articles. For this analysis, both SAS Viya text analysis and VOSviewer were used. This was done to compare and contrast the results of the two methods.

The results of the keyword analysis conducted by VOSviewer is presented in Figure 7, where the size of the circle represents the frequency of occurrence. The lines show the links between the keywords. The colours of the keywords represent clusters of terms that most often appear together.

The term flexibility is found to occur in two different ways, which are ‘flexibility’ and ‘energy flexibility’. This splitting of the term helps to explain why flexibility is not the most common keyword in the body of literature. However, each of these terms has been isolated and thus the connections between them can be studied in depth.

Firstly, ‘flexibility’ is isolated and the resulting connections are shown in Figure 8. Within its cluster, the most closely associated words to ‘flexibility’ are ‘demand response’ and ‘smart grid’ with ‘prosumer’, ‘market design’, ‘load management’, and ‘power systems’.

Secondly, the term ‘energy flexibility’ is isolated and studied in Figure 9. Like the ‘flexibility’ figure, there are strong connections to ‘smart grid’ and ‘demand response’, but also strong connections to ‘renewable energy’ and ‘energy management’.

The keyword analysis conducted in SAS Viya is highlighted in Figure 10. The word cloud shows the most common keywords in all 997 papers. The size of the word represents the frequency of occurrence with the term ‘smart grid’, occurring 196 times in the papers studied.

This shows that ‘prosumer’ and ‘flexibility’ are often used alongside the much more widely used terms ‘smart grid’ and ‘demand response’. This trend may begin to change as more research is carried out on both ‘prosumer’ and ‘flexibility’.

To provide a complete overview of prosumer flexibility, the top five most frequent terms used in the keywords, as analysed by SAS Viya Text analysis, are presented in the following sections. These keywords are smart grid, demand response, microgrid, prosumer, and distributed energy resources. In each of the following sections, the term is described in detail and its relation to the concept of prosumer flexibility is presented.

5. Key Issues from the Literature

This section will discuss each of the top five most common words, as identified in the text analysis conducted with SAS Viya. Each of the identified issues is important enough to have a whole field of dedicated research behind them, but the interactions between them, and other issues, are essential for the concept of prosumer flexibility. The cyclical interaction between the terms is shown in Figure 11 below, where the prosumer owns the DERs which participate in demand response (DR) programs enabled by smart grids and microgrids, who have prosumers as their participating members.

5.1. Smart Grid

The term ‘smart grid’ was the most commonly used keyword of the 997 papers analysed. Despite the term occurring 196 times in the literature, there is no real agreed-upon definition of smart grid. Through VOSviewer, the term ‘smart grid’ was isolated and the connections to other terms are shown in Figure 12. As can be expected, it forms a central part of the keyword analysis with strong connections to several important topics in this research field.

Although smart grid is the most commonly used keyword in the body of literature analysed, a consensus exists about the concept, the technologies needed and the requirements of a smart grid [34,35]. This clear consensus is best described by the definition of a smart grid provided by [36] as an advanced digital power system using bidirectional power flows, which is capable of self-healing while also operating in a resilient, adaptable, and sustainable manner using forecasting techniques to manage various types of uncertainties. Another widely used definition is provided by the European Commission’s Strategic Energy Technologies Information System. In their document, entitled:

“Strategic Deployment Document for Europe’s Electricity Networks of the Future”.

They define a smart grid as an intelligent electric network that incorporates the actions of all agents who use the grid to efficiently provide electricity that is sustainable, economic, secure [37]. In these definitions, smart generally applies to the use of technologies and software that increases the ability of the grid to operate autonomously. This can benefit the grid and the users in both a short-term operational timeframe as well as in the long term. Various technologies and software, such as supervisory control and data acquisition systems, Energy Management systems (including Home Energy Management Systems) and Demand Management systems can be utilised. These technologies are used to allow the grid to incorporate more active participation from its users [38].

Relating to prosumer flexibility, the advanced technologies and software of the smart grid can be used to accurately track, measure and predict the demand of customers in a very granular manner [39]. This information can then be used to estimate the demand response or flexibility potential of the customers. Also, these smart grid technologies can be used to increase the ease and speed of communication of customers, not only with the centralised system operator, but also with other customers as well [39]. Smart grids can also play a major role in increasing the penetration of RES (including various types of DERs). Shortages or surpluses of electricity can be rapidly quantified and actions (such as demand response) can be activated to balance the grid [40]. Several other smart grid systems and technologies can be used (such as Advanced Metering Infrastructure, Substation Automation, and Preventive/Self-Healing Mechanisms) [36,41,42].

Prosumer flexibility sits within the wider area of demand-side flexibility. Demand-side flexibility can cover a wide range of flexibility operations, including resources owned and operated by a wide variety of agents (including prosumers, system operators and utilities) [43]. Prosumer flexibility is dealing with the flexibility resources of individual customers who are actively participating in the distribution system. These customers can use a variety of tools and technologies to participate, but the key aspect is that they own and operate the technologies and are actively choosing to engage with the distribution system for a variety of reasons—for example, for financial gain or environmental consciousnesses.

As can be expected with smart grids being the most frequently used keyword in the literature on prosumer flexibility, smart grids are seen as a key enabler of increased prosumer flexibility. Smart Grids can assist in providing access to flexibility markets for a variety of different actors [44].

Despite the potential of smart grids to enable prosumer flexibility, there are still many concerns and challenges that smart grids need to overcome [36,42]. Ensuring the security of supply of energy with a RES based system is a major obstacle to overcome. However, there are a host of potential solutions to this issue including Energy Storage Systems, small modular generators and multi-energy systems [45,46].

A key smart grid technology for unlocking prosumer flexibility is the smart meter. These devices can measure, record, transmit and receive information about the energy use of a customer with very fine granularity and can help to provide rapid feedback to customers about their energy use and thus potential energy costs [47]. System operators and utilities can also use these devices to dynamically alter the price according to the demand and supply of electricity which can help to either incentivise load curtailment or increase the use of flexibility resources. Information about the real-time supply and demand of energy across a system will be essential for any energy system of the future and smart meters can help to record and spread this information [48].

Another technology, or more specifically a group of technologies, that can help to harness the potential of prosumer flexibility is the Home Energy Management System (HEMS). Broadly speaking, HEMSs are composed of a collection of controllable appliances whose energy use can be altered to meet the needs of the occupants in an optimal manner [19,49]. HEMS, like smart meters, provide valuable information on the energy usage profiles of various technologies and can manage these profiles to suit the needs of the inhabitants. HEMS can also incorporate DERs to more effectively manage the production and consumption of electricity within a household. The overarching goal of a HEMS is to ensure that the energy needs of the inhabitants are met as efficiently as possible but there are other goals that the HEMS may be required to meet such as lowering the energy bill, increasing the self-consumption of electricity from the household’s DERs or actively participating in DR programs [50,51].

While HEMS can assist in unlocking the potential for prosumer flexibility, there are some concerns and obstacles to its widespread adaptation. The major concern is providing connectivity to a wide variety of devices that are manufactured by different suppliers and thus the interoperability of devices will be a key obstacle to overcome [52]. Again, the issue of data privacy and security is of great concern to the users of HEMS. Methods to safeguard this data will need to be developed before HEMS can become widely used [53]. A concept that is linked to HEMS is the idea of smart buildings. Smart buildings can offer similar characteristics to HEMS but with commercial and office buildings as the main focus rather than residential buildings [54].

5.2. Demand Response

The second most frequent keyword within the studied literature was ‘demand response’. This is unsurprising as DR is a foundational concept and has been studied, for instance in [41,50]. Demand response provides flexibility to the electric system as it can alter the consumption of electricity through the use of changing prices or by offering incentives to customers to alter their load at certain times or during certain events [55,56]. The connections surrounding the term ‘demand response’ are shown in Figure 13.

The official definition of DR is given by the US Federal Energy Regulatory Commission and it defines DR as the following:

“changes in electric usage by end-use customers from their normal consumption patterns in response to changes in the price of electricity over time or to incentives payments designed to induce lower electricity use at the time of high wholesale market prices or when the system reliability is jeopardized” [57].

This definition covers prosumer flexibility and, as such, prosumer flexibility can be thought of as a tool that can be used in DR programs. The results of effective DR programs are to increase the electric system’s flexibility, responsiveness and adaptability according to various signals (for example changing costs of electricity) [58,59]. There exists a split between price-based and incentive-based DR. Price-based DR, as the name suggests, uses changes in the price of electricity to alter the demand of customers, whereas incentive-based DR programs use incentives (either financial or non-financial) to alter consumption patterns. Effective DR programs can deliver increased flexibility to the electricity system as well as reduced costs for the consumer, amongst other benefits [56,60]. Demand response programs have existed for numerous years, but it is recently where the majority of interest in DR programs has occurred [61,62]. DR programs can reduce electricity costs for customers and increase the penetration of RES [47,63].

One interesting application of DR programs to prosumer flexibility is the use of aggregators to combine a large number of individual customers to gather significant flexibility to participate in existing markets. DR, through aggregation, can utilise incentive-based or price-based DR programs. Incentive-based programs can include direct load control, load curtailment, demand bidding, or emergency demand reduction programs, while price-based DR can utilise changing electricity tariffs to induce the required behaviour from consumers [56]. The use of a direct control mechanism provides the aggregator with direct control of an individual’s resources and thus allows the aggregator to be certain of the timing and quantity of energy available to participate in any market. As mentioned previously, smart meters are an essential technology not only of smart grids but also of effective DR programs. Meters will need to be able to receive and send accurate price signals to both the operator as well as the prosumer [64]. Moreover, this will be essential for aggregators to gather sufficient information about the consumer’s power usage to participate effectively in DR programs. The different effects of DR programs on the load profiles are shown in Figure 14. These include peak clippings which is a reduction in the peak load, valley filling, which incentivises power use in low demand periods to reduce the peak to average load ratio, Load Shifting, which incentivises load being moved over time from high to low periods of demand and Flexible Load Shaping, which helps to smooth out the load profile over time. In addition, there can be energy efficiency measures that aim to reduce the magnitude of the entire load profile and strategic load growth, which aims to do the opposite, i.e., to increase the load demanded.

Despite these clear benefits, there are still some issues and challenges surrounding DR programs. These issues are the lack of appropriate DR markets, polices, effectual forecasting techniques, appropriate control methods, as well as issues relating to communication [3,56]. A further challenge to the effective use of DR by prosumers emerges from the prosumers themselves. Consumer behaviour is incredibly difficult to predict [20,65]. Individuals have a wide variety of priorities that change over time, age, income level, weather conditions and are often not consistent [11,20,66]. This is made more challenging when more than one individual lives in a household. Another significant challenge is associated with privacy and data security. Participants’ data used in a DR program will need to be kept safely and securely [67,68].

5.3. Microgrid

The term microgrid was the third most frequently keyword in the studied literature. There is more consensus about the definition of microgrid compared to smart grid but there still exist several different definitions of the term microgrid. As with the term ‘flexibility’, there are two different ways that microgrid is used in the literature. The first is ‘microgrid’ and the second is the plural ‘microgrids’. The connections between each of the two terms are shown in Figure 15 and Figure 16 respectively.

In this paper, the definition used by the Department of Energy of the United States is used and it defines a microgrid as the following:

“A group of interconnected loads and distributed energy resources within a clearly defined electrical boundary which acts as a single controllable entity with respect to the grid. A microgrid can disconnect from the grid to enable it to operate in both grid-connected or islanded mode” [69].

Hirsch et al. further generalises this description to have three requirements of a microgrid. These three requirements are (1) that is must be possible to identify the part of the distribution system to which the microgrid is connected to, (2) the resources used within the microgrid should be used together in a coordinated manner rather than in coordination with distant resources, and (3) the microgrid can operate regardless of if it is connected to the wider grid [70].

Researchers in [71] define a microgrid as a cluster of loads and DERs and these microgrids can have a variety of objectives.

Generally, microgrids are composed of a variety of renewable energy sources, combined heat and power plants, energy storage systems, controllable loads [72,73] The microgrid can also disconnect itself from the larger network and operate in islanded mode to supply energy to meet the local demand [74,75].

Microgrids can optimise the local resources to meet several criteria, such as minimising the energy cost to users or minimising the amount of energy imported from the wider grid, and this can then increase the amount of flexibility within the microgrid [76,77]. Aggregations of several smaller distributed energy resources to participate in energy markets can be done through microgrids and this is needed as these markets usually have some thresholds on the minimum size required for their participation [78]. Using microgrids to offer flexibility services through the forecasting and balancing of local supply and demand, which reduces the burden on distribution system operators, is also possible. microgrids can be seen as an additional layer in the hierarchical control system, sitting between prosumers, community energy systems and DSOs, for example [79].

5.4. Prosumer

The fourth most commonly used keyword in the 997 papers studied was prosumer. The connections associated with this term are shown in Figure 17 with ‘smart grid’, ‘energy transition’ and ‘renewable energy’ among the strongest connections. ‘Smart meters’, ‘blockchain’ and ‘energy hubs’ are technologies used to enable the growth of prosumers, so it is reasonable for them to have relatively strong connections.

A review on the topic of prosumption is provided by [80]. The authors show the origins of prosumer and prosumption are an article in 1980 by Alvin Toffler, but that the concepts behind the terms extend back to perhaps the age of hunter-gatherers. A major shortcoming that the authors point out is the view that consumption or production occurs as binary activities and that, actually, both production and consumption have always had elements of prosumption embedded within them. The authors highlight the role that technological change can play in the rise of prosumption. This role is discussed in depth in this current review article.

It is proposed by [81] that prosumption, rather than production or consumption, has always played a dominant role in capitalist economies. A key point of the article is that prosumer capitalism is largely reliant on the abundance and effectiveness of the means of production, rather than valuing production based on scarcity and efficiency. This is important for prosumers using PV systems in distribution networks. The PV systems generate electricity which is based on an abundance of natural resources (sunlight) rather than a scarce resource (generally coal or natural gas) even if the efficiency of the PV system is lower than the traditional fossil fuel-generating unit.

A thorough examination of the prosumer concept is offered by [82]. In this work, he discusses the prosumer in terms of its impact on marketing and this is relevant as he discusses the various reasons why a consumer may choose to become a prosumer in some aspects. Understanding this behaviour shift will be key to harnessing the full potential of prosumers, especially in the electricity sector. Kotler identifies characteristics of activities that are most likely to be taken over by a prosumer. These characteristics are that the activity should promote large cost savings, require low skill levels, require low time and effort and result in significant personal satisfaction. These characteristics are emerging in the electricity sector, especially surrounding the use of small-scale PV systems.

In the context of energy systems, prosumers are those customers who decide to invest in distributed energy resources (mostly solar PV) for a variety of reasons so they can satisfy a portion of their electricity demand and, in some hours, even produce more than they consume, possibly through storing some of the self-generated energy, thus becoming prosumers [5,6]. This extra electricity can be exported to the electrical grid and/or shared with other consumers within their local area and, in this manner, prosumers can help to meet the environmental, social, and economic issues surrounding increasing energy demand [83]. This energy sharing can take the form of local peer-to-peer markets such as those studied in [84,85] Not only can prosumers export excess electricity but they can actively modify their demand [86]. This active modification of demand, energy trading within the local area, as well as their self-consumption mean that prosumers can play a key role in future energy systems [87,88].

Traditionally, customers had a passive role in the energy system as pure price-taking agents. This role is changing as prosumers can now take a more active role in the energy system, including setting the price of the excess electricity that they might want to sell. This increased decision-making power of prosumers can be based on several criteria such as minimizing costs or maximizing the social welfare of their community [89].

In the past, the major aspects of the interaction between customers and the distribution system operators revolved around technical and economic aspects, but new dimensions are becoming important with the rise of the prosumer. These new dimensions include behavioural, social and organizational aspects [90].

Prosumers can offer flexibility services in many ways. They can optimise their electricity use and production and other groups can use this information for supply/demand forecasting and requirements for ancillary services, including flexibility. Prosumers can also relinquish control of their various distributed energy resources to DSOs or aggregators who can use these devices to help manage the grid, for example, the German Energy company, Sonnen, offers residential battery systems which can be controlled by a third party to help manage the electrical grid [91].

The topic of designing energy markets to account for the rise of prosumers was studied by [92]. In the paper, they state that there is a fundamental difference between prosumer-focused markets and traditional platforms for engagement in the energy sector such as demand–response markets. This is because prosumers are not solely price-taking agents, and can also actively offer services to other market agents. This active engagement is a key defining trait of prosumers. In the paper, the authors identify three possible market arrangements of prosumers, which include: peer-to-peer markets, prosumer-to-interconnected, or ‘island’ mode systems or organised prosumer groups. The authors describe these markets in detail.

Also, the authors describe several cross-cutting issues that a prosumer-based market may bring. These include technical issues related to control and management strategies of various distributed energy resources, issues related to the capacity of distribution grids to handle an influx of DERs. The authors also raise concerns about the economic and regulatory impact of prosumers relating to the possible grid defection and associated ‘utility death spiral’ that may emerge from an uncontrolled and sudden shift to using DERs [92].

Finally, the authors also discuss concerns relating to the social and behavioural changes that prosumers may bring. Chief among these concerns are information asymmetries, range anxiety over EV driving, and issues related to the perceived privacy risks of sharing data [92].

5.5. Distributed Energy Resources

The fifth most frequent term found in the text analysis was distributed energy resources (DERs). This is expected, as DERs are the key technologies that allow prosumers to actively participate in the energy system. The connections within the body of literature to ‘distributed energy resources’ are shown in Figure 18. The following terms occur frequently:

• Smart grid
• Energy transition
• Distributed generation
• Smart meters
• Solar PV
• Battery storage
• Blockchain

Generally, DERs are a grouping of various technologies, most commonly solar PV, ESS (both stationary storage and electric vehicles), heating and cooling technologies and various smart appliances that provide the user some flexibility, and these resources are generally thought to bring some positive benefits to the electricity system [93,94]. The following sub-sections will highlight the five most common DER types as identified in the body of literature and present a short description of each. These five DER types are stationary energy storage systems, thermal loads, heat pumps, electric vehicles, and PV systems.

5.5.1. Energy Storage Systems

Energy storage systems help to shift energy across time, typically from periods of high availability and low prices to periods of high demand and high prices and can have significant benefits to the electrical network (especially in the distribution network) [8,95]. There is a whole suite of available types of ESS and some of them are shown in Figure 19. For prosumers, the power required would fall below 100 kW and the energy needed would be below 1 MWh, which means the technologies best suited to prosumers would be those in the lower-left section of the figure. Energy storage systems, particularly lithium-ion battery ESS, are becoming widespread in their adoption and use in prosumer portfolios. This has been fuelled by the increasing amount of variable renewable energy and the dramatic falling capital costs of these ESS [96,97].

ESS are especially valuable as they can be deployed in a variety of contexts, both large scale and small scale, behind the meter or in front of the meter. They can also be used to provide ancillary services to the surrounding grid due to their fast response times. ESS can be effectively used to reduce a portion of the need for peaker plants, which are the most expensive and often the most polluting plants to run [45]. The variety of ESS also means that as each type of ESS has its advantages and disadvantages, it is possible to develop a portfolio of ESS to optimally suit the needs of a certain location or community [94,98,99]. By storing energy for periods of high demand, ESS can provide significant flexibility as the ESS can act as both a source and a sink of energy depending on the requirements of a certain time. ESS can also reduce peak demand and thus postpone costly grid infrastructure upgrades [100,101]. ESS can provide multiple services to the grid and a number of these services work synergistically and this allows the owner of the ESS to profit from multiple revenue streams (for example, energy arbitrage or frequency control), which helps to make the business case for ESS that often have high capital costs [102].

In summary, ESS can provide the following main benefits: an increase in the effective use of RES, a reduction in the peak power required at a given time and an increase in both the reliability and stability of the grid. With that being said, there are a few drawbacks to the use of ESS, which include the costs associated with them, additional infrastructure and management of the assets, and uncertainty around the novel services and business models.

5.5.2. Thermal Loads

Another common grouping of DERs is thermal loads. These are generally thermostatically controlled loads with some heat storage capacity and these loads operate within a band of upper and lower temperatures. Domestic hot water boilers are the most common of these types of loads and may be able to provide a significant amount of flexibility to distribution systems but this flexibility may vary significantly with many factors [103].

Another group of devices with a large potential for demand response and flexibility are white goods (these include refrigerators, dishwashers, clothes washers and clothes dryers). The potential for flexibility will depend on the operating regime and the level of interconnectedness with Home Energy Management Systems [49,104] There may be significant potential for flexibility on very short time scales as these devices can be delayed to better suit the needs of the owner.

5.5.3. Heat Pumps

Heat pumps may also be able to increase the flexibility resource of prosumers [105]. The operation of these devices can be optimally timed and thermal storage can be used to increase the flexibility offered by the heat pump [106]. There are several issues with this operational regime for heat pumps as the increased cycling could increase the need for maintenance and operating the heat pumps at non-optimal temperatures reduces the efficiency of the device [107].

5.5.4. Electric Vehicles

Electric vehicles may provide a significant amount of prosumer flexibility to the electrical grid, but the quantity and timing of this flexibility will be dependent on a large number of factors [108,109]. Different charging regimes ranging from simple on-off strategies to dynamic vehicle-to-grid regimes may be prevalent in the future and all offer different amounts of flexibility to prosumers and distribution grid operators [110]. Vehicle-to-grid services are archetypal prosumer flexibility services as they involve allowing the distribution system operator to utilise a prosumer-owned DER in exchange for some reward. The number of EVs within a small geographic area will also have a significant impact on the local distribution grid and these effects will need to be managed [111]. Stationary ESS can shift energy across time and EVs (through vehicle to grid technology) can also shift energy across the spatial dimension [112,113]. This may come in handy to help tackle the issue of geographical clustering of EVs due to economic factors as the EVs could be driven away from areas of high concentration to other areas (either from home to work or from work to home), thus helping diffuse the concentration of EVs at a particular location. Optimised charging of fleets of electric vehicles can support the local distribution grid in terms of demand response programs, peak shaving, valley filling, and ancillary services [114].

5.5.5. PV Systems

PV Systems are some of the most commonly installed DERs. These systems have seen extraordinary cost reductions in the past few years, which have helped to make them a widely accepted technology for generating electricity, either at the utility scale or the smaller prosumer scale [115,116]. PV systems are not a traditional DR load, as they generate power, but there is a degree of controllability to the technology, which means they are well suited to DER installations. PV systems can be used to generate power for self-consumption as well as charging stationary ESS or EVs [117]. PV systems are thus crucial for the prosumer concept as they allow consumers to generate electricity that can be used immediately, stored for later use or sold either to the local utility or to other consumers.

5.5.6. General Issues to Consider

Despite the advantages of DERs that have been discussed in the preceding sections, there remain some significant issues with their large-scale deployment [33,118]. One identified aspect is the tendency for DERs to cluster in certain areas or neighbourhoods for social or regulatory reasons [20]. This concentrating effect can place a further strain on distribution networks, which may already be operating in a sub-optimal manner and may need expensive infrastructure upgrades to deal with the requirements of the DERs. In addition to this geographic clustering effect, DERs may also create issues relating to new peak demand hours or other synchronization effects, for example, when low priced electricity is available, there may be a significant increase in demand and many consumers attempt to take advantage of this low-price period. There are some non-technical solutions that can be deployed to enhance the uptake of DR programs using DERs. These include regulatory and policy mechanisms as well as the creation of new energy markets [3,119].

The authors in [71] raise the issue that the uncontrolled use of distributed energy resources can have significant negative effects on the power system. Thus, efficient control strategies are needed for these DER assets.

A major enabler of efficient local energy markets with high penetrations of prosumers is the high forecasting accuracy of both production and consumption [120]. Applying forecasting techniques to individual consumers may lead to significant errors and this can, in turn, be a major barrier for the development of efficient local energy markets. The work done by [120] shows that by aggregating consumers (or prosumers), forecasts for energy demand are significantly improved.

In general, a large portion of the existing literature on DSF has mostly focused on the following major types of appliances: heating, ventilation and air-conditioning (HVAC) systems, washing machines and dryers, and water heaters, as they are widespread, have a relatively large energy demand and can be used in DR programs. A few studies compare residential appliances in terms of demand response potential [64] and these studies generally rely on intensive customer survey data, which is traditionally difficult to obtain from customers.

Various metrics have been proposed to quantify the flexibility offered by DERs. Some studies have classified flexibility according to a more traditional, supply-side definition of flexibility, which includes ramping magnitude (power), ramping cycle frequency, and the response time of an asset [121]. For DERs, additional metrics have been proposed such as minimum power available, the recovery time of the asset after deployment and the availability of the asset [122]. In addition to these extra metric requirements, the flexibility of DER assets relies heavily on the behaviour profile of the owner and this is extremely difficult to predict for individual owners (and assets) but this issue becomes easier to solve with the use of flexibility aggregators [43,123].

Using DERs in local energy markets can bring benefits to the user, as described in [124], where the authors state that there is an inverse relationship between the benefits of using RES and the distance between the RES and the end-user.

Novel DSF metrics have been proposed. These include the Flexibility Index of Aggregate Demand (FIAD) and the Percentage Flexibility Level (PFL) [89], which statistically analyse aggregated demand to help define time-varying flexibility resources. These indicators have some drawbacks, mostly around the assumptions relating to the way the fluctuations in load are assumed to be constant, and this has led to the development of the Modified Flexibility Index of Aggregate Demand (MFIAD) [122]. Other metrics suggested include a duration-of-use framework to develop new contracts for flexibility, as proposed by [125]. A metric that included the ability of DR programs to assist the grid is presented by [126]. This metric takes into account a Flexibility Affect Factor which measures the ability of a flexible resource, including demand response programs, to satisfy certain technical characteristics relating to the operation of the power system [126].

Research has shown that DSF cannot be described in a single indicator but rather should rely on several characteristics to fully describe the flexibility potential of a DER asset and this issue is addressed by [2], who state that to properly estimate the potential of a DER asset to provide flexibility, three aspects should be considered: the ability of the asset to provide flexibility, the wider system’s ability to provide flexibility and the system’s need for the flexibility. A further key aspect relating to the system’s need for flexibility is the variation of demand with temperature and this should include issues like different load curves due to long-term changes in the climate [2].

Like previous terms discussed in this paper, DERs also suffer from some drawbacks and challenges. This is especially true with DERs as they often provide novel technologies and business models to the electricity system, which has traditionally been very conservative in adopting new technologies [3].

There are examples of prosumers using DERs to generate, store and trade electricity within their local communities. Some of these examples include a peer-to-peer energy trading program in Malaysia where prosumers can use solar PV panels to sell excess electricity either to the national grid or to other consumers [127]. There has also been a similar project in Norway where a project sought to enrol previously passive consumers to become prosumers who could sell excess energy to the local grid or other consumers [128]. A project between Italian utility, Evolvere, and ABB Group hopes to use blockchain technology to encourage peer-to-peer energy trading [129]. An important project by the Australian Energy Market Operator (AEMO) seeks to investigate the potential for DERs within the energy system to help identify the markets and frameworks to help increase the penetration of DERs [130]. More significantly, California has one of the most successful distributed generation programs, which has been running since 2001 and as of August 2019 has funded 8999 completed projects with a cumulative capacity of 850 MW and another 260 MW of capacity in 4772 projects under development [131]. Piclo (formerly Open Utility) is a pioneering energy software company whose platform Piclo Flex allows flexibility, providing resources connected in certain areas in the UK to submit bids for flexibility services [132]. The first flexibility tenders were submitted in March 2019 to deliver flexibility in the years 2019/2020 and 2020/2021. Enera, a project between EPEX SPOT, a power exchange, EWE AG, an energy group, TenneT DE, a German Transmission System Operator (TSO) and two German Distributions System Operators (DSOs), Avacon NETZ and EWE NETS is enabling flexibility services to reduce the amount of curtailed renewable energy production, mostly from wind farms [132]. The Grid Operators Platform for Congestion Solutions (GOPACS) is a collaboration between the Dutch TSO and four DSOs (Stedin, Liander, Enexis Groep, and Westland Infra) and was launched in January 2019 [132]. Instead of being a platform, GOPACS acts as an intermediary between network operators and the markets through its connection with the Energy Trading Platform Amsterdam. A joint venture project between Agder Energi, a Norwegian utility, and Nord Pool, the power exchange, was established as NODES in 2018 and is being used in three pilot projects to help identify required flexibility services [132].

6. Trends Identified

While reviewing the existing literature, several key trends emerged. This section will briefly highlight some of these trends associated with prosumer flexibility.

Associated with prosumer flexibility and aggregation is the concept of community energy systems, where a collection of engaged prosumers works together to integrate DERs to increase the share of self-consumption of electricity and also potentially participate in local energy markets [133]. There exists a knowledge gap in evaluating the flexibility potential of these communities [102].

With smart devices and Internet of Things (IoT)-enabled devices becoming more readily available, their potential to provide real-time information to various stakeholders is being explored. This could affect prosumer flexibility by providing incentives to prosumers to increase the level of flexibility to engage further in DR programs in a very granular manner, both in terms of time and location [134].

The literature has shown a marked increase in the interest in prosumer flexibility and a clear trend to see is that from a technological level, existing technologies and operational strategies are largely sufficient to fully exploit prosumer flexibility. What is hindering the process is the regulatory and market design [16,79,132,135]. These areas will be key research topics in the coming years if the full potential of prosumers is to be harnessed.

A major trend identified in the literature and the wider field of energy systems is the move towards the increased electrification of end-use sectors such as mobility and heating as the energy system shifts towards a decarbonised and digitalised future [63,136,137,138]. This trend has the potential to radically reshape energy demand, including the demand profiles associated with different types of customers and thus this trend will need to be managed correctly. The increasing digitalisation of the energy system also poses challenges around the complexity of managing a grid containing huge amounts of smart connected devices. This digitalisation also brings opportunities for the analysis of large data sets by advanced techniques to extract useful information and increase the system flexibility.

One of the more pronounced trends is the aggregation of prosumers into collections where they can have a meaningful impact on the wider power system either through the use of commercial Virtual Power Plants or local energy communities. This is done to offer energy or ancillary services to various markets. Key to this trend is the increased temporal and spatial granularity that various markets are beginning to offer [119,123]. Peer-to-peer electricity trading is one such market design that has the potential to unlock significant prosumer resources [139,140].

7. Conclusions

This paper has provided a review of the literature surrounding prosumer flexibility. This was done through a textual analysis of relevant academic articles and the results of the textual analysis provided a structure to discuss several key issues surrounding prosumer flexibility. The opportunities and challenges of the different key aspects were discussed, and a holistic view of prosumer flexibility has been provided. This work complements other review papers in prosumer flexibility, demand-side flexibility as well as research concerned with quantifying the flexibility potential of DER assets.

This work has shown that the vast majority of research in flexibility has focused on the grid or supply side, but that this is slowly changing as the importance of having engaged consumers is becoming recognised. This paper has shown a significant rise in the amount of research around prosumer flexibility in recent years and it is expected that the amount of research will continue to increase as the active participation of consumers in the energy system increases.

This paper has synthesised a large body of research and identified several key trends in the research. The most relevant keyword in the body of literature was ‘smart grid’ followed by ‘demand response’. This can be expected, as these two fields are much more developed research fields. It is expected that as the concept of prosumers gains momentum, the keyword of prosumer will be much more widely cited in research.

This work shows that a textual analysis of a large body of related literature can provide interesting information and identify important trends with the chosen body of literature. Tools like VOSviewer and SAS© Visual Text Analytics provide important features for scientometric analysis.

There is an exciting mix of technical solutions being developed to make full use of the potential for prosumers to provide flexibility services to the electric grid. The vast majority of the technologies discussed in this paper are commercially available and are cost-competitive when compared to traditional forms of electricity generation.

With that being said, there are several barriers or obstacles to the widespread provision of flexibility services by prosumers. Some of these barriers include:

• Inadequate markets to fully harness the potential of prosumers. Current markets lack both the scale and scope to be fully inclusive to all participants in the energy sector. This will require a significant change in both what products energy markets offer as well as the conditions to entry of these markets. These changes to the market rules are slowly happening as regulators see the significant additional value that prosumers can provide to markets;
• Tariff and regulatory regimes that do not incentivise DER ownership by consumers. Special attention should be paid to designing regulations which incentivise the adoption of DERs by prosumers in a controlled manner and these incentives should be aware of the behavioural and societal effects that a rise in prosumers may bring. This type of regulation is beginning to emerge, especially in various EU member states as new regulatory regimes are introduced which actively promote consumer participation;
• A lack of information provided to consumers. Providing easily understandable and accurate information to consumers relating to both the technical and financial aspects of owning DERs is key to allow consumers to make informed choices;
• Inadequate business models and financing instruments. With the emergence of prosumers, novel business models and financing instruments are needed in order to make DER ownership easy and attractive to consumers. In this regard, regulatory agencies have a difficult task ahead. These agencies can spur business model innovation by introducing new regulations, but will still need to ensure that these regulations still protect the best interests of consumers;
• Privacy and data security concerns. Society is grappling with the issue of personal data use by third parties. The energy sector, with its tight relationship to ICT systems, will also have to devise mechanisms to ensure that consumer’s data is safe, secure and only used for the intended purposes. This area will be a key obstacle to overcome if the potential of active consumer participation is to be realised.

The trends and issues identified in this paper are extensions of those identified in [92]. A thorough review of the state of the art of the literature concerning prosumer flexibility was needed to identify opportunities, potential bottlenecks, and challenges that a future electricity system may face. The issues raised in this paper will become more pressing and important as the electricity system undergoes a transition to a decentralised, decarbonised, digitalised and democratic future where the consumer plays a much more central role in their energy choices.