Evaluation Metrics to Assess the Most Suitable Energy Community End-Users to Participate in Demand Response

Abstract

In the energy sector, prosumers are becoming relevant entities for energy management systems since they can share energy with their citizen energy community (CEC). Thus, this paper proposes a novel methodology based on demand response (DR) participation in a CEC context, where unsupervised learning algorithms such as convolutional neural networks and k-means are used. This novel methodology can analyze future events on the grid and balance the consumption and generation using end-user flexibility. The end-users’ invitations to the DR event were according to their ranking obtained through three metrics. These metrics were energy flexibility, participation ratio, and flexibility history of the end-users. During the DR event, a continuous balancing assessment is performed to allow the invitation of additional end-users. Real data from a CEC with 50 buildings were used, where the results demonstrated that the end-users’ participation in two DR events allows reduction of energy costs by EUR 1.31, balancing the CEC energy resources.

1. Introduction

Today’s goal to reduce carbonization of electrical power systems aids the significant penetration of distributed energy resources with an expectation to achieve, in the European Union (EU), a total share of 27% of renewable generation and a reduction of 40% of greenhouse gases by 2030 [1]. The integration of renewable energy sources and the increasing number of regular consumers that become prosumers transform the old centralized, unidirectional electricity grid into a more decentralized grid with bidirectional energy and communication flow [2]. As such, the existence of prosumers enables the concept of energy sharing in an energy community, where these entities can share their surplus of energy with other members [3]. This provides the opportunity for end-users in citizen energy communities (CEC) to be critical players in the energy sector, further contributing to the supply-demand balance by having more active participation, not only as prosumers but also by participating in demand response (DR) programs. Demand response can be defined as actively altering the normal energy consumption habits of end-users in reaction to variations in electricity prices, or in response to incentives to reduce consumption during high wholesale market prices, or when the systems dependability is compromised [4].

Participation in DR programs helps integrate electric loads and renewable energy sources cost-effectively [5]. In this context, end-users represent a potential source of energy flexibility and, by engaging in DR programs, can play a significant role in the transition to the decarbonization of energy systems, resulting in a clean energy future [6]. From a technology perspective, energy flexibility procurement can be done through the usage of internet of things (IoT) devices integrated into residential households, transforming them into smart homes, maximizing user comfort while minimizing the cost of electricity [7]. These smart homes are linked in a smart community through IoT devices, measuring and communicating detailed energy consumption information to energy management systems [8]. These management systems can precisely control individual home appliances via the use of power meters or smart plugs, operating in response to DR programs [9]. In this context, the application of clustering algorithms is important since they can facilitate the analysis of large amounts of data collected from smart meters [10].

Previous studies, such as [11,12], already present methodologies oriented towards end-user participation in DR events. In [11], a method is presented in which end-users are assigned a trustworthy rank according to their past participation experience. In this way, the aggregator would invite those with the highest trustworthy rank to participate in the DR event. However, since this ranking only depends on past participation, it is not possible for an end-user who missed some DR events to improve their ranking. In [12], a model based on clustering algorithms is presented to cluster several loads with similar consumption patterns to improve the DR events’ accuracy. Moreover, in order to incentivize participation, this study considers that the largest clusters have the lowest DR application costs. In this way, the cluster with lower DR application costs is prioritized during the DR event. The results demonstrated that end-users were compensated accordingly to their participation, reducing their electricity bills. Nonetheless, the study in [12] has its limitations regarding how it ranks the loads, where smaller clusters are penalized.

This paper proposes a novel methodology based on clustering algorithms for end-user participation in DR events in CEC. Considering the previous studies, the novelty of the presented methodology is ranking the end-users through the combination of three metrics based on clustering algorithms. This enables the correct selection of end-users to be invited to the DR program. With the combination of these three metrics, the end-users can have more opportunities to participate in DR events since the presented ranking is based on different types of end-user data, namely the end user’s flexibility, participation ratio, and historic flexibility. Furthermore, this methodology can analyze future events on the grid and balance the consumption and generation using the DR program. The presented methodology also considers a continuous balancing evaluation during the DR event to enable the invitation of additional end-users. This methodology was tested and evaluated using a simulation of a CEC based on real data. The results show that this novel DR methodology is very promising, as it allowed for the correction of the respective critical periods calculated, enabling the creation of new metrics such as shifting and fairness.

This paper is divided into seven main sections. After this first introductory section, a survey of related work is presented in Section 2. The proposed methodology of this paper is presented in Section 3. The case study and the proposed scenarios are detailed in Section 4. Section 5 presents the results, and they are discussed in Section 6. Finally, the main conclusions are presented in Section 7.

2. Related Work

This section presents a concise summary of approaches developed with the concepts used in this paper, namely CEC, community member active participation, and DR programs. Characteristics of each model are considered, focusing on their benefits and constraints and providing case studies to support them.

2.1. Citizen Energy Communities

According to its clean energy and decarbonization objectives, Directive 2019/944 of the European Parliament and of the Council of 5 June 2019 was enacted, providing new guidelines that will ultimately bring considerable benefits for the consumers, the environment, and the economy. These objectives aim to tackle global climate change with possible solutions on achieving carbon neutrality at the EU level by 2050. Furthermore, Directive 2019/944 acknowledges and regulates the concept of CEC, stating that “empowering jointly acting renewables self-consumers also provides opportunities for renewable energy communities to advance energy efficiency at household level and helps fight energy poverty through reduced consumption and lower supply tariffs” [13].

Choosing between different models of CEC is an obstacle to overcome due to the rush for climate neutrality and an increasingly intricate energy system. As there is no single solution for all the disadvantages of these models, various alternatives should be combined. In [14], the authors analyze energy communities located in heating-dominated climates and the impact of the local sharing of the generated energy from photovoltaic (PV) panels. This perspective resulted in the conclusion that by “aggregating individual energy demands, communities can offer local flexibility services such as relieving network congestion and avoiding peak withdrawal from the electricity grid.” [14]. Moreover, these results can help local governments understand the impact of these new CEC, allowing the development of supportive policies, and promoting them to end-users.

The CEC concept shows promise with energy efficiency; however, several factors limit its application into reality, one of them being a large number of decision variables such as the number and type of participants and distributed technologies. As such, reference [15] presents a mixed-integer linear programming-based optimal planning approach to overcome these challenges, benefitting financially each community participant as well as the whole community regardless of the electricity tariff used. In that actual case study, the model used was able to significantly reduce the total energy costs by 15% and the carbon dioxide emissions by 34%. This suggests that reducing CO₂ emissions and total annual energy costs can be done by transferring and sharing renewable energy between end-users and applying an optimal selection and operation of the distributed energy technologies.

2.2. Community Member Active Participation

The creation of the Clean Energy for All European Package resulted in the active participation of citizens in the energy policies of the EU and its Member States. These policies intend to enable individual end-users to take an active part in the energy transition. In [16], a model’s financial viability, applicability, and acceptance for the community and individual consumers and producers are explored. In addition, [16] defines two types of energy communities, the CEC and the Renewable Energy Community (REC), where each household or individual does not need to have any renewable generation source to join as a member in any energy community, promoting end-users to take an active part. Concluding that these energy communities will be adopted and accepted by the Member States if there is a reduction of electricity costs for participants, legal guidelines on reduced grid costs, taxes, and fees must be implemented.

Due to the concept of the REC being recent and not as straightforward as traditional market operators, end-users typically have several concerns about investing and participating in these energy communities. In [17], an approach is presented that supports end-user decision-making in an REC. In this case, an optimization of the product portfolio of the community is produced along with every end-user’s energy generation, needs, and tariffs. Furthermore, the model community demonstrates the movement of energy between end-users and evaluates the feasibility of the model. Applying a genetic algorithm, they calculated the optimal solution that maximizes the investment’s net present value. Various energy approaches and incentives were used in this community, resulting in different outcomes. The increase of incentives on energy generation resulted in a large generation capacity—however, incentives on shared energy made possible an optimal portfolio size based on end-user demands. Lastly, the sharing of energy was perceived as more encouraged than self-consumption.

Commonly, energy communities mainly comprise prosumers such as small households with photovoltaic generation. Consequently, in [18], an optimization model is proposed to schedule peer-to-peer transactions via the local electricity market, grid transactions in the retail market, and battery management. With the opportunity given to prosumers to choose between transacting energy between community members or with the retailer, the authors were able to study four scenarios, including the analysis of the impact of storage systems in the community. This resulted in energy savings of 11–13% with the use of a battery storage system; additionally, by including peer-to-peer transactions, the community can potentially save up to 25% of total costs.

2.3. Demand Response Programs

Lately, due to the rise of energy consumption and consequent reduction of energy resources, debates have surged on the importance of energy management. One of the approaches in the literature has been the usage of DR programs [19]. DR programs are characterized by altering the energy consumption patterns of end-users in smart communities to shave peak load demand substantially and decrease energy waste and energy prices [20].

Several attempts to apply DR programs to households can be found in the literature, as illustrated in [21], where a novel approach used the machine-learning algorithm Conditional Inference Tree to categorize households with the support of energy behavior data. This characterization could identify possible energy savers and request a different DR program. This increases the cost-effectiveness of the application of these programs and causes a reduction of critical peak usage.

DR programs aim to reach a balance between supply and demand. In [22], a methodology is proposed that utilizes the characteristics of households obtained from population census, activity-based energy demand models, and surveys on the intent to participate in energy shifting and DR programs. The authors obtained reliable quantification of household segments’ impact on DR flexibility by employing this method. Most noticeably, this study was able to show that appliances such as laundry machines could deliver reducible flexibility during peak hours when using DR programs, reducing the load on the electrical grid and consequent overall price of electricity. A similar study proposed a DR model for households to mitigate the intermittency in a smart grid equipped with renewable energy sources [23]. The simulation results show that this model can contribute to the effective operation of the smart grid as well as decrease the energy costs of several houses.

In the literature, other studies like [12,24] present DR models based on direct load control (DLC). In [24], a method is presented that tries to reduce the negative impact that DLC has on consumers while efficiently managing energy resources. In that method, consumers enrolled in DR programs establish a contract with the utility company, specifying how controllable loads can be used. The results demonstrated that the method allows better management of energy resources. In [12], a study based on DR models that consider the k-means algorithm and DLC is proposed. This study focuses on efficiently managing energy resources and minimizing the community’s energy cost by reducing the consumption of several appliances from different consumers with similar patterns. This method was tested using real data from three small communities, where the results showed its efficiency. Although DLC produces significant benefits, this technique can negatively impact the end-user’s lifestyle as it may also discourage end-users from participating in future events. Thus, to enhance end-user engagement to participate in DR events, the proposed work in the present paper will not consider DLC.

In the context of DR programs, several studies based on clustering algorithms can be found [25,26]. For example, in [25], a dynamic clustered community-based home energy management system is highlighted to support a microgrid. The results show that this system could reduce the peak to average ratio by 21% and the cost by 4% for the presented case study. Furthermore, in [26], a cluster-based optimization strategy is presented to assist an aggregator in defining day-ahead energy market demand and supply bids. This strategy was tested with a case study of 10,000 Iberian prosumers, where the results showed a reduction of execution times and costs of more than 20% in comparison to an inflexible bidding system. Based on this approach, applied to aggregators to maximize their profits in the electricity market, the proposed work on this paper will apply the use of clustering algorithms for DR participation, enabling the classification and selection of ideal end-users by considering the context of each DR event.

3. Proposed Methodology

According to the reviewed literature, the presented studies have a particular focus on the participation of the end-users in energy communities, where the end-users can have their energy cost and demand optimized through their participation in DR programs or peer-to-peer transactions. However, none of the studies consider the ranking of end-users, which is essential to consider during the planning of DR events, for example.

In this way, this paper proposes a novel methodology that assesses the CEC’s end-users participating in DR events. Figure 1 illustrates the six steps of the proposed methodology.

The first step is the day-ahead energy forecasts that enable the perception of the next-day scenario. Then, critical periods where DR programs can be launched to balance the community’s grid are identified. In the next step, end-users are queried regarding their flexibility for the critical periods. Then, considering the flexibility of each end-user and their historical participation data, the community members are classified through clustering algorithms to create a ranking. Next, the invitations for the most suitable end-users are sent after the end-user ranking is calculated. The final step happens during the DR event, where the community energy balance is closely monitored to track the end-users’ participation and to check if the balance is verified.

The proposed methodology is based on an IoT architecture that considers three cooperative entities: the community manager, the end-user, and the respective IoT devices that monitor and control energy loads and resources. Each of these entities can process data; the community manager has the highest level of processing, being responsible for most of the DR event planning.

In this paper, the presented methodology focuses on the end-user ranking to correctly select the ideal end-users who will be invited to participate in the DR programs. In the reviewed literature, none of the studies considered ranking the end-users through the combination of different metrics that evaluate end-user data. Instead, the majority used only the participation history [11], pattern [12], or flexibility [26] to rank the end-users. With this regard, the novelty of the presented methodology is the combination of three metrics to rank end-users, namely the end users’ flexibility, participation ratio, and historic flexibility. The goal of this combination of metrics is to give more opportunities to the end-users to be selected to participate in DR programs.

3.1. Forecast

Initially, the methodology’s first step consists of performing the day-ahead forecast for different data. Forecasting models are essential components to enable energy management ahead of time, aiming to increase efficiency [27]. Therefore, the use of forecasting models in the literature is common, such as to forecast energy consumption [28], PV energy generation [27], wind energy generation [29], and end-user flexibility for DR participation [30]. In the scope of this paper, day-ahead energy forecasts for the CEC—namely consumption, PV generation, and flexibility—are performed.

To enable the use and testing of the proposed DR methodology, a forecasting model based on convolutional neural networks (CNN) was conceived, allowing the forecasting of the different data types from the CEC. In the context of this paper, the forecasting algorithms are used as auxiliary tools for the proposed methodology. The forecasting algorithms were tuned using different trials. Figure 2 shows the forecasting model’s results, highlighting the forecasted CEC consumption ($F_C$), generation ($F_G$), and flexibility ($F_F$).

The dataset used has one year of historical data (i.e., 8760 records), where each record includes the time frame and respective values of consumption, generation, and flexibility of each end-user. In the case of consumption and flexibility, only working days (i.e., 6288 records) were considered. Each CNN was trained with this dataset, where the data was divided into train (70%), validation (20%), and test (10%).

The CNN model has three layers: the input layer, hidden layer, and output layer. In the case of the day-ahead forecasting model for consumption and flexibility, 17 inputs were considered in the first layer, while 16 were considered for PV generation.

In the forecasting of three CEC data types, these inputs were considered: the respective historical data, 11 lagged variables related to the data type that will be forecasted (t + 1,…, t + 7; t + 25,…, t + 28), and four time-related variables to represent, in two dimensions, the day and year. These four time-related variables represent the time frame in 2D. In addition, in the case of the CEC consumption and flexibility forecast model, the identification of the weekday is also considered as a variable. The hidden layer has 256 neurons and is configured through a rectified linear activation function. The output layer only has one neuron representing the forecasted value related to the data type. To train the CNN algorithm, batches with a size of 32 samples and 30 epochs were considered.

3.2. Identification of Critical Periods

Subsequently, the forecasts are submitted for the community manager’s analysis to determine if critical periods need to be corrected through DR events. Depending on the context, these periods correspond to different situations, such as peak load periods and grid congestion. For example, in the case of peak load periods, it is necessary to perform peak shaving that can be done through demand-side management strategies, technologies that use energy storage systems, and electric vehicles [31]. In the case of grid congestion, this, occurring more at the distribution and transmission level, can be dealt with by strategies based on energy markets [32]. Similarly, these periods can also correspond to moments where it is necessary to balance grid resources, that is, to adjust consumption according to the respective generation, and the adjustment needed can be to increase or decrease consumption [33].

In the context of this paper, the respective critical periods correspond to moments where generation is not sufficient to mitigate consumption, and where the available flexibility could reduce the consumption to an optimum value. In other words, the critical periods are instances where Equations (1) and (2) are verified.

$$F_C\ge F_G$$

(1)

$$F_G\ge F_C-F_F$$

(2)

In Figure 2, the critical periods where energy balancing can be done are identified: at 11:00, at 12:00, and at 14:00. In these periods, the available flexibility allows the reduction of consumption to level off with generation.

3.3. Assessment of the Community Members Participation

In case there are critical periods, the community manager requests end-users to assess how much flexibility they can provide in the respective periods. This query process is based on the flowchart illustrated in Figure 3.

The end-users, after the request, calculate how much flexibility they can provide in the respective periods. The flexibility provided by an end-user varies according to the DR model established in the agreement with the community manager, where three possibilities are considered in the proposed methodology. The first possibility is the use of the aggregated flexibility of the residence, where this flexibility only represents the possible reductions in consumption. The second possibility is the use of two types of flexibility, allowing reducible and shiftable loads. Finally, in the third possibility is the use of individual loadsflexibility for each. In the scope of this paper and to facilitate the analysis of the proposed methodology, the end-users will use the first possibility, providing aa aggregated flexibility value. The interaction and communication between the community manager and the end-users is based on the Hypertext Transfer Protocol (HTTP).

3.4. Community Member Ranking

In this step, the community manager makes the ranking of the end-users using unsupervised learning models, like k-means, to identify which will be the participants for the DR event. This ranking is based on previous studies [9,10], where the logic of the ranking and clustering of the end-users participating in DR events is considered. Figure 1 shows the different processes that occur in this step. The respective ranking takes into account the flexibility that end-users can provide in period t (obtained in Section 3.3). It also considers different end-user historical data, namely the total number of times the end-user has participated in DR events, the total number of requests made to the end-user to participate in DR events, the percentage of end-user participation and, finally, the average energy reduction in DR events.

In this way, with the previously mentioned data, $N$ metrics can be created, i.e., ways to evaluate end-users, where each metric performs the clustering through two types of data. The two types of data are performing the clustering of end-users considering the percentage of participation and average energy reduction in DR events of each end-user.

The proposed methodology considers the use of three ($n$= 3) metrics: the percentage of participation vs. the average energy reduction, the percentage of participation vs. flexibility for period $t$, and the total number of times the end-user participated vs. the average energy reduction. All end-users are evaluated in each metric $n$, and the combination of the three will result in an end-user ranking table. The silhouette coefficient method is used to find the optimal number of clusters of each metric $n$. Each metric $n$ results in $C$ clusters, where each one has a centroid of the type (x, y). Each coordinate of the centroid corresponds to the average of the cluster elements for the respective data type.

After performing the metric $n$, it is necessary to apply a points system to them to evaluate the ranking of each of the end-users. In order to make this possible, firstly, in the metric $n$, the evaluation parameter is defined, i.e., the coordinate to which more importance will be given in the analysis of cluster centroids. In this way, the clusters are ordered by evaluation parameter in a decreasing order so the point system in the metric can be subsequently applied. In general, the logic of the proposed point system consists of assigning, per metric, 20 points to the end-users that belong to the best cluster, and the remaining clusters are assigned a score according to their relevance. Thus, if an end-user is always in the best cluster of each metric, it will have $20 n$ points, $n$ being the number of metrics under consideration (for this paper, $n$= 3).

Equation (3) shows how the points of cluster $i$, of a given metric $n$, are calculated:

$$ratin{g}_{n,i,e}=\frac{v{p}_i}{{\sum }_{j=1}^{C}v{p}_j}\ast 20$$

(3)

where vp_i represents the value of the evaluation parameter of cluster $i$, i.e., the centroid coordinate of cluster $i$. In Equation (3), vp_i is divided by the total sum of all ($j=\left\{1, 2, \dots , C\right\}$) clusters’ evaluation parameters to obtain their percentage concerning the other clusters. Then, this percentage is multiplied by 20 to obtain the respective points of the cluster$i$. It should be noted that the points of an end-user $e$ are the same as the points of the respective cluster to which it belongs.

In each metric $n$, to assure that the best cluster has a score of 20, Equation (4) is applied, which consists in adding to the score of cluster $i$ the score of the other clusters with fewer points. Considering that the clusters are ordered in descending order, the final points ($F\_ratin{g}_{n,i,e}$) of cluster $i$ is calculated as follows:

$$F\_ratin{g}_{n,i, e}=ratin{g}_{n,i,e}+{\displaystyle \sum }_{j=i+1}^{C}ratin{g}_{n,j,e}$$

(4)

Therefore, if we consider the final points of the best cluster ($i=1$), the calculation would be the points of cluster $i$ plus the points of the other remaining clusters ahead (e.g., for $i=2$, the other clusters will be $j=\left\{3, 4, \dots , C\right\}$). In this way, it is guaranteed that the best cluster has 20 points. For the remaining clusters, the process is the same, except for the last cluster. In the last cluster ($i=C$), its final points are equal to the points calculated with Equation (3).

After performing the previous processes in the $n$ metrics, it is possible to calculate the score of end-user $e$, through Equation (5). This equation, considering the scores of the clusters where end-user $e$ was allocated in the $n$ metrics, adds the scores calculated by Equation (4).

$$Rankin{g}_e={\displaystyle \sum }_{n=1}^N{\displaystyle \sum }_{i=1}^{C}F\_ratin{g}_{n,i,e}$$

(5)

For this study, the end-user ranking performed considers two months of historical data, along with the flexibility available for the critical periods.

3.5. Invitation to the Demand Response Event

After the end-user classification, the community manager defines the DR event participants. This way, according to the amount of energy needed for the critical periods, the first-ranked end-users capable of providing the necessary energy reduction are invited as main participants for the DR event. In the scope of this paper, the objective is to achieve a balance in energy consumption and generation at a CEC level. Therefore, not all end-users should be invited to not compromise the balance, as the participation of all end-users could decrease the consumption excessively below the generation.

The end-users that are not invited the day before will be identified as reserves, as they can be invited in real-time during the DR event. It should be highlighted that each end-user has a contract with the respective community manager. In this contract it is defined if the end-user is obliged or not to participate in the DR event, when it is invited to do so, and if the end-user must provide the entirety, or part, of the flexibility. To promote DR event participation, the members of the energy community could use an automation solution for energy management, promoting the use of automated demand response programs where no direct interaction of the user is necessary, after the previous parameterization of the users to define their preferences and restrictions.

DR participation invitations can be made on two occasions. The first, which is always done, consists of sending the invitation at the end of the day before the DR event, which allows the participants to be notified in advance. The second occasion only occurs when the DR event is being monitored. This monitoring is done in a loop every 10 min, where it makes it possible to check whether the DR event is having its effect, i.e., if the community’s consumption has reduced to the desired level. In this way, if the reductions made by the main participants are not enough to reduce the community’s consumption, the reserve participants are invited in real time to assist in the DR event, where further corrections to the CEC consumption are applied.

4. Case Study

The case study used in this paper consists of an energy community with 49 residences and one public building (in this case, a library) equipped with several appliances. The public building and 11 residences have PV generation. In total, the community has a contracted power of 341.55 kVA and an installed power of PV panels of 55.2 kVA. Additionally, the end-users’ consumption and generation are monitored and collected in 15 min periods through IoT devices. In this case study, end-users can opt to participate or not in DR events. In the case that they opt to participate, it is considered that end-users perform reductions equal to the value of the available flexibility.

Table 1. Appliances used in the energy community.

Appliances	Flexible Load	Rigid Load
Air conditioner	X
Dishwasher	X
Fridge	X
Washing machine	X
Tumble dryer	X
Lighting	X
Water Heater	X
Television		X
Microwave		X
Kettle		X

shows the loads available for individual monitoring and control in the community’s end-users. From these loads, seven are considered flexible loads; that is, equipment where consumption can be reduced. The remaining three are loads where consumption cannot be adjusted.

In the scope of this paper, DR events will be launched to balance the community, consisting of requests to end-users to reduce their consumption during critical periods to level the community consumption with the PV generation. This type of DR event considers the aggregated flexibility of each end-user, wherein this way, the confidentiality of the end-user’s load data is maintained.

Figure 2 illustrates the values forecasted by CNN for the community’s consumption, flexibility, and generation that are used as a case study to test the DR methodology. In this figure, it is possible to verify three critical periods at 11:00, 12:00, and 14:00. Of these three critical periods, the periods of 12 and 14 h will be highlighted in the later section, where we intend to balance the community’s electrical grid through the flexibility of the end-users. For example, in the case of the first critical period at 12:00, it is necessary to reduce consumption by around 1.6 kWh, while in the critical period of 14:00, it is necessary to reduce it by 1.4 kWh.

Regarding the classification of end-users, this paper considers three metrics:

• Metric 1: the first metric uses the percentage of participation and the average energy reduction, where the evaluation parameter is the percentage of participation;
• Metric 2: the second metric considers the total number of times the end-user participated and the average energy reduction, where the evaluation parameter, in this case, is the average energy reduction;
• Metric 3: the third metric considers the percentage of participation and flexibility for period $t$, with the respective flexibility being the evaluation parameter.

5. Results

This section illustrates the results obtained from the implementation of the proposed methodology.

Table 2. Input data for the end-user ranking—critical period of 12:00.

End-User	Total Request (#)	Total Participation (#)	Participation (%)	Average Energy Reduction (kWh)	Flexibility (kWh)
1	35	24	0.69	1.73	2.26
2	44	33	0.75	1.7	0.07
3	30	21	0.7	1.91	0.58
4	33	23	0.7	0.91	0.04
5	32	27	0.84	2.01	0.03
6	37	27	0.73	1.87	0.04
7	27	18	0.67	0.91	0.1
8	48	31	0.65	1.81	0.28
9	28	22	0.79	0.68	0
10	24	21	0.88	1.88	0.04
11	28	22	0.79	0.91	0.06
12	24	22	0.92	2.21	0.35
13	33	24	0.73	2.54	0.04
14	33	23	0.7	1.83	0.07
15	30	20	0.67	1.57	0.54
16	36	29	0.81	1.52	0.04
17	30	24	0.8	0.98	0.08
18	31	19	0.61	1.34	0.03
19	35	25	0.71	1.47	0.03
20	34	26	0.76	1.47	0.04
21	32	27	0.84	0.86	0.13
22	31	21	0.68	1.56	0.06
23	35	26	0.74	0.7	0.07
24	37	27	0.73	1.88	0.07
25	30	22	0.73	1.48	0.07
26	42	32	0.76	1.23	0.03
27	28	21	0.75	1.5	0.07
28	37	24	0.65	1.43	0
29	29	23	0.79	1.38	0.3
30	35	28	0.8	1.47	0.05
31	40	31	0.78	1.91	0.06
32	38	35	0.92	0.73	0
33	25	19	0.76	1.83	0.41
34	36	27	0.75	2.19	0.07
35	26	16	0.62	1.39	0.3
36	30	23	0.77	1.7	0
37	50	37	0.74	1.51	0.03
38	34	24	0.71	1.73	0.06
39	31	23	0.74	1.61	0.09
40	30	22	0.73	1.36	0.06
41	41	25	0.61	1.05	0.07
42	31	21	0.68	0.89	0.06
43	33	27	0.82	0.69	0.07
44	31	26	0.84	0.76	0.09
45	34	21	0.62	0.79	0
46	26	15	0.58	0.94	0.05
47	31	24	0.77	0.95	0.05
48	27	18	0.67	0.74	0.09
49	27	21	0.78	0.95	0.09
50	21	12	0.57	0.95	0.06

shows the inputs used to perform the end-user classification through the k-means algorithm. This table illustrates for each end-user, from left to right, the four historical data considered and the forecasted flexibility in the critical period $t$. This table shows flexibility corresponding to the critical period of 12:00 as an example. The historical data represent the last two months.

The data in Table 2 were used to perform the three metrics mentioned in the previous section. It should be noted that end-users with zero flexibility in the critical periods are excluded from the classification, as they cannot contribute to the DR event. As can be seen, five end-users have no flexibility: 9, 28, 32, 36, and 45.

Table 3. Score of the Top-15 End-users—critical period of 12:00.

End-User	Flexibility (kWh)	Metric 1	Metric 2	Metric 3	Score
1	2.26	14.67	14.58	20	49.25
8	0.28	14.67	20	3.62	38.29
12	0.35	20	14.58	3.62	38.2
2	0.07	14.67	20	0.8	35.47
31	0.06	14.67	20	0.8	35.47
13	0.04	20	14.58	0.8	35.38
3	0.58	14.67	14.58	3.62	32.87
34	0.07	20	9.3	0.8	30.1
10	0.04	14.67	14.58	0.8	30.05
38	0.06	14.67	14.58	0.36	29.61
14	0.07	14.67	14.58	0.36	29.61
37	0.03	4.8	20	0.8	25.6
26	0.03	4.8	20	0.8	25.6
11	0.06	9.67	14.58	0.8	25.05
49	0.09	9.67	14.58	0.8	25.05

demonstrates respective output from the ranking process, illustrating the top 15 end-users that will be invited to the following day’s DR event for 12:00.

Considering the first three end-users, end-users 1, 8, and 12, the data in Table 2 and Table 3 show that in metric 1, only end-user 12 obtained the maximum score since they have a higher percentage of participation than the others. End-users 1 and 8 have the same score in this metric because they achieved a similar percentage of participation. In the case of metric 2, only end-user 8 has a maximum score because they have a significantly higher value of the total number of participations. End-users 1 and 12 scored the same despite the values of their average energy reduction being slightly different. This is because they have similar values for the total amount of participation. For metric 3, end-user 1 has the maximum score, while end-users 8 and 12 share a considerably lower score. This difference in scores on this metric is because end-user 1 provides far greater flexibility than end-users 8 and 12.

Figure 4 and Figure 5 show, respectively, the real-time monitoring performed in a 10-min loop during the DR events that occurred during the critical periods of 12:00 and 14:00.

In each figure, the respective total community energy consumption and PV generation, the aggregated balance, and the respective corrections that were made during the DR event are illustrated. The total community energy consumption shown in the figures already includes the reduction made by the DR participants, i.e., it already includes the flexibility of the invited end-users. In these figures, the vertical axis on the left is for the aggregated balance and corrections (i.e., lines presented on the graph), while the vertical axis on the right is for the total consumption and generation of the community (i.e., areas presented on the graph). The respective aggregated balance corresponds to the aggregation of the community energy balance during the DR event, where the latter represents the total community consumption less the total community PV generation and the flexibility of the participants who were invited for the DR event. It should be noted that the aggregated balance only starts after instant 0, thus ignoring the consumption and generation values existing at that instant. If the DR event successfully balances the community, the last value of the aggregated balance should be close to zero.

The respective corrections are required when the aggregated balance is positive during the DR event. This means that more end-users need to be invited to decrease the overall community consumption. Therefore, these corrections are implemented in periods where the aggregated balance is greater than zero. In these situations, the following X reserve end-users are invited to participate, respecting the ranking positions. The value of X is defined by the sum of end-users’ flexibility that needs to meet with the currently needed reduction (i.e., enough to reduce the aggregated balance).

These corrections are based on the end-users’ flexibilities forecasted on the previous day. Each time a correction is implemented, it is implemented in the period in question and the remaining periods. This adds the flexibility of the reserve end-users chosen for that correction to the main end-users that were invited the day before. If the following period after the first correction turns out not to be enough, another correction is made, where the flexibility of the additional X reserve end-users that were not called in the first correction is used.

It is essential to highlight that the flexibility provided by the end-users during the monitoring of the respective DR events is different from the flexibility forecasted on the previous day. This is the direct result of the forecast errors and end-user participation deviation.

Table 4. Energy costs and the energy balance in the CEC for different scenarios in the two DR events.

	DR Event	Independent	BAU	All Flexibility	Correction 0	Correction 1	Correction 2
Cost (€)	12:00–13:00	2.469	0.268	−0.251	0.148	−0.146	−0.152
From the grid (kWh)		−41.730	0.000	−4.751	0.000	−2.777	−2.885
To the grid (kWh)		44.267	2.537	0.000	1.408	0.000	0.000
Cost (€)	14:00–15:00	1.746	0.417	−0.685	−0.022	−0.381	−0.472
From the grid (kWh)		−25.440	0.000	−13.610	−0.438	−7.574	−9.372
To the grid (kWh)		29.577	4.136	0.000	0.000	0.000	0.000

presents, for different scenarios, the energy cost and the energy balance in the CEC grid for the two discussed DR events, allowing a deeper analysis of the methodology presented in this paper. In the case where the energy balance is negative (excess of PV generation), it is considered that the energy is sold at a value that represents 50% of the respective dynamic tariff.

The “Independent” scenario represents the case where DR events and the CEC do not exist. In this case, the end-users cannot share their PV generation, and each end-user buys or sells their energy in case of excess PV generation. The “All Flexibility” scenario represents the case where all flexibility provided by end-users is considered in the DR event. This scenario makes it possible to see the maximum reduction obtained with the respective DR event. Finally, the business as usual (“BAU”) scenario represents the case where DR events do not exist, but the concept of CEC exists, allowing the sharing of PV generation among end-users. The “Correction 0” scenario depicts the impact of the participation of the main end-users during the DR event, where in Figure 4 and Figure 5 this correction is denominated as the Aggregated Balance. “Correction 1” and “Correction 2” depict the impact of the participation of the reserve end-users during the DR event.

6. Discussion

The results in Table 3 and Figure 4 demonstrate that the aggregated balance only considers end-user 1 since their flexibility would be sufficient to mitigate the 1.6 kWh of the predicted excess consumption. From Figure 4, it can be seen that initially, PV generation is higher than consumption, making the aggregated balance negative. However, 20 min after the event starts, consumption exceeds generation, and the respective aggregated balance becomes positive, with a value of 4.4 kWh at the end of minute 40. In this way, to reduce the aggregated balance to zero or a value close to it, Correction 1 is applied at minute 40, which considers the flexibility foreseen on the previous day of 38 end-users. These end-users are the ones who succeed the first ranked end-user, where in total it was predicted to provide about 4.4 kWh of flexibility during the DR event.

The application of Correction 1 was not enough, and, at minute 50, Correction 2 was applied. This second Correction considers the forecasted flexibility of the six remaining end-users, which is equal to 0.62 kWh. As shown in Figure 4, Correction 2 had little impact due to the little flexibility remaining.

In the period of 12:00, at the end of this DR event, and only considering end-user 1 participation (i.e., the blue line in Figure 4), it is possible to verify that the aggregated balance would be positive, that is, it was not enough to handle the respective critical period. Thus, with the continuous monitoring of the DR event every 10 min, it was possible to make the necessary corrections. At the end of the event, Corrections 1 and 2 (dashed red line in Figure 4) reduced the aggregated balance to a negative value, thus dealing with the critical 12:00 period. Furthermore, at the end of this event, total PV generation exceeded the value of total community consumption, further contributing to the reduction in the value of the aggregated balance.

In the case of Figure 5, the aggregated balance also only considers the flexibility of end-user 1. This is due to the forecasted flexibility on the previous day for this end-user being 4.4 kWh, which is more than enough for the 1.4 kWh of excess consumption. In this case, it is also seen that at the beginning of the DR event, there is an excess of PV generation. However, this only occurs until minute 20. Thereafter, between minutes 20 and 30, the total consumption of the community exceeds the total PV generation, causing the value of the aggregated balance to increase such that, at minute 30, the aggregated balance become positive. Thus, Correction 1 is applied where it was necessary to use the flexibility of the eight subsequent end-users to attenuate the value of the aggregated balance equal to 3.36 kWh. As shown in Figure 5, when comparing the aggregated balance with and without Correction 1 (i.e., lines blue and green in Figure 5), it can be seen that the reduction made was significant. However, the value is still positive at minute 40. Thus, Correction 2 is applied on top of Correction 1. In this case, an additional 11 end-users were invited to participate at minute 40, where they were expected to be able to provide, in total, 2.26 kWh of flexibility. As can be seen at minute 50 (dashed red line in Figure 5), both corrections caused the aggregated balance value to become negative, with Correction 2 reaching a significantly higher reduction than expected.

In the period at 14:00, at the end of the DR event, the aggregated balance that only considers the flexibility provided by end-user 1 would be enough to mitigate the critical period (represented by the blue line in Figure 5). However, corrections were still made since it is difficult to know how the aggregated balance will behave during the DR event. In this case, the corrections made would not be necessary at the end of the DR event.

Considering the monitoring done in both DR events, it can be seen that end-users provided more flexibility during the critical period of 14:00. A possible explanation for this is that the critical period of 12:00 comprises the lunchtime of most end-users. Thus, there is not much flexibility for each end-user, which consequently causes several end-users to be called to participate during the respective DR event.

Regarding the energy values illustrated in Table 4, these correspond to the real data of the CEC during the hours of the DR events, where these diverge from the forecasted values.

Through the comparison of the scenarios presented in this table, the economic benefits can be highlighted. Comparing the Independent and BAU scenarios, it is possible to verify the economic benefits obtained considering the CEC concept, where the benefits are approximately EUR 2.2 in the first DR event and EUR 1.33 in the second. With the All Flexibility scenario, the maximum energy reduction can be made, wherein at the end of both DR events, there is excess generation. Comparing this scenario with the BAU scenario, it can be seen that the profits of using all the flexibility provided by the end-users are about EUR 0.52 in the first event and EUR 1.1 in the second. Finally, comparing the BAU and Correction 2 scenarios, the economic benefits obtained with the DR events are verified, where in the first event they are EUR 0.42, and in the second they are EUR 0.89. With these two DR events, the total economic benefits are EUR 1.31.

Although achieving promising results, this study is susceptible to improvements. A future improvement is the consideration of shifting flexibility and the respective new metric to assess it. In this way, the DR model is less limited and enables better use of PV generation. Furthermore, to increase the efficiency of DR event monitoring, one other possible improvement is the implementation of a 10-minute-ahead forecast algorithm for the CEC’s energy resources.

7. Conclusions

This paper highlights a methodology based on the classification of the end-users of an energy community to perform demand response events. Through forecasting and clustering algorithms, the proposed methodology makes it possible to calculate the critical periods in which it is necessary to make corrections through demand response events. Additionally, the proposed methodology classifies the end-users, facilitating the choice of which end-users should be invited to participate in each event.

The methodology was tested in a community with 50 participants that use renewable photovoltaic generation. The case study considered includes two critical periods where energy balancing in the community could be performed considering the day-ahead forecasts. The results show that the respective critical periods were corrected during the monitoring of the demand response events. In the case of the demand response event at 12:00, it was found that there was a lack of flexibility since the event took place during lunchtime. In the case of the event at 14:00, it was found that end-users were able to provide higher energy flexibility.

Although promising, the methodology also enables the improvement by considering shiftable loads and other metrics. Furthermore, another improvement is the implementation of a 10-minute-ahead forecasting algorithm to track the demand response event and allow the invitation of new end-users could even improve the real-time balancing of citizen energy communities.