DSO-Aggregator Demand Response Cooperation Framework towards Reliable, Fair and Secure Flexibility Dispatch

Abstract

Unlocking flexibility on the demand side is a prerequisite for balancing supply and demand in distribution networks with high penetration levels of renewable energy sources that lead to high volatility in energy prices. The main means of fully gaining access to the untapped flexibility is the application of demand response (DR) schemes through aggregation. Notwithstanding, to extract the utmost of this potential, a combination of performance-, financial-, and technical-related parameters should be considered, a balance rarely identified in the state of the art. The contribution of this work lies in the introduction of a holistic DR framework that refines the DR-related strategies of the aggregator towards optimum flexibility dispatch, while facilitating its cooperation with the distribution system operator (DSO). The backbone of the proposed DR framework is a novel constrained-objective optimisation function which minimises the aggregator’s costs through optimal segmentation of customer groups based on fairness and reliability aspects, while maintaining the distribution balance of the grid. The proposed DR framework is evaluated on a modified IEEE 33-Bus radial distribution system where a real DR event is successfully executed. The flexibility of the most fair, reliable and profitable sources, identified by the developed optimisation function, is dispatched in an interoperable and secure manner without interrupting the normal operation of the distribution grid.

1. Introduction

Global energy prices have been steadily rising since mid-2021 as electricity demand spurred by the post-pandemic recovery fuelled significant tightness in the energy market. This “ripple effect” was particularly pronounced in Europe where Russia’s invasion of Ukraine added unprecedented pressure to the European energy market. With the Versailles Declaration agreed to in March 2022, the EU leaders of the 27 member states agreed to phase out the EU’s dependence on Russian fossil fuels as soon as possible. The challenge becomes even more difficult to overcome as the recent drive towards the unrestrained integration of intermittent low carbon energy sources, such as renewable energy sources (RES), can potentially jeopardize the security of supply as well as the economic operation of the power system. Innovative use of demand flexibility to meet power system needs can end natural gas and coal dependence according to the latest International Energy Agency initiative [1], while ensuring the alignment between renewable energy generation and demand. Exploiting the untapped flexibility of the demand side introduces a more pro-active and effective approach for energy transactions instead of investing in new non-renewable transmission-connected generation capacity. However, strict market and grid-related regulations exclude single small-scale electricity customers from participating in the provision of such services; thus, third parties, such as aggregators, must undertake the role of summing those multiple flexibility volumes. Aggregators are being lauded as critical entities in providing these valuable electricity services, acting as intermediaries between the small- and medium-scale consumers and the electricity market stakeholders at higher levels, such as the distribution system operators (DSOs) [2]. The most common approach for extracting these flexibility volumes is through demand response (DR), the scope of which has been expanded to also include system balance improvements for the distribution network [3].

Many recent studies focus on facilitating the role of aggregators into the distribution-level electricity market to improve market efficiency, while emphasizing the role of DR [4,5,6,7,8,9,10]. In this context, various DR frameworks can be found in the literature, such as the bi-level energy trading model presented in [11] that incorporates RES plants and DR aggregator (DRA) simultaneously. In this model, social welfare is maximized at the upper level, while the profit of DRA is maximized at the lower level. The authors of [12] propose a bilateral integrated DR model with load aggregator as the core in integrated energy systems (IES). Through this model, the aggregator can achieve a bilateral DR via a top-down chain of benefits, while the balance of the system as well as the interests of involved stakeholders are facilitated based on a hierarchical Stackelberg game approach. Similarly, a two-level optimization algorithm is presented in [13]. The first stage of the first-level optimization problem maximizes the day-ahead profit of DRAs and plug-in hybrid electric vehicle parking lot aggregators (EVPLAs). At the second stage of the first level problem, the DSO maximizes its net revenue for the day-ahead scheduling horizon, minimizing the energy not supplied costs. At the second level problem, the DSO maximizes its net revenue for the real-time scheduling horizon by minimizing the mismatch of control variables and the expected customer interruption costs (CICs). Uncertainty in flexibility provision is also addressed in [14], where a set of aggregators provide their flexibility to the DSO under a fair and incentive-compatible flexibility mechanism. A reliability rate was introduced in [15] towards identifying trustworthy customers for a specific DR target, whereas authors in [16] focus on the loss of load probability (LOLP) reliability index to identify and deliver incentive-based DR that will improve the overall nodal reliability by offering higher incentives to less reliable nodes.

In the near future, it is expected that DSOs will have a broader role as neutral market facilitators, offering equal opportunities to all aggregators to sell their services [17]. Observability of the distribution network will enable aggregators and DSOs to improve flexibility procurement for more economically efficient grid management and strengthen the balancing conditions of the distribution network [18]. Many literature approaches focus on the interactions between the DSOs and aggregators that aim to identify and resolve grid constraints, with the most important being the methodology specified in the USEF flexibility transfer protocol (UFTP) [19,20]. In this methodology, USEF addresses congestion management or grid-capacity management through congestion points that are published by the DSOs and exploited by the aggregators. A recent study [21] introduces a pricing mechanism where the DSO runs the local day-ahead market data, and the aggregators purchase energy based on the distribution locational marginal prices that depend upon grid conditions (losses/congestion). An interplay model for energy flexibility management through end-users, aggregators and DSO is proposed in [22]. The model presents a method where market players cannot make decisions independently without declaring their desired actions to the DSO first. Another coordination architecture in which an aggregator and distribution operator coordinate to avoid distribution network constraint violations, while preserving private information of both parties, is proposed in [23]. Alternatively, the authors of [24] suggest a methodology where the aggregators use representative data provided by the DSO to formulate a set of equations that characterize the magnitude of those network variables and are considered as key for safe network operation when DR allocation takes place.

Besides the exploitation of various optimisation functions, modern electricity frameworks necessitate the consideration of complementary factors, such as interoperability and security [25,26].

A number of publications have tackled the coordination of DR based on individual performance-related parameters assigned to the aggregator’s customers, while others addressed flexibility provision considering network operational constraints. However, a holistic framework that addresses a combination of performance-related parameters alongside cost and technical ones is completely overlooked. Our approach contrasts with that of prior work by coordinating a selection of customer performance and cost parameters that can affect the DR-related strategies of the aggregator and subsequently its revenues. In parallel, the proposed approach also considers network technical parameters that serve two purposes. The first one is to directly send DR signals to any local aggregator that is suitable for restoring a distribution grid operational issue. The second purpose is to protect the balance of the distribution grid from any operational issues that could be inadvertently caused due to the aggregator’s strategies. This is achieved through a newly proposed bi-level constrained-objective optimization function. The contribution of this work lies in the introduction of the aforementioned novel optimization function that goes a step further from the traditional approaches that consider performance, economic and technical parameters in an uncoordinated and uncorrelated manner and outputs the most reliable combination of customer groups that can offer their available flexibility volume in a fair and profitable fashion, while maintaining the stability of the grid at all times.

The introduction of the reliability aspect establishes that the aggregator participates in DR events in its full committed capacity, while the fairness aspect safeguards that the selection of customers is distributed among the whole portfolio of the aggregator. The first one minimizes the penalty costs of the aggregator as reliable customers are prioritized, while the latter ensures that all customers are going to be selected at some point, thus motivating them to remain under the aggregator ’s portfolio and subsequently increasing its portfolio. Both performance indices collectively aim to lead to increased revenues for any aggregator exploiting the proposed DR framework.

Those two introduced indices are combined with technical parameters related to the grid balancing conditions through power and voltage constraints. At the same time, following well-known open standards and employing blockchain technologies, the framework ensures interoperability, security and data integrity for all the involved parties. In the context of the presented work, a detailed case study that acts as a proof-of-concept that verifies the employed functionalities of the proposed DR framework is conducted. In summary, the proposed DR framework considers (i) customer performance, (ii) cost parameters and (iii) network technical parameter, while ensuring data security as well as interoperability.

The rest of the paper is structured as follows: Section 2 presents an overview of the proposed DR framework, including a detailed description of the two levels of the optimisation function as well as the horizontal complementary functionalities. The results of testing the proposed DR framework on a modified IEEE 33-bus radial distribution system are presented in Section 3. Important concluding remarks appear in Section 4 of the paper.

2. Methodology

In the problem of enabling optimal flexibility provision, a holistic DR framework that enables interoperable and secure DR activation for DSO-aggregator coordination is developed. The backbone of the proposed framework is a bi-level optimisation function that aims to minimize the aggregator’s costs while ensuring the normal operation of the distribution network through technical constraint evaluation.

A high-level overview of the functionalities employed by the proposed DR framework is illustrated in Figure 1. The DR framework is intended to operate at the individual aggregator level where distribution network observability is established. All available information is used as an input to derive a decision about the optimal combination of customers and their flexibility volume based on each DR signal and the activities of the aggregator in the electricity market. After a DR signal is initiated by the DSO, a preliminary check that the total flexibility volume of the aggregator can meet the total requested flexibility is performed. In that case, the optimisation function procedure runs. Otherwise, the DR signal is rejected. The two levels of the optimisation function utilised by the proposed DR framework simultaneously address both cost and customer performance parameters as well as the distribution network technical criteria. By doing so, not only lowers the risk associated with the DR customer selection, but also risk-averse bidding strategies, occurring due to various grid violations, are foreseen and avoided. The decision about the optimal combination of customers that can participate in the current DR signal is then fed as an output to the aggregator.

A DR signal activation ends with the flexibility extraction from the customers, followed by the flexibility provision to the DSO. As added-value, the proposed DR framework ensures communication interoperability as well as secure interaction between all the involved energy stakeholders through the exploitation of its horizontal complementary functionalities, the OpenADR standard [27] and blockchain technology.

Even though the focus of this work is the aggregator, other market players (e.g., utilities, flexibility traders, etc.) could also employ the framework. Moreover, the proposed DR framework, and subsequently the developed optimisation function, can be applied to any type of contracts (dynamic and/or static) between the DSO and aggregator, as well as between the aggregator and his customers, while the technical parameters utilised in the optimisation function enable the exploitation of the developed framework for any network topology.

An overview of the assumptions made as well as a detailed description of the two optimisation levels and the horizontal complementary functionalities are presented in the following sections.

2.1. DR Framework Assumptions

The proposed DR framework aims to optimise flexibility provision in an electricity landscape where a DSO-aggregator coordination mechanism is already established. Sharing information on network topology in real-world applications is not yet allowed. However, it is expected that the necessity for creating equal opportunities for all stakeholders to enter the electricity markets will render the distribution network topology observable to all aggregators. This is expected to follow the local flexibility market paradigm, where aggregators associated with the market provide an accessible level-playing field that allows all service providers to compete fairly to deliver flexibility in the most cost-effective manner. The visibility level will surely depend on the regulations of each country, while the aggregators will not have access to all the data flows and information, rather than the inputs and outputs related to their role. In this context, it is assumed that the DSO provides indirect access to the distribution network topology to the aggregator. To this end, both the DSO and aggregator must coordinate for safeguarding the balance of the distribution network in a manner where the DSO sends a direct signal to an aggregator to address a local congestion problem related to grid balance. The aggregator, who alleviates the problem through a flexibility provision, is compensated based on a direct bilateral contract price agreed with the DSO. Moreover, the proposed DR framework enables the aggregator to concurrently participate in other flexibility markets besides congestion management, while considering the balance of the distribution network coverage.

2.2. First Optimisation Level—Cost and Performance

To address the cost and customer performance variability in flexibility aggregation, the first level of the optimisation function utilised by the developed DR framework introduces two new indices: the fairness index (FI) and the reliability index (RI). The FI and RI are introduced for the first time within the concept of DR and flexibility aggregation and represent the equal distribution of DR signals to all customers as well as their reliability to flexibility commitment, respectively. The two proposed indices act as risk management mechanisms by prioritizing the group of customers that can reliably participate in a DR event by meeting the requested flexibility volume, while ensuring that the aggregator utilises all the customers within his portfolio. A fair distribution of flexibility requests to all the customers will enlarge the portfolio of the specific aggregator due to the increased willingness of other customers to enroll. Preventing potential penalties due to unsuccessful flexibility provision by prioritizing reliable customers, while enlarging the aggregator’s portfolio through fair distribution, leads to increased revenues for any aggregator exploiting the proposed DR framework. The two proposed indices are integrated in the first level of the optimisation function along with the typical cost and availability indices. The first one facilitates the minimization of the total cost of the aggregator, while the latter ensures that the selected customers are not scheduled to participate in the electricity market throughout the day; thus their available flexibility volume can be exploited. At this level, the optimisation function derives all the available possible combinations with which their aggregated flexibility volume can meet the total requested flexibility while resulting in a fair and reliable solution.

2.3. Second Optimisation Level—Technical

To maintain the balance of the distribution network, the proposed optimisation function considers flexibility aggregation, scheduling and disaggregation capabilities under the constraints of maintaining the normal operation of the distribution network at all times. This entails the identification of any voltage or line-loading issues, including time and specific location, occurring within the investigated network topology. In addition, through this second level of the optimisation function, the correct flexibility volume for maintaining the grid balance is estimated.

2.4. Optimisation Function Model Formulation

The proposed objective function considers the minimization of the total cost of the aggregator, constrained by the technical parameters of the distribution network that are obtained through optimal power flow (OPF) analysis.

Suppose that customer k can change his demand from d_k,0(t) [kWh] (initial value) to d_k(t) [kWh] during the tth hour where a DR event occurs based on the value which is considered for the incentive and the penalty included in the contract. Then, the change in the demand, or equally the estimated flexibility provided by each customer, is calculated using:

$$\Delta d_k\left(t\right)=|d_k\left(t\right)-d_{k,0}\left(t\right)|$$

(1)

If I(t) [€/kWh] is paid as incentive to the customer in the tth hour for each kWh flexibility, as part of the contract with the aggregator, then the total compensation of the customer for participating in DR signals will be as follows:

$$P(\Delta d_k\left(t\right))=I_k\left(t\right)·\Delta d_k\left(t\right)$$

(2)

If the customer who has been enrolled in the mentioned DR programs does not commit to his obligations according to the contract, he will be faced with a penalty. If the penalty price for inadequate flexibility provision is denoted by pen_k(t) [€/kWh], then the potential total penalty cost is equal to the difference between the requested flexibility for the current DR event, Δd_k(t), and the average flexibility volume (AvgFlex_k(t − 1)) that the customer k offered in all previous events (t − 1).

$$\begin{array}{cc}\hfill & PEN(\Delta d_k\left(t\right))=\hfill \\ \hfill & pe{n}_k\left(t\right)·\left[\Delta d_k\left(t\right)-AvgFle{x}_k(t-1)\right]\hfill \end{array}$$

(3)

In this case, the total revenue for the customers who participate in the DR is calculated as follows:

$$P(\Delta d_k\left(t\right))=I_k\left(t\right)·[d_{k,0}\left(t\right)-d_k\left(t\right)]-PEN(\Delta d_k\left(t\right))$$

(4)

In order to prioritise those who are reliable and offer the exact amount of requested flexibility on a regular basis, the RI which depends on the data recorded until the previous DR event (t − 1) is introduced and is estimated based on the following equation:

$$\begin{array}{cc}\hfill R{I}_k\left(t\right)=& R{I}_k(t-1)-\frac{ReqFle{x}_k(t-1)-\Delta d_k(t-1)}{TotalFlex(t-1)}\hfill \\ \hfill & +P{I}_k(t-1)·\frac{\Delta d_k(t-1)}{TotalFlex(t-1)}\hfill \end{array}$$

(5)

where ReqFlex_k(t − 1) [kWh] is the last requested flexibility volume, PI is a binary indicator used for identifying if the customer participated in the last DR event, while TotalFlex(t − 1) [kWh] is the total flexibility volume provided by all N customers for all past DR requests and can be estimated by:

$$TotalFlex(t-1)=\mathsf{\sum }_k^{N}ReqFle{x}_k(t-1)$$

(6)

The higher the RI index, the better reliability performance of the executed DR will be.

In order to evenly distribute DR requests among customers, an absolute fairness index (AFI) per customer is introduced, which is defined as the ratio of the total number of requests sent to customer k to the total number of requests for all customers.

$$AF{I}_k\left(t\right)=\frac{TotalRe{q}_k(t-1)}{{\mathsf{\sum }}_k^{N}TotalRe{q}_k(t-1)}$$

(7)

In addition to the AFI, a capacity fairness index (CFI) is considered in order to fairly assign the requested flexibility volume based on the maximum (MaxFlex) and minimum (MinFlex) flexibility capacity that each customer k can realistically provide and the average flexibility volume (AvgFlex) he has offered in all previous requests. This index aims to exploit the flexibility volume of each asset at its maximum offered capacity.

$$CF{I}_k\left(t\right)=1-\frac{MaxFle{x}_k-AvgFle{x}_k(t-1)}{MaxFle{x}_k-MinFle{x}_k}$$

(8)

All variables related to the DR participation of each customer k (i.e., ReqFlex, TotalReq, AvgFlex) are stored and updated for each time interval that the proposed DR framework is executed. The values of the maximum and minimum available flexibility are defined in the contract based on the deferrable loads of each customer. Both the AFI and CFI concern the fairness aspect of the developed optimisation function.

2.4.1. First Level Optimisation

Considering the above, the proposed optimisation function that aims to minimize the total cost of the aggregator by allocating all available assets based on total cost and reliability of his customers as well as a fair approach that will help the participants become more actively engaged can be defined as:

$$\begin{array}{cc}\hfill Optimisationweight=min& \\ \hfill \left\{\mathsf{\sum }_k^N\left(P(\Delta d_k\left(t\right))·\frac{1}{R{I}_k\left(t\right)}·\frac{1}{AF{I}_k\left(t\right)}·\frac{1}{CF{I}_k\left(t\right)}\right)\right\}\end{array}$$

(9)

The result of the optimisation function (Optimisation weight) is a value that represents the effect of each combination of customers on the aggregator’s costs. The lower the weight is, the lower the expected cost will be.

In order to achieve optimal DSO-aggregator coordination, several technical constraints must be considered. To this end, the developed optimisation function (9) is subject to constraints that ensure voltage as well as active and reactive power at both bus- and line- levels at all times. The variable that relates the optimisation function with the technical constraints is the available flexibility of customer k, Δd_k(t).

2.4.2. Second Level Optimisation

The bus-level active and reactive power balance are maintained through:

$$P{D}_i\left(t\right)-P{C}_i\left(t\right)+\mathsf{\sum }_{i^{\prime }}{P}_{i,i^{\prime }}\left(t\right)=0\forall i,i^{\prime }\in I,\forall t\in T$$

(10)

$$Q{D}_i\left(t\right)-Q{C}_i\left(t\right)+\mathsf{\sum }_{i^{\prime }}{Q}_{i,i^{\prime }}\left(t\right)=0\forall i,i^{\prime }\in I,\forall t\in T$$

(11)

The above constraints retain a balance between the active and reactive loads at bus i and time t [PD_i(t), QD_i(t)] with the respective changes that resulted due to the flexibility provision [PC_i(t), QC_i(t)]. The total active load, PD_i(t), at bus i is equal to the total consumption of all customers connected to that bus:

$$P{D}_i\left(t\right)=\mathsf{\sum }_k^N{d}_{k,i}\left(t\right)$$

(12)

while the total active power provision, PC_i(t), at bus i is equal to the total flexibility (upwards or downwards) provided by all customers connected to that bus:

$$P{C}_i\left(t\right)=\mathsf{\sum }_k^N\Delta d_{k,i}\left(t\right)$$

(13)

Active and reactive line flows are calculated as:

$$\begin{array}{c}\hfill P_{i,i^{\prime }}\left(t\right)=G_{i,i^{\prime }}{V}_i^2\left(t\right)+V_i\left(t\right)V_{i^{\prime }}\left(t\right)G_{i,i^{\prime }}cos[{\delta }_i\left(t\right)-{\delta }_{i^{\prime }}\left(t\right)]\\ \hfill +V_i\left(t\right)V_{i^{\prime }}\left(t\right)B_{i,i^{\prime }}sin[{\delta }_i\left(t\right)-{\delta }_{i^{\prime }}\left(t\right)]\\ \hfill \forall i,i^{\prime }\in I,\forall t\in T\end{array}$$

(14)

$$\begin{array}{c}\hfill Q_{i,i^{\prime }}\left(t\right)=-B_{i,i^{\prime }}{V}_i^2\left(t\right)+V_i\left(t\right)V_{i^{\prime }}\left(t\right)G_{i,i^{\prime }}sin[{\delta }_i\left(t\right)-{\delta }_{i^{\prime }}\left(t\right)]\\ \hfill -V_i\left(t\right)V_{i^{\prime }}\left(t\right)B_{i,i^{\prime }}cos[{\delta }_i\left(t\right)-{\delta }_{i^{\prime }}\left(t\right)]\\ \hfill \forall i,i^{\prime }\in I,\forall t\in T\end{array}$$

(15)

where G_i,i′ and B_i,i′ represent the real and imaginary parts, between the bus i and i′, of the respective element in the bus admittance matrix. The voltage magnitude and phase angle at bus i and time t are described by V_i^t and δ_i^t, respectively. The real and imaginary parts G_i,i′ and B_i,i′, as well as the voltage magnitude and phase angle at bus i, are estimated based on the inputs provided through the network topology.

In addition, the power factor at load points should remain constant when the load is curtailed or shifted:

$$P{D}_i\left(t\right)Q{C}_i\left(t\right)=Q{D}_i\left(t\right)P{C}_i\left(t\right)\forall i\in I,\forall t\in T$$

(16)

The bus voltage is one of the most essential and significant safety and service quality indices. In this case, the bus voltage limits are maintained through:

$$\underline{V}\le V_i\left(t\right)\le \overline{V}\forall i\in I,\forall t\in T$$

(17)

where V_i^t is the voltage magnitude of the ith bus, while $\underline{V}$ and $\overline{V}$ are the allowed lower and upper voltage magnitudes, respectively. All utilised voltage values are in p.u.

Line flow capacity limits are ensured as:

$$\overline{-S_{i,i^{\prime }}}\le S_{i,i^{\prime }}\left(t\right)\le \overline{S_{i,i^{\prime }}}\forall i,i^{\prime }\in I,\forall t\in T$$

(18)

where

$$S_{i,i^{\prime }}\left(t\right)=\sqrt{P_{i,i^{\prime }}^2\left(t\right)+Q_{i,i^{\prime }}^2\left(t\right)}\forall i,i^{\prime }\in I,\forall t\in T$$

(19)

while load change at each time is limited by the consumption load:

$$0\le P{C}_i\left(t\right)\le P{D}_i\left(t\right)\forall i\in I,\forall t\in T$$

(20)

The required flexibility for restoring the bus voltage to normal operating conditions is based on a voltage sensitivity analysis by performing a linearization of the system around the operational point that results from a load flow calculation. Linearizing the load flow equations around the actual operating point leads to the following equation system:

$$\left[\begin{array}{cc}{J}_{P\theta }& J_{P\upsilon }\\ J_{Q\theta }& J_{Q\upsilon }\end{array}\right]\left[\begin{array}{c}\partial \theta \\ \partial \upsilon \end{array}\right]=\left[\begin{array}{c}\partial P\\ \partial Q\end{array}\right]$$

(21)

The above equation system indicates that changes in the voltage magnitude ($\upsilon$), during the violation, and angle ($\theta$) due to small changes in the active (P) and reactive (Q) power can be directly calculated from the load bus Jacobian matrix (J_P$\theta$, J_P$\upsilon$, J_Q$\theta$, J_Q$\upsilon$). For example, if P is set to 0 (constant), the sensitivities of the type $\partial \upsilon /\partial Q$ are calculated using:

$$\partial \upsilon ={\tilde{J}}_{\mathit{Q}\upsilon }^{-1}\partial Q=S_{\upsilon \mathit{Q}}\partial Q$$

(22)

where

$${\tilde{J}}_{\mathit{Q}\upsilon }=-J_{\mathit{Q}\theta }{J}_{\mathit{P}\theta }^{-1}{J}_{\mathit{P}\upsilon }+J_{\mathit{Q}\upsilon }$$

(23)

The variation of voltage magnitude at each busbar can be described by a linear combination of small reactive power variations according to:

$$\partial {\upsilon }_i=S_{i1}\partial Q_1+\dots +=S_{in}\partial Q_n$$

(24)

In this case, the diagonal elements S_i1 of S represent the voltage variation at bus i due to a variation of the reactive power at the same point. The non-diagonal elements S_ij describe the voltage variation at busbar i due to the variation in reactive power at a different point in the network. Positive $\partial \upsilon /\partial Q$ sensitivity indicates stable operation of the investigated network. High sensitivity means that even small changes in reactive power cause large changes in the voltage magnitude, thus the more stable the system, the lower the sensitivity. High voltage sensitivities are indicative of weak areas of the network. By applying a modal transformation to (22), the $\partial \upsilon /\partial Q$ sensitivity can be expressed as an uncoupled system of the form:

$$\partial \tilde{\upsilon }=T^{-1}{S}_{\upsilon Q}T\partial \tilde{Q}={\tilde{S}}_{\upsilon Q}\partial \tilde{Q}$$

(25)

where ${\tilde{S}}_{\upsilon Q}$ is a diagonal matrix whose elements correspond to the eigenvalues of the sensitivity matrix $S_{\upsilon Q}$, while υ = T$\tilde{\upsilon }$ and $Q=T\tilde{Q}$. Therefore, the voltage variation at each mode (in the matrix derived from the modal transformation) depend only on the reactive power variation at the same mode:

$$\partial {\tilde{\upsilon }}_i={\lambda }_i\partial {\tilde{Q}}_i$$

(26)

The eigenvalues ${\lambda }_i$ provide the required information regarding the voltage balance of the network. If ${\lambda }_i$ is positive, the modal voltage increase and the modal reactive power variations are in the same direction; thus, the network is stable. The magnitude of the eigenvalue indicates how far or close one voltage mode is to instability. The right eigenvectors of S_$\upsilon$Q correspond to the matrix $T=[{\upsilon }_1\dots {\upsilon }_n]$, while $T^{-1}=[{\omega }_1^T\dots {\omega }_n^T]$ corresponds to the left eigenvectors matrix. The participation factor of bus k to mode i is defined by the product of the kth component of the left and right eigenvector of mode i:

$$P_{ik}={\omega }_{ik}{\upsilon }_{ik}$$

(27)

The participation factor gives an indication of the extent of the influence the variation of active power on a node has on voltage mode. To this end, to avoid a voltage violation event and maintain grid balance, the aggregated flexibility of bus i, PC_i(t) should be equal to the estimated active power P_ik.

In case line overloading occurs, then the total required flexibility for restoring the network’s normal operation is estimated by:

$$TotalFle{x}_{i,i^{\prime }}\left(t\right)=\frac{Violatio{n}_{i,i^{\prime }}\left(t\right)-100}{100}{P}_{i,i^{\prime }}\left(t\right)$$

(28)

where Violation_i,i′(t) is the load percentage of the line between the bus i and i′ and is calculated based on the network topology inputs. Subsequently, to avoid a line violation event, the aggregated flexibility of bus i, PC_i(t) should be equal to the TotalFlex_i,i′(t).

The nonlinear constrained optimisation function is solved based on sequential quadratic programming which is an iterative method used in mathematical problems and follows the procedure of solving a sequence of optimization subproblems, each of which optimizes a quadratic model of the objective subject to a linearization of the constraints. The outcome of the objective function is the optimal combination of customers along with their respective flexibility volume that can meet the total flexibility request with the minimum cost and without affecting the balance of the network.

2.5. Horizontal Complementary Functionalities

To further support the viability of the proposed methodology, two added-value functionalities have also been developed, towards presenting a semantically interoperable and secure framework. For the former, an ontology based on the OpenADR standard has been developed [28] for formal data validation and integration with other standards, whereas a communication component [29] that interconnects systems with heterogeneous communication protocols, formats and data models allows for a transparent exchange and consumption of data.

For the latter, a permissioned blockchain-based platform based on hyperledger fabric [30] is employed for ensuring a decentralized and trustworthy infrastructure. These entities participate in an authenticated, Byzantine fault-tolerant consensus algorithm, which is decentralized by design and provides for tamper-resilience and liveness in the presence of (arbitrary) failures. Moreover, to promote fully automated contractual agreements among participants of DR schemes in the context of different marketplaces in a trustworthy and verifiable fashion, we leverage the power and expressiveness of smart contracts. These are automated agents that “live” in the blockchain and play an integral part of the proposed DR framework [31] as they mediate and monitor transactions, provide transparency, as well as, enforcement of contractual clauses by regulating energy supply and payments and potentially incurring penalties.

3. Results and Discussion

3.1. Test Case Description

In this section, the performance of the proposed DR framework is evaluated based on a hybrid test network comprising a physical microgrid (MG) and nanogrid (NG) network connected to a simulated distribution network. The reason for creating this hybrid test network is to investigate the applicability and effectiveness of the proposed DR framework under real conditions where an MG is interacting with an NG, and their joint operation directly affects a nearby distribution network connected to the same primary substation. Both the MG and the NG are physical parts of the University of Cyprus (UCY) campus where full monitoring and control capabilities are enabled. The inability to control the nearby connected physical distribution network is addressed through the utilisation of a simulated IEEE 33-bus test system that is modified to represent the unavailable physical distribution network. The physical MG comprises 14 tertiary buildings that span a broad variety of typologies and uses, along with large shares of distributed energy resources (DERs), such as PVs. Similarly, the physical NG (PVTL NG) includes PVs, a battery energy storage system (BESS) and EVs. Even though the IEEE 33-bus test system is modified to include both domestic and commercial electricity customers, the network topology and line characteristics of the system remain the same. To consider the effect of RES integration in the distribution network, the domestic customers are equally divided to consumers and prosumers.

In order to be able to evaluate the impacts of both the physical and simulated parts of the hybrid test network in a unified environment at the same time, the topology of the MG, NG and modified IEEE 33-bus test system were modelled in a power system analysis software application, DIgSILENT. The modelled test network provided the additional ability of testing various distribution network balancing issues that otherwise would be impossible to physically create.

The characteristics for the MG and NG models are based on their physical counterparts, while the consumption and production datasets as well as BESS and EV profiles for the modelled MG and NG are fed in real-time to the models through the installed smart meters (SMs) across the UCY campus. Deferrable loads, such as chillers, dimming lights and smart AC split-units, that can be exploited as sources of flexibility for participating in the DR events are also considered and controlled in real time. The load profiles for the IEEE-bus test system were based on previous studies [32,33]. The modelled hybrid test network used for the evaluation of the proposed DR framework is illustrated in Figure 2. As can be seen in the figure, the test network consists of the primary substation, where two feeders (Feeder 1 and 2) are delivering electricity to the physical MG and NG as well as a third feeder (Feeder 3) that connects the modified IEEE 33-bus test system. At normal operating conditions, the line loading remains below 100% of the line capacity, while the voltage levels at the buses are maintained between 0.9 and 1.1 p.u. of the nominal voltage.

The integration of a Python programmed integrating script (PPIS) is developed in order to integrate the proposed DR framework, the test case and the interactions between the DSO and aggegator in one unified environment. In this way, any control strategy applied to the modelled microgrid and nanogrid components is carried out in their respective physical ones through the PPIS. This resembles a hardware-in-the-loop approach which enables the testing of the functionalities of the proposed DR framework in a semi-real environment where the embedded physical parts are capable of interacting with the simulated ones, thus rendering the evaluation results more accurate. In addition, the developed PPIS allows the demonstration of the interoperable and secure functionalities of the proposed DR framework.

3.2. Test Case Modelling Parameters and Assumptions

In this test case, it is assumed that the DSO takes the role of the price maker who compensates the aggregator at a contracted price for alleviating distribution grid violations in his area of responsibility. The contracted price between the DSO and the aggregator is based on a cost–benefit analysis (CBA) conducted by the national DSO, the Electricity Authority of Cyprus (EAC) [34].

Based on this CBA, the price that the DSO is willing to pay for each unit of flexibility energy [MWh] is related to the total flexibility energy units required for congestion avoidance. Following the CBA results, in this test case, it is assumed that the flexibility events can be divided into the categories depicted in

Table 1. Flexibility event categories.

Flexibility Level	Feeder Congestion (of Nominal Capacity)	Occurrence Frequency	Price (EUR/MWh)
Critical Flexibility	120%	10%	157.99
Normal Flexibility	105–119%	40%	110.67
Non-critical Flexibility	95–104%	50%	94.54

.

In this test case, it is also assumed that the aggregator is a price taker with respect to the DSO, but by contrast a price maker with respect to the flexibility price he offers to his customers. The aggregator’s business model, of course, is based on sharing a percentage of the achieved savings from the optimized portfolio with the participating customers. However, to persuade a customer to participate in flexibility programmes that will affect his thermal or visual comfort levels, an attractive incentive must be offered. Hence, it is expected that the earnings for the provider of the flexibility (customer) will be higher than the aggregator’s. In this respect, it is assumed that the aggregator will compensate his customers with a percentage between 60 and 90% of the flexibility price offered by the DSO for each successful DR activation. The sharing percentage level that the aggregator offers to his customers is assumed to vary based on the maximum flexibility capacity, duration and number of DR requests. Therefore, it is assumed that customers who can provide flexibility for long periods of time will be compensated less (lower sharing portion) than the ones who can provide flexibility for short periods.

Non- or insufficient delivery may result in a penalty. Penalty calculations need to be differentiated depending on the market and the risk posed. In this study, a penalty equal to one-sixth of the contractual fee is assumed. Considering the aforementioned assumptions, a flexibility price and the respective penalty is assigned to each customer/asset (building or facility) of the physical microgrid and nanogrid based on their availability periods (max duration and frequency) as well as the maximum flexibility capacity.

3.3. Test Case Scenario and Results

In order to verify the integrated functionalities of the proposed DR framework, a real possible scenario for flexibility provision is investigated. More specifically in this scenario, a flexibility request is initiated from the national DSO, the EAC, due to a congestion problem occurring within the area of the UCY campus. The role of the aggregator in the investigated scenario is undertaken by the UCY, where the various facilities and buildings located within the physical microgrid and the nanogrid are considered to be the DR customers. Each customer is represented by the available flexibility (either static or range based on the flexibility source) and the compensation price for the flexibility provision. In this investigated scenario, a virtual congestion problem is created by increasing the electricity demand of two simulated buildings implemented in the modelled test network. The two simulated buildings represent the physical library and residential building blocks located in the microgrid network. This scenario is practically possible as a congestion problem could arise due to a potential electricity demand increase of the library and residential building blocks that typically appears during the mid-day hours, where students return to their dorms or visit the library facilities during lecture breaks. The increased demand of those two buildings will overload the line of Feeder 2 to which those buildings are connected. As shown in Figure 3, a line loading violation occurs at the second feeder of the microgrid between 14:15 and 14:30. The line loading rises to 106.09% and 105.69%, at 14:15 and 14:30, respectively. These line violation incidents fall under the category of congestion problems in the distribution network and must be addressed locally through flexibility provision.

Following the proposed DR framework, a DR request is initiated by the DSO (EAC) to the local aggregator (UCY). Both violation levels correspond to a normal flexibility event. The proposed DR framework identifies the available and applicable customers who can participate during the specific time of the DR event. Only the assets connected to the second feeder can effectively contribute in this particular DR event, as it is a local congestion problem.

An overview of the associated assets, including the available flexibility volume, the contracted prices and the performance indices, is presented in

Table 2. Overview of assets associated with the local congestion event.

						Performance Indices (%)
Asset ID	Normal Flexibility Price [€/kWh]	Penalty [EUR/kWh]	Minimum Flexibility Volume [kWh]	Maximum Flexibility Volume [kWh]	Average Flexibility Volume [kWh]	Reliability	Absolute Fairness	Capacity Fairness
121CA	0.0926	0.0154	28	32	32	0.55	0.77	0.82
122CA	0.0760	0.0127	25	30	29	0.67	0.62	0.72
123CA	0.0841	0.0147	41	45	43	0.73	0.69	0.73
124CA	0.0777	0.0130	38	44	42	0.81	0.73	0.69
124CB	0.1013	0.0169	52	59	58	0.59	0.81	0.53
125CA	0.0976	0.0163	25	27	26	0.66	0.59	0.52
126CA	0.0768	0.0128	33	37	33	0.71	0.79	0.69
127CA	0.0890	0.0148	28	28	28	0.68	0.83	0.72

. As already indicated, the minimum and maximum flexibility volume is defined in each contract, while the average flexibility volume and the performance indices are estimated based on historical DR events participation.

The minimum required flexibility for restoring the line loading below the nominal level while maintaining the distribution network balance is estimated to be 138 kWh for the whole period of the violation. The outcome of the bi-level optimisation function is the optimal combination (minimum optimisation weight) of assets (customers) accompanied by the individual flexibility volume that each asset must provide. The aggregated value of all individual flexibility volumes is equal to the total required flexibility. For comparison reasons,

Table 3. Combination of aggregator assets.

No.	Combination of Assets	Flexibility Volume per Asset [kWh]	Total Cost [EUR]	Optimisation Weight
1	122CA, 124CA, 126CA, 127CA	29, 44, 37, 28	10.95	27.3924
2	121CA, 124CA, 126CA, 127CA	29, 44, 37, 28	11.43	27.5839
3	122CA, 123CA, 124CA, 126CA	25, 41, 38, 34	10.91	27.9267
4	121CA, 122CA, 124CA, 126CA	28, 29, 44, 37	11.05	28.4938
5	122CA, 123CA, 126CA, 127CA	28, 45, 37, 28	11.24	28.8458

shows the first five out of a total of thirty different combinations of assets that can meet the requested flexibility volume while satisfying the grid constraints. As depicted in the table, even though the third combination is the most profitable for the aggregator, as it would cost the least (EUR 10.91) for triggering, the results of the optimisation function demonstrate that the first combination of assets (122CA, 124CA, 126CA, 127CA) is the optimum selection as it would result in a more reliable and fair option, while the cost for triggering is marginally (EUR 10.95) higher than the most profitable option.

As can been seen in Figure 1, every transaction between the proposed framework and the external stakeholders (i.e., DSO and aggregator) is based on the OpenADR standard and is issued to the blockchain, establishing interoperability, security and integrity. More specifically, after the identification of the optimal solution, the aggregator proceeds to the extraction of the flexibility from the selected customers. Based on the proposed DR framework, this transaction is issued to the blockchain. The issuance of a DR request and its successful delivery as well as all the executed transactions are verified using a certified blockchain environment, the hyperledger blockchain explorer [35] tool as depicted in Figure 4.

As shown in Figure 4, a transaction is defined by a coded ID, a validation code and its payload hash. Those are followed by the creator and endorser of the flexibility request, in this case the UCY, which takes the role of the aggregator. The read set portion of the read–write set is used for checking the validity of a transaction, while the write set portion of the read–write set is used for updating the versions and the values of the affected keys.

The DR request from the aggregator (vtnID) is directed towards the “Energy Center 3” customer (targetID). This information is included as part of Write Key #5 along with the flexibility extraction signal of −28,000 W (value) which is requested by the aggregator for the specified 30 min period (startTime, endTime). Finally, the payload encodes a reward, which is equal to compensation, assuming that the “Energy Center 3” customer successfully dispatches the requested amount of flexibility over the DR signal’s active period. Following the issuance of a DR request and upon its successful delivery, the status of the previously issued DR request transitions to an active status. The proposed DR framework concludes when the aggregator, after the end of the request’s active period, issues a completion transaction, which is also stored on the blockchain. Besides the status of the DR request that transitions to a completed status, the “Energy Center 3” is compensated as indicated in the initial payload of the request. The transactions between the rest of the selected customers as well as the flexibility provision to the DSO is executed in a similar manner. The requested flexibility is physically extracted by the available deferrable loads of all the selected customers through hardware commands originated by the PPIS. The real consumption alteration due to the flexibility provision is measured by the SMs installed at each building and is fed back to the test environment in order to verify that the operation of the proposed DR framework restored the grid back to normal operating conditions. As shown in Figure 5, the line loading of all three feeders is below the nominal limit, highlighting the successful completion of the DR event, where the overloading violation at Feeder 2 is recovered and the balance of the whole network is maintained.

3.4. Computational Performance Evaluation

In order to gain helpful insights on the performance of the proposed DR framework when applied in a real-life electricity market, a computational performance evaluation was undertaken. This evaluation focuses on the runtime of the proposed DR framework in an attempt to identify any potential bottlenecks related to the hardware used.

The computational performance evaluation is based on a 64-bit Windows 10 Professional operating system with an Intel Xeon E5-2650 v.4 CPU and 16 GB RAM. The CPU is clocked at 2.20 GHz. The modified IEEE 33-bus test system shown in Figure 2 was used for the performance evaluation. The total number of busbars (low and medium voltage) and assets (consumption, production, storage) used in the investigated network are summarized in

Table 4. Number of buses used in the investigated model.

	Low Voltage	Medium Voltage
IEEE 33	33	-
UCY micorgrid	13	13
UCY nanogrid	10	-
Total	56	13	69

and

Table 5. Total number of assets used in the investigated model.

	Consumption	Production	Storage
IEEE 33	17	16
UCY micorgrid	16	16
UCY nanogrid	12	2	1
Total	45	34	1	80

, respectively. The size of the dataset comprising the energy profiles of the investigated distribution network is relatively small (0.063 MB), while the Internet connection bandwidth is very high at 1 Gbps, meaning that there is no lag in the communication between the server and the equipment.

The evaluation is separated into seven scenarios, where the number of available assets that can participate in flexibility provision changes. The maximum number of available assets is limited to 8, as this is the maximum number of assets connected to a single feeder or busbar in the investigated distribution network. Each scenario is then divided into 3 sub-scenarios where the flexibility volume range offered by each asset changes between 2, 5 and 10 kWh. The evaluation considers all functions executed between the origination of a DR signal until its distribution to the final end-users/assets. The performance evaluation results are shown in the two following figures. The runtime tendency of the algorithm as the number of available assets and their flexibility volume range increases is exhibited in Figure 6. For each one the three sub-scenarios, the linear as well as the exponential trend line projection are added as a reference point for comparison. The runtime as a function of the flexibility volume for the scenarios where 2, 5 and 8 assets are available for flexibility provision is shown in Figure 7. As expected, the runtime drastically increases with the increase of the investigated possible combinations that lead to the optimal solution. More specifically, for the first 2 sub-scenarios where the flexibility volume is tested at 2 and 5 kWh, it can be seen that the increase of the runtime is almost similar to the linear trend line projection. On the contrary, the runtime for the third sub-scenario, where the flexibility volume range per asset is 10 kWh, increases and gradually approaches the exponential trend line projection. The same conclusions can be derived by estimating the slopes of each curve. As shown in Figure 6, the slope is 29.653, 63.476 and 121.19 for the 2, 5 and 10 kWh flexibility volume, respectively. The higher positive slope for the third sub-scenario verifies the steeper upward tilt to the curve, meaning that as the number of assets and the flexibility volume increases, there are higher computational requirements.

It is important to note that the results are indicative and concern the virtual machine and distribution network used for this performance evaluation. The specifications of the hardware used in this evaluation are low, leading to the very high runtime of approximately 15 min for the worst-case scenario (8 assets are available, and each asset can offer up to 10 kWh of flexibility). It is obvious that in a real-life electricity market environment where assets are requested to participate in balancing the market, this level of runtime is prohibitive. However, this can be alleviated by utilising hardware with much higher specifications. This performance evaluation was undertaken to assess the runtime as a function of the different factors, such as the power network characteristics, the available assets that can participate in an upcoming DR event and the flexibility volume that each asset can offer. For more credible results, benchmarking should be performed on various operating systems and hardware to properly identify the impact of higher-spec systems on the performance of the proposed DR framework.

4. Conclusions

The recent electricity market crisis provides a glimpse of a future where a low-carbon electricity system that is not properly managed or stress-tested against scarcity and volatility could result in recurrent electricity price surges and impede the decarbonisation trajectory. The facilitation of demand response (DR) and the introduction of aggregators can restrain price spikes and ensure energy security but at the same time can ultimately change how the distribution system operators (DSOs) manage their grids. In this paper, the authors present a novel DR framework for DSO-aggregator coordination that introduces a new interoperable and secure cooperation layer that also utilises a constrained-objective optimisation function considering technical and energy market constraints to refine the DR-related strategies of the aggregator towards optimum flexibility provision. The performance of the proposed DR framework is evaluated based on a hybrid test network, where a real possible scenario of a line overloading problem is investigated. The results highlighted that the proposed DR framework selects the optimal combinations of assets in terms of profitability, reliability and fairness while restoring the balance of the distribution network. The holistic approach followed by the proposed DR framework is showcased through the deployment of its OpenADR-inspired blockchain functionalities for all transactions in the investigated scenario, offering on top of everything else secure interoperability. The results of the investigated scenario act as a proof-of-concept that demonstrates the holistic functionality of the proposed DR framework. Investigation of additional scenarios for showcasing the interaction of the DR framework with the electricity market will be addressed in future work. The proposed DR framework can be seen as a key for enhancing the DSO-aggregator coordination as well as a pathway for facilitating the role of the aggregator in a fully liberalized electricity market.