A Decentralized Architecture Based on Cooperative Dynamic Agents for Online Voltage Regulation in Smart Grids

The large-scale integration of renewable power generators in power grids may cause complex technical issues, which could hinder their hosting capacity. In this context, the mitigation of the grid voltage fluctuations represents one of the main issues to address. Although different control paradigms, based on both local and global computing, could be deployed for online voltage regulation in active power networks, the identification of the most effective approach, which is influenced by the available computing resources, and the required control performance, is still an open problem. To face this issue, in this paper, the mathematical backbone, the expected performance, and the architectural requirements of a novel decentralized control paradigm based on dynamic agents are analyzed. Detailed simulation results obtained in a realistic case study are presented and discussed to prove the effectiveness and the robustness of the proposed method.


Introduction
The intermittent power profiles generated by renewable power generators in power grids considerably perturb the bus voltage magnitude [1][2][3], which may go outside the allowable admissible range, especially during critical operating conditions (i.e., high generation and low load demand [3][4][5]).These events are not infrequent in existing power networks, which have been traditionally designed by assuming the passivity of all system buses, without considering the presence of distributed and dispersed generators [6].Hence, increasing of renewable power generators in these networks, driven by the modern sustainable environmental policies, is causing severe and complex phenomena that need to be carefully addressed [7][8][9].In this context, the research for new and more advanced online voltage control systems, aimed at regulating the reactive power injected/absorbed by distributed generators, represents one of the most promising research directions for smart power grids [10][11][12].To address this issue, the use of centralized architectures, traditionally used for voltage control in power systems, could not be suitable in short-term scenarios [13], since it asks for a significant upgrade of the communication and computing resources, to effectively solve constrained optimal power flow problems [14].Moreover, the deployment of reliable state estimation algorithms, which is a prerequisite of optimal power flow-based regulation techniques, is still an open problem due to the limited number of installed sensors [15].This has stimulated the research for alternative control techniques for the coordination of the reactive power injected/absorbed by distributed generation units, which process only local data [16][17][18].Recent experimental results have demonstrated that these local control techniques could be effectively used in existing distribution networks, but they may not provide sufficiently accurate results, especially in the presence of many renewable power generators [11].To solve this problem, in [19][20][21] the voltage regulation problem is solved by a centralized computing framework, which collects and processes the sensor data streaming, and sends back the corresponding set-points to the voltage controllers.Although the results obtained by this hierarchical voltage control paradigm are highly accurate, a reliable and pervasive communication system is required to connect all the sensor/controllers to the central processing system, which makes the entire control architecture extremely vulnerable.Furthermore, this method asks for an accurate model of the power system and a reliable estimation of the load/generation patterns, which are highly unpredictable, due to rapidly changing operating conditions [22].Moreover, as recently outlined in many research works, these kinds of centralized control architectures are characterized by low scalability levels, since an increased grid complexity could ask for unaffordable computing resources, further hardware redundancies, higher communication bandwidth, and larger data storage resources [23].All these limitations could hinder the application of centralized control architectures in modern power systems, where the constant growth of grid complexity and the need for a massive pervasion of renewable power generators ask for more scalable, and more flexible control architectures.In this framework, the deployment of decentralized architectures based on cooperative controllers, which infer global information about the actual power system operation by exchanging and processing only local data, has been recognized as one the most promising enabling technology for solving the voltage control problem.The concept is to try keeping the bus voltage magnitudes very close to the nominal value by adjusting the reactive power generated by the distributed generators, without asking for sophisticated communication hardware and computational resources.In particular, in [24,25] a decentralized technique for voltage regulation based on mutually coupled oscillators has been proposed.The main idea is to couple each grid sensor to a first-order oscillator equipped with decentralized consensus protocols, which converge to the global variables characterizing the actual power system operation.Thanks to this paradigm, the distributed voltage controllers can infer the global variables without the need for a fusion center, which collects and processes all the sensor data.In [26], a two-stage control technique for decentralized voltage regulation in active networks has been proposed.During the first stage, all the local controllers adjust the voltage at the monitored bus, by only processing local data; in the second stage, a proper coordination strategy is activated to properly balance the reactive power absorbed/injected by each controller.A similar approach is proposed in [19], where two control techniques are conceived, to avoid excessive reactive power absorption/injection by each distributed controller.In particular, the first technique is based on the decentralized estimation of the average reactive power generated by all the controllers.The second approach is based on the estimation of the average current that controllers inject at the point of common coupling, which is then used to define the optimal set of each controller.More recently, new techniques based on multi-agent systems (MASs) have been proposed for decentralized voltage regulation.In particular, in [27] a MAS-based decentralized technique for voltage control in distribution networks, and an incentive mechanism aimed at stimulating renewable power generators to support the grid voltage are designed.The main idea is to allow the local control agents to compute the voltage sensitivities by cooperating only with their neighborhood, without the need for an arbitration agent, which collect all the agent's measurements.Starting from these sensitivity coefficients, each agent identifies its local voltage control strategy by maximizing its own profit.An alternative decentralized approach to compute the voltage sensitivities, which is based on the data surface fitting technique, has been proposed in [28].This technique requires the knowledge of the network characteristics, and it is robust to changes in network parameters.The decentralized optimization of the local voltage control strategy is obtained in [29] by employing a computing paradigm, which aims at minimizing a quadratic voltage mismatch error objective using gradient-projection updates.To solve this problem two dynamic scenarios have been considered, which include an asynchronous scheme for the decentralized parameters update, and a time-varying communication scheme for highly variable network operation states.A similar solution approach has been proposed in [30] for voltage control of radial distribution grids with photovoltaic generators operating in voltage support mode, and distributed storage systems.In this paper, the voltage optimization problem has been led to the design of a decentralized disturbance-feedback controller, which minimizes the expected value of a convex quadratic cost function, subject to robust convex quadratic constraints on the system state and input.This decentralized problem has been solved by deriving an inner approximation, which enables the efficient computation of an affine control policy via the solution of a finite-dimensional conic program.Although these techniques show their potential of decentralized architectures in solving the online voltage regulation, their performance, in terms of grid voltage magnitude deviations, may be not satisfactory [31].This limitation mainly stems from the fact that the computed control action is not based on a real picture of the system operating condition.Hence, new and more effective decentralized voltage control architectures should be looked for.In trying to address these problems, this paper proposes a novel decentralized control architecture for the online voltage regulation in active power grids.The proposed solution leverages the mathematical framework of consensus/synchronization control theory in MASs.MASs consist of groups (i.e., ensemble) of dynamical systems exchanging their information and interacting with each other through wireless/wired communication networks, to agree, for example, upon a certain quantity of interest.Many real systems in nature and human society can be modeled as MAS and many researchers, inspired by natural occurrence of flocking and formation forming, have focused their work on synchronization, consensus and coordination of these latter [32], i.e., in controlling the whole network in order to produce a common behavior by applying distributed algorithms and to guarantee a smart group behavior.Examples in engineering deal with the coordinated motion of autonomous vehicles [33][34][35], the phase or frequency synchronization in large power grids [36], and the synchronization of wireless sensor networks [37].
By leveraging this paradigm, an electrical grid can be controlled by exploiting N cooperative smart agents that, exchanging their state information through communication networks, impose a common voltage magnitude value to the whole electrical grid.Specifically, each smart controller uses the information received by neighbors, via a communication interface, to locally control the voltage magnitude of the monitored bus, while aiming at achieving a synchronization behavior to desired voltage magnitude, as imposed by the network generator without the need for fusion data center.This allows the smart controller to locally decide how much reactive power it needs to inject, or absorb, to reach the desired voltage asset for the whole power grid.
The current literature exploits decentralized control strategies only for the control of the generators (see the survey [31] and the references herein).Our approach, differently, aims to guarantee that all the buses, both generation and load, synchronize to the common reference imposed by the generators, while reducing power losses.The proposed approach hence provides a self-organized power grid, which has the ability to cope effectively with the problems that might occur using only local interactions and providing "plug and play" capability.A case study developed on the IEEE 30-bus network confirm the effectiveness of the approach, and shows its robustness regarding electrical load variations.
Finally, it is worth mentioning that the large-scale deployment of the proposed control paradigm asks for the conceptualization of new tools aimed at facilitating operational data acquisition and handling in inter-operable formats.In this domain the current research activities are oriented in conceptualizing advanced computing paradigms aimed at handling complex systems by information semantics [38].The main idea is to develop a power system ontology able to adapt to different use-cases, which could be used to define and specify complex events and actions that run on an event processing engine.To this aim the Authors are developing a distributed and cooperative information framework aimed at exploiting the semantic representation of power system measurements for transparently exchanging data and information between the local voltage controllers, and with the Energy Management Systems of the Distributed System Operator.The proposed framework is based on specific software components, which support decentralized semantic sensor data exchanging and a distributed middleware aimed at processing massive and heterogeneous sensor measurements.The details of this framework will be presented in a future works.
The paper is organized as follows.In Section 2 the problem is formulated, while in Section 3 the proposed decentralized solution is analyzed.Section 4 is devoted to a case study; concluding remarks are presented in Section 5.

Problem Formulation
The online voltage control function identifies, for each power system state Γ, the set-point y of the grid controllers that minimizes an objective function J subject to several equality and inequality constraints g(Γ, y).
This problem can be formalized by the following constrained optimization problem: where is the control vector, whose components are the grid controller set-points, Q dg,i is the reactive power injected by the i-th (i = 1, . . ., N g ) distributed generator available for the regulation; Q cap,j is the vector of the reactive power injected by the j-th (j = 1, . . ., N c ) capacitor bank and m is the tap position of the HV/MV line tap changing transformer.Please note that the control vector y takes value in the solution space Ω: The vector function g(Γ, y) describes the problem constraints in terms of allowable ranges for the bus voltage magnitudes (i.e., V min,q ≤ V q ≤ V max,q , q = 1, • • • , N), and maximum allowable currents for the n l power lines (i.e., The objective function could take into account both technical and economic aspects, and it is typically expressed as a weighted sum of O normalized design objectives: The typical design objectives that should be minimized are: • the active power losses: where P g and P l are the total active power generated and absorbed on the network; • the average voltage deviation: where V q and V q are the current and the desired voltage at the node q respectively, and N is the number of nodes; • the maximum voltage deviation: Since the design objectives are competing, the voltage control problem has no unique solution and a suitable trade-off between objectives must be identified.

A Decentralized Solution of the Optimal Voltage Regulation Problem
To address the voltage regulation problem, an innovative solution based on a decentralized architecture is discussed here.This architecture is based on a network of N cooperative smart controllers, each regulating the voltage magnitude of a specific bus (called node too).
In this operating scenario, each controller device is equipped with three basic components: 1. a set of sensors measuring the available set of local electrical variables (i.e., voltage magnitude, active and reactive bus power); 2. a dynamical system, whose state is initialized by sensor measurements and evolves interactively with the states of nearby controllers, according to a properly designed distributed control strategy; 3. a communication interface, carrying the interaction among controllers by transmitting the state of the dynamical system and receiving the state transmitted by the other nodes.
The idea is to leverage the theoretical framework of multi-agent dynamical systems [39] to design and implement distributed cooperative control strategies allowing the optimal energy management of the whole power grid.To this aim, the N cooperative controller devices are modeled as a one-dimensional network of dynamical agents, in which each agent uses only its neighboring information to locally control the voltage magnitude of the bus, while aiming to achieve certain global coordination with all other agents.This mathematical framework is represented in Figure 1 as the composition of the following main interrelated components: (a) agent dynamics that describe the dynamics of each bus controller device; (b) communication topology, which indicates how and if a controller device obtains information about other agents, depending on the presence/absence of connecting lines; (c) distributed control action, which is implemented at the single-controller device level, and depends on both the state variables of the bus controller itself, and on information received from neighboring controller devices through the communication topology.Use of this paradigm allows the voltage controllers to assess, in a totally decentralized way, many important variables characterizing the actual operation of the grid.Thanks to this feature, each controller knows both the variables characterizing the monitored bus (sensed by in-built sensors) and the global variables describing the actual performance of the entire power grid (assessed by checking the state of the dynamical system).This allows each controller to (i) assess the evolution of the objective function describing the voltage regulation objectives and (ii) identify the proper control actions aimed at minimizing this function.From this perspective, a power grid made of N c capacitor banks, N g generators and N = N g + N c buses can be managed by N cooperative smart controllers.Specifically, if the i-th smart controller is associated with the i-th generation bus (i = 1, • • • , N g ) of the grid, then it aims at regulating the i-th bus voltage magnitude so to achieve the desired voltage V * i .Conversely, if the j-th smart controller is associated with the j-th capacitor bank bus (j = 1, • • • , N c ), then it aims at controlling, via distributed cooperative algorithm, the j-th bus reactive power generation capability in order to: (i) guarantee that its voltage magnitude achieves the desired optimal voltage value (according to ( 6)), as imposed by the N g generators within the electric grid; (ii) reduce power losses (according to ( 5)).

Agent Dynamics
Within our theoretical framework, each smart device j (j = 1, • • • , N c ) for the j-th capacitor bank bus of the power grid is described by the following dynamical system: where Q j (t) [p.u.] represents the reactive power of the j-th capacitor bank bus; u j (t) is the cooperatively distributed control input that drives the reactive power, and hence the voltage magnitude, of the electrical node.It is evaluated by exploiting both local electrical measurements and electrical networks information.Conversely, we assume that each smart device i (i = 1, • • • , N g ) for the i-th generation bus within the electrical grid is described by the following dynamical system: where V i (t) [p.u.] represents the voltage magnitude of the i-th generator; u i (t) is the control input that drives the voltage magnitude of the electrical node so the achieve the desired voltage V * i .Please note that the i-th generator acts as a leader for the whole SG by imposing the reference voltage magnitude V i (t) for the capacitor bank buses.

Communication Topology
The communication topology indicates how and if a smart controller q (q = 1, • • • , N, with N = N g + N c ) obtains information about the other smart devices p (p = 1, • • • , N, with q = p).
The connections among the N c cooperative smart controller for the capacitor bank buses can be modeled as a directed graph (digraph) G N c = (V, E , A) of order N c characterized by the set of nodes V = {1, . . ., N c } and the set of edges E ⊆ V × V.The topology of the graph is associated with an adjacency matrix with non-negative elements A = α jρ N×N , being ρ = 1, • • • , N c .In what follows, we assume α jρ = 1 in the presence of a communication link from the smart device j to device ρ, otherwise α jρ = 0.Moreover, α jj = 0 (i.e., self-edges (j, j) are not allowed).The presence of edge (j, ρ) ∈ E means that device j can obtain information from the device ρ, but not necessarily viceversa.
The presence/absence of connections among the N c cooperative smart controller and the N g smart controller for the generation buses is instead described by the matrix A 1 = α ji N c ×N g , whose elements α ji = 1 in the presence of a communication link among the smart device j and the device i, otherwise α ji = 0. Finally, we highlight that in our application we assume that the pairs (j, ρ) and (j, i) can communicate if there exist a power transmission line among them.

Control Design
The voltage regulation control problem for the power grid can be solved by achieving two control objectives, namely: 1. designing the control strategy u i (t) in ( 9) , leveraging local electric information, to opportunely manage the voltage magnitude of the bus i so to reach and maintain the desired reference voltage value V * i , i.e., lim ) the voltage magnitude of the i-th electrical node; 2. designing the distributed control input u j (t) in ( 8), leveraging both local and neighboring electrical information, to opportunely manage the reactive power of the bus j, hence updating its voltage magnitude V j (t) until it converges to the common reference behavior imposed by generators within the SG, i.e., lim t→∞ ∑ being V i (t) the voltage magnitude of the i-th generation bus and V ρ (t) the voltage magnitude of the neighboring nodes ρ To attain the control goal in (10), we consider, according to the literature (see e.g., [40]), for each electric node i the following Proportional controller: where k i is the proportional gain to be properly tuned according to the maximum admissible voltage variations for the i-th node.Conversely, to attain the control objective in (11), we propose, for each electric node j the following consensus-based control protocol that updates its action based on the errors among the electrical state information: where α jρ models the presence/absence of communication link among the bus j and the bus ρ; α ji models the presence/absence of communication link among the bus j and the generator bus i; k j and b j are the control gains to be tuned so to guarantee that the reactive power of the bus j does not exceed a pre-fixed operating range [Q j,min ; Q j,max ].
Finally, we highlight that the exploitation of the distributed cooperative control input (13) guarantees that all the N c buses within the SG closely converge to the common behavior imposed by the N g generators, hence guaranteeing the optimal management of the electrical grid.

Case Study
To validate the effectiveness of the approach proposed in Section 3, we consider here the voltage regulation problem for the IEEE 30-bus test system depicted in Figure 2. The power grid is made of N g = 6 generators (i.e., node 1, 2, 5, 8, 11, 13), N c = 24 capacitor banks, N = 30 buses, and n l = 41 lines.Load information, lines impedance, as well as reactive power limits, are reported in [41].Numerical analysis has been carried out by exploiting the MATLAB c platform.
The initial condition for the N cooperative agents within the grid as well as the selected control gains for both the control protocol in (12) and the one in (13) are listed in Table 1.Finally, the desired voltage value V * i for the generation buses N g are selected as follows:   Results in Figures 3-5 show the effectiveness of the proposed control approach in ensuring the control goal in (10) and (11).Indeed, as depicted in Figure 3, thanks to the control action (12), the smart controllers for the generation buses ensure that the corresponding voltage magnitude converge to the desired values V * i in 1 [s].Accordingly, the smart controller device j (j ∈ N c ) dynamically acts on the reactive power generation capability of the j-th electrical node (i.e., it produces or absorbs the bus reactive power; see Figure 5) and due to cooperative control action in (13) guarantees that its voltage magnitude converges in 1 [s] to the desired optimal value [1.02; 1.05] [p.u.] as imposed by the N g generators of the electrical grid.Indeed, the mean grid voltage of the N c electrical nodes, i.e., V mean = 1.03 [p.u.] is equal to the mean voltage imposed by the N g generators.This confirms the benefit of the proposed decentralized approach in ensuring the electrical grid synchronization to the reference behavior imposed by generators only leveraging local and neighboring information and without the need for knowing global information about the whole grid.The proposed approach hence provides a distributed control solution allowing the power grid to be self-organized.The benefits of the approach are the scalability of the solution, which makes the grid to be easily re-configurable, and the low computational burden required for the controlling purposes.Finally, we remark that due to the scalability features of the proposed approach, the convergence settling time still remains equal to 1 [s] when increasing the number of devices, differently from the centralized solution where this increasing number may significantly affects the dynamic performance of the whole grid.

Robustness to Variable Load
Here we show the robustness of the proposed approach by considering percentage variations of all the loads compared to the nominal values reported in [41].Specifically, as illustrative example, we take into account that load, indicated with L(t), varies over time according to Figure 6 (i.e., maximum variations of ±50%).Results in Figures 7 and 8 disclose that despite the presence of load variations acting on the whole grid, the proposed approach promptly reacts to load changes by recovering the desired optimal voltage magnitude as imposed by the N g generators.Specifically, in the time interval 5 ≤ t < 10 it is possible to appreciate that after increasing of the 30% of all the loads within the grid (see Figure 6), the smart controllers dynamically regulate the voltage magnitude of the j-th bus via production or absorption of reactive power.The same behavior also occurs when there exist both a variation of 50% in the time interval 10 ≤ t < 15 and a variation of −50% in the time interval 15 ≤ t < 20.

Conclusions
MAS-based architectures are considered as the most promising enabling methodology for decentralized voltage regulation in modern power distribution systems, where the massive pervasion of renewable power generators could hinder the deployment of hierarchical and centralized control paradigms.Although the conceptualization of MAS-based solutions has been widely explored in the literature, their deployment in realistic operation scenario is still at its infancy, and several open problems need to be addressed in order to identify the most effective computing paradigm, which reliably solves the voltage regulation problem, exhibiting high resilience to internal and external perturbations, high scalability to support an exponential growth of distributed energy resources, and low computational requirements.
In the light of these needs, this paper proposed a novel decentralized control architecture for the online voltage regulation in active power grids, which is based on a network of cooperative dynamic agents equipped with consensus protocols, and interacting with each other through wireless/wired communication networks.Thanks to the adoption of this cooperative paradigm, each agent regulates voltage magnitude of a specific generation/load bus so to achieve a synchronization behavior to desired voltage magnitude without the need for fusion data center.This feature makes the proposed solution self-organized, decentralized, and scalable, hence resulting a promising alternative to solve the voltage regulation problem in modern smart grids.
Simulation results, carried out for the realistic case of study of the IEEE 30-bus test system, confirm the effectiveness and the robustness of the approach in ensuring that all the electrical buses reach and maintain the desired synchronization behavior.

Figure 1 .
Figure 1.Cooperative smart controller network as a multi-agent dynamic system.

Figure 3 .
Figure 3.Time history of the voltage magnitude V i (t) [p.u.] for i ∈ N g .

Figure 4 .
Figure 4. Time history of the voltage magnitude V j (t) [p.u.] for j ∈ N c .

5 .
Time history of the reactive power Q j (t) [p.u.] for j ∈ N c .

7 .
Robustness with respect to variable load: time history of the voltage magnitude V j (t) [p.u.] for j ∈ N c .

Figure 8 .
Figure 8. Robustness with respect to variable load: time history of the reactive power Q j (t) [p.u.] for j ∈ N c .
05, 1.02, 1.05, 1.03, 1.05, 1.02] [p.u.].Our aim is to show how the proposed solution can ensure in a totally distributed fashion the desired optimal voltage magnitude of the whole grid with reduced power losses.