Overvoltage Prevention and Curtailment Reduction Using Adaptive Droop-Based Supplementary Control in Smart Inverters

: Recent developments in the renewable energy sector have seen an unprecedented growth in residential photovoltaic (PV) installations. However, high PV penetration levels often lead to overvoltage problems in low-voltage (LV) distribution feeders. Smart inverter control such as active power curtailment (APC)-based overvoltage control can be implemented to overcome these challenges. The APC technique utilizes a constant droop-based approach which curtails power rigidly, which can lead to signiﬁcant energy curtailment in the LV distribution feeders. In this paper, different variations of the APC technique with linear, quadratic, and exponential droops have been analyzed from the point-of-view of energy curtailment for a LV distribution network in North America. Further, a combinatorial approach using various droop-based APC methods in conjunction with adaptive dynamic programming (ADP) as a supplementary control scheme has also been proposed. The proposed approach minimizes energy curtailment in the LV distribution network by adjusting the droop gains. Simulation results depict that ADP in conjunction with exponential droop reduces the energy curtailment to approximately 50% compared to using the standard linear droop.


Introduction
Renewable energy sources (RES) have extended the capability of the grid to fulfill an ever-increasing load demand. The benefits of RES are reduced cost of energy production, increased generation capacity, reduction in energy curtailment, and improved voltage profile across the feeder [1]. However, inconsistency of photovoltaics (PV) generation accounts for numerous challenges in low voltage (LV) distribution networks [2]. Some of these challenges are sustained power interruptions, voltage regulation, harmonics, and voltage sags [3]. Smart inverters with appropriate control strategies can tackle these challenges to increase the profitability from RES like PV in LV grids.
Voltage regulation describes the ability of a system to maintain constant voltage under different load conditions. According to American National Standards Institute (ANSI) C84. 1-2016 [4], voltage standards for service voltage limits are classified as Range A and Range B limits. The voltage between 0.950 p.u. and 1.050 p.u. of nominal voltage lies under Range A, and the voltage between 0.917 p.u. and 1.058 p.u. of nominal voltage for 240 V service voltage lies under Range B. Note that the voltage can be within Range B for only a short duration and frequency, and thus corrective measures are necessary to constrict the voltage level within Range A limits. When the voltage is subjected to beyond Range B, overvoltage conditions exists in the LV distribution network. Furthermore, the voltage limits usually provided for distributed generation (DG) protection spans from 0.88 p.u. to 1.10 p.u. of nominal voltage. Even though the feeder voltage surpasses Range B (1.058 p.u.), DG protection is not enabled unless voltage exceeds 1.10 p.u. This scenario leads to overvoltage conditions in the LV feeder. Even if the protection is enabled for voltages below 1.10 p.u., it results in complete outage of the given DG, which is undesirable. In such cases, overvoltage in the system causes high power consumption by the load, increase in transformer losses, and high power dissipation in induction machines [5]. Recent revisions in IEEE 1547 standard [6], smart inverter functions like Volt-Var and Volt-Watt control have been encouraged to mitigate the overvoltage problem. These functions controls active and reactive power for reducing voltage at the point of interconnection of DG.
In the literature, overvoltage in distribution systems has been dealt with different methods such as expanding the conductor size of the feeder, using voltage regulators, modification of the taps of LV transformers [7], integration of energy storage systems (ESSs) like batteries [8][9][10], and inverter control approaches like Volt-Watt (active power curtailment (APC)) and Volt-Var (reactive power compensation) methods [11][12][13][14][15][16]. Prior research has merged reactive power compensation with additional methods for overvoltage prevention [17,18] which tends to be expensive, increases losses in the system, and requires communication links between controllers. Using reactive power compensation, higher transformer and distribution system losses were reported in [19,20] compared to droopbased APC control in LV feeders. Overvoltage control from smart inverters using droopbased APC approach (Volt-Watt control) has been studied in residential feeders with high PV penetration in [21] and small-scale wind turbines in [22]. In the APC-based method, smart inverters curtail power locally to deal with overvoltage issues. A constant droop coefficient is set in this approach to control the voltage for worst condition. However, this leads to forced curtailment that reduces the power injection by PV inverters during irrelevant situations. To overcome this constrained curtailment, Wang et al. [23] proposed power control method by continuously predicting the bounds of active power to frequently calculate the droop values. Furthermore, Alyami et al. [24] proposed adaptively adjusting the curtailment for maintaining the voltage under bounds along with maximizing PV generation. The non-adaptability of droop-based voltage control has also been identified in many studies [25,26]. Most of these literature works only use a linear droop function and do not investigate some other droop function apart from linear droop. Moreover, unlike conventional droop techniques, adjusting droop values based on irradiance and/or loading conditions can limit the curtailment compared to constant droop-based techniques.
In this work, we propose a machine learning-based method named supplementary adaptive dynamic programming (ADP) controller to adjust the droop gains. The ADP is utilized for generating a supplementary signal that adjusts the droop value to minimize curtailment. In addition, the droop value is changed by small values based on the supplementary signal so that the system remains relatively stable and does not deviate from its normal operation. The control framework designed with ADP assists in establishing a robust and optimal control concurrently by solving the Hamilton-Jacobi-Bellman equations of optimal control iteratively using the reinforcement learning concept [27]. Similar techniques have been endorsed to tackle different power systems problems [28][29][30][31][32][33]. A preliminary study on implementation of APC method with constant droop and ADP control on a simplified distribution model has been presented in [34,35]. In this paper, we extend the ADP controller to quadratic and exponential droops on a typical suburban network in North America. The test system has been modeled for low-voltage PV clusters with solar irradiance profile as input. The controller will be tested in more complex benchmarks with a greater number of houses in different configurations as part of our future work.
The main contributions of the paper are twofold: First, we aim to study the impacts of different droop functions in APC-based inverter control for LV distribution networks with PV. Second, we aim to design the APC-based smart inverter control with different droop models and ADP as a supplementary controller for overvoltage prevention for reduced curtailment in the LV distribution network. The control methods are further compared on its outcome on overvoltage prevention, curtailed energy, and power curtailment. The paper is organized as follows: the control formulation of inverter based APC control and the proposed method with supplementary control are presented in Section 2. The description of the test system and base case scenario are given in Section 3. Section 4 presents the results and the conclusions are described in Section 6.

Droop Control for Active Power Curtailment
Overvoltage prevention using APC technique is more effective in comparison to reactive power compensation in LV distribution feeders [36]. Droop-based algorithms are well known for coordinating power curtailment in PV inverters to prevent overvoltage without any form of communication between those PV inverters [21]. Figure 1 illustrates how a droop-based controller works. The main objective of the controller in this work is to limit the voltage below the upper limit of Range B voltage denoted as V cri2 (1.058 p.u. of nominal voltage) and to curtail the power after the voltage exceeds the upper limit of Range A voltage, denoted as V cri1 (1.05 p.u. of nominal voltage). The power injected by the smart PV inverter P inv into the grid with droop-based control is calculated using where P MPPT is maximum PV power output for a particular solar irradiance (kW), m is represented as droop coefficient (kW/V), and V(t) is measured voltage at point of connection (POC) of the PV with the LV feeder at time t. When the voltage at POC, V(t) operates in normal limits (Range A), i.e., V(t) < V cri1 , at that time the total PV generation is injected into the LV network, i.e., P inv = P MPPT . When V(t) operates beyond the normal limits, i.e., V(t) > V cri1 , then PV power injected to LV feeder is reduced by a droop factor based on (1). During this condition, the power curtailment P c is calculated using The droop coefficient obtained from (1) is a constant value, and the power curtailment occurs in a linear fashion with the droop coefficient as the slope of the function, as shown in Figure 2. As the voltage at POC exceeds V cri1 (1.050 p.u.), the droop-based APC begins to curtail PV power injection, until the voltage reaches V cri2 (1.058 p.u.), where maximum curtailment occurs. The same logic can be summarized using values in per unit form. When the voltage difference at the POC (∆V) is 0, there is no power curtailment. Similarly, when the voltage difference at the POC (∆V) is 0.008 p.u., the power curtailment is 1 p.u. Similarly, a set of ∆V and power curtailment values were created and used to obtain different functions by analytically solving the quadratic and exponential functions to have minimum and maximum curtailment at V cri1 and V cri2 , respectively. The quadratic droop and exponential droop function are given by (3) and (4), respectively. P c,quad = 1.52 × 10 4 (∆V) 2 (3) P c,exp = 0.00516e 650(∆V) (4) Figure 2. Linear, quadratic, and exponential droop for APC method.

Proposed Adaptive Droop Control Design
The fixed droop-based inverter control curtails PV power based on a constant factor for all scenarios, leading to unnecessary PV power reduction [23]. The aim is to minimize the power curtailment (whenever possible), restricting the voltage within the boundary condition of overvoltage. For this, a supplementary controller based on ADP is proposed to adjust droop coefficient while decreasing PV power curtailment. Figure 3 shows the control strategy of ADP assisted droop-based power curtailment that adapts the droop value to minimize power curtailment, such that V(t) < V cri2 (1.058 p.u. of 240 V), and 0 < P inv ≤ P MPPT . The input signals to ADP controller are voltage difference, power curtailment, and the time-delayed signals of voltage difference and power curtailment, which provides information on both objectives, i.e., the overvoltage prevention and the power curtailment minimization [31].
ADP is a machine learning-based approach to optimize the performance of dynamic systems over time [32,34]. Figure 4 shows the ADP controller with two neural network structures: the action network and the critic network. The action network generates the control action and the critic network optimizes the weights of the action network, based on a "reward" signal. In this case, the action network has a 6-6-1 structure with , and x 6 (t) as input signals. x 1 (t) is the p.u. value of the positive voltage difference between V(t) and V cri1 (1.050 p.u. of 240 V). x 2 (t) and x 3 (t) are the p.u. values of one-time step and two-time step delayed values of the voltage difference, respectively. Likewise, x 4 (t) is the p.u. value of the power curtailment. x 5 (t) and x 6 (t) are the p.u. values of one-time step and two-time step delayed signals of the power curtailment, respectively. The six hidden nodes are denoted as H 1 to H 6 , and the output node is denoted as u(t). The output of the action network is the droop value, denoted as m ADP (t) which adjusts the constant droop coefficient m c to compute the total droop coefficient m T (t) for the proposed combined controller. The critic network has a 7-6-1 structure with seven inputs: , and u(t). Similar to the action network, it also has six hidden nodes H 1 to H 6 and an output node J(t).  The critic network is formulated in such manner that optimal cost function J * (x(t)) satisfies Bellman principle of optimality, such that where r(x(t)) is the immediate cost incurred by u(t) at time t, and U c is a heuristic term used to balance. The output of critic network updates the weight of action network, as the value of U c is set 0 for our design. The reward signal is defined in a binary format with "0" representing success, or " − 1" representing failure. Here, the reward signal is defined as where c, a 1 , a 2 , a 3 , a 4 , a 5 , and a 6 are constants. The reward signal is devised to restrain the voltage from exceeding V cri2 at the same time minimizing the power curtailment. The value of c in the reward signal design is represented as "1" for positive value of the voltage difference and the power curtailment. This would ensure a negative reward would be applied in the ADP algorithm until the minimum curtailment is achieved or the voltage crosses the limit. The reward signal updates the weights of the action network appropriately. The weighting constants for the voltage difference signals (a 1 , a 2 , and a 3 ) were assigned equal weights of 0.1. The highest weight of 0.4 was set to the power curtailment weight constant (a 4 ) and similarly, the weight constants of one-time step and two-time step delayed signals of power curtailment (a 5 and a 6 ) were set to values of 0.2 and 0.1, respectively. Further details on the convergence and stability aspects of the ADP controller are briefly discussed in previous study [33].

System Benchmark
The system model of a residential suburban LV distribution feeder given in [21], shown in Figure 5, was used in this work. The system consists of twelve houses with residential PV installations in each house. The feeder has a single-phase 75 kVA, 14.4/0.24 kV distribution transformer with split-phase supply at the secondary side that connects to the grid. The network has a 100 m long feeder line with each house connected by a 20 m long feeder line. Each house is capable of generating 5 kW p of electrical energy from PV, and the annual generation from these houses is~14,892 kWh. Table A1 lists the LV transformer  parameters, and Table A2 lists the parameters for single phase PI section line. The PV inverter, and the house load models are both modeled as voltage dependent current sources to depict the instantaneous power flow from and to the houses, and PVs.

Solar Irradiance and Load Profile
The model was developed in MATLAB/Simulink [37] with inputs as solar irradiance data and load profile. The solar irradiance data was taken from SAMS software for Minneapolis, Minnesota for a sunny day [38]. The PV size was 5 kW p for all the twelve houses. The household daily load profiles were estimated using Poisson process and Queue theory and this method of estimation has been presented in [38][39][40]. These techniques model the energy consumption profile for each household appliances based on their consumption pattern which is summed up to get the load profile of each house. Moreover, these profiles also have realistic variations in energy consumption between the houses.
To generalize the different load profiles of the twelve houses, the average load is plotted in Figure 6, with solar irradiance data, and load profiles for a day. The resolution of one hour was taken for the solar irradiance data and load profiles, which is good enough to approximate steady state conditions. The profiles shows that high PV generation and low loading condition occurs at 1 pm in the modeled case.

Base Case
For the base case, the model was simulated without any inverter control where the PV inverters inject full maximum power point power until the voltage at POC reaches 1.10 p.u., after which the system shuts down if the voltage exceeds 1.10 p.u. Figure 7 shows the voltage for the base case simulation, without any droop control. The voltage profile for houses at the same POC of the LV distribution network are the same and equal in values. The maximum voltage of 1.065 p.u. occurred at 1 pm in houses 11 and 12, that was above V cri2 . Similarly, houses 7 to 12 undergo overvoltages, that is, the voltage exceeds V cri2 , during the 24 h simulation time period. The energy output from PV was computed by determining area under the power curve. The energy output from the PV inverters from all the houses summed up to be 489.60 kWh for the 24 h period.

Results
This section consists of results from the benchmark model with two case scenarios: a droop-based APC method with linear, quadratic, and exponential droop functions, and APC with the supplementary ADP controller added to each of the three different droop functions. All the cases have same PV power data and corresponding load profile for each house as described in the previous section.

Linear Droop
This case represents the method in which all the PV inverters have linear droop-based APC control. All the PV inverters have same droop coefficient which linearly curtails power. The droop coefficient is calculated so that the curtailment occurs between critical voltages, i.e., 1.05 p.u. and 1.058 p.u. The droop coefficient was calculated as where P max is the required maximum power that needs to be curtailed by PV inverter to limit the voltage from V cri2 to V cri1 . After applying droop-based APC control, the maximum voltage at houses 11 and 12 is reduced to 1.054 p.u. from 1.065 p.u of base case as shown in Figure 8, which is less than V cri2 . Similarly, none of the houses exceed the voltage limit of V cri2 . Figure 8 also shows the power curtailed during APC implementation with linear droop. The maximum power curtailed at 1 pm was 2.17 kW. However, due to the power curtailment, the total energy output from PV reduced from 489.60 kWh in the base case to 466.87 kWh with the linear droop-based APC. The total energy loss due to curtailment was 22.72 kWh.

Quadratic Droop
In this case, the PV inverters are controlled with quadratic droop-based APC method, where the droop values are guided by a quadratic function. Figure 9 shows the voltage profile of the twelve houses and the power curtailment with the droop-based APC method with a quadratic function. The maximum voltage of 1.056 p.u. occurs at 1 pm in houses 11 and 12, which does not exceed the critical value V cri2 (1.058 p.u.). The maximum voltages in houses from 7 to 12 is higher in the APC method with quadratic function than in APC method with linear droop. This happens because the droop function is guided by a quadratic function rather than a constant value, which better fits the objective to lower the curtailment. The voltages in houses are pushed to the boundary limits which facilitates lower curtailment. The maximum power curtailment is 1.98 kW at houses 11 and 12, which is less than that in constant droop-based APC method. The power curtailments for houses 7 to 12 are less than that of APC method with linear droop. The total energy loss due to PV curtailment is 15.27 kWh.

Exponential Droop
This case has droop-based controllers in PV inverters driven by an exponential droop function. Figure 10 presents the voltage profile of the twelve houses and power curtailment after using APC method using the exponential function. The maximum voltage at houses 9 to 12 is 1.057 p.u., which is less than V cri2 . Houses 9 to 12 do not experience overvoltage at any time during the 24 h period. The maximum power curtailment is 1.92 kW for houses 11 and 12, which is slightly less than that in APC method with quadratic droop. The curtailments from house 7 to 12 are the lowest in this method compared to above droopbased APC techniques as the function has the lowest curtailment for the same voltage difference among the three techniques. The energy loss in APC method with exponential droop is 12.40 kWh.

Discussion
From the above results, the efficacy of altering droop function from the fixed value is demonstrated which maintains the voltage profile of the farthest houses at the upper boundary of the operating condition. The power curtailment for overvoltage prevention decreases for each house in such approach as shown in Table 1. The energy output from the PV inverters increases and energy curtailment decreases as the fixed droop is changed to a quadratic or exponential function. The energy curtailed in each house decreases in both quadratic and exponential droops resulting in reduced overall energy loss than in linear droop. This proves that altering the droop coefficient can reduce unnecessary power curtailments in PV inverters in APC technique. Further, the numerical results with the proposed control strategy that can achieve the same objective are demonstrated.  The APC method with adaptive droop using ADP was implemented in each PV inverter of the twelve-house benchmark. The simulation time step was taken as 100 µs, so the time delay for one time and two time-delayed feedback signal of frequency were 100 µs and 200 µs, respectively. The ADP was implemented using the s-function block of MATLAB/Simulink. The initial weights of the action and critic network of ADP were randomly initialized between −0.1 and 0.1. The voltage at the POC and power curtailment were given as inputs in per unit form so that it can be used with any network parameters. The neural networks were then trained offline based on the procedure described in [29].
As the voltage at the POC is same for the symmetrical houses in every 20 m distance, a single ADP controller was used for each house pair to update the droop values during the training of the ADP. The network is assumed to have a simple communication link between each house pair for exchanging voltage, power curtailment, and droop coefficient values. The ADP controllers are used in odd numbered houses, and the average value of POC voltage and power curtailment of the houses in the house pair are used for training purpose. The adjustment for the droop values is sent to the even numbered houses by its house pair through the same communication link. The training was continued until reduced power curtailment was achieved with the voltage at POC being under allowable limits for all the houses. The proposed method was tested using the final trained weights. The testing phase for the ADP constitutes online learning, set up with a learning rate of 0.001. This procedure of reinforcement based learning aids in updating the control action, which leads to improved performance of the controller designed. Figure 11 presents the voltage profile of the twelve houses and the power curtailment after using APC method using ADP. The maximum voltage at houses 9 to 12 is 1.057 p.u., which is less than V cri2 , and higher than in APC method with linear droop. The maximum power curtailment is 1.25 kW for houses 11 and 12. The maximum curtailments from house 7 to 10 are 1.11 kW. The adjusted droop after the action of supplementary control ADP is shown in Figure 12. The first six houses have constant droop function as calculated in Equation (7). This constant droop is adjusted for houses 9 to 12 by ADP, which leads to lower curtailment while allowing the voltage towards the upper bound limits. The energy output from PV inverters has increased to 470.83 kWh compared to 466.87 kWh in linear droop-based APC and energy curtailment has decreased to 18.77 kWh.

Quadratic Droop with ADP
The quadratic function in Equation (3) was used for designing adaptive droop using ADP. The neural network was trained and tested in similar manner as in the linear droop case. Figure 13 shows the voltage profile using quadratic droop APC method with ADP. The maximum voltage is in marginal boundary of 1.058 p.u. (V cri2 ). The power curtailment for houses 11 and 12 increased slightly but decreased for houses 7 to 10. The total energy loss due to the PV curtailment has reduced to 12.98 kWh using the quadratic droop-based control with ADP.

Exponential Droop with ADP
Likewise, the exponential function in Equation (4) was used for designing adaptive droop using ADP. Figure 14 shows the voltage profile using quadratic droop APC method with ADP. In this case also, the maximum voltage is in marginal boundary of 1.058 p.u. (V cri2 ), which illustrates that the power curtailment in houses 7 to 10 has decreased to 10.86 kWh.

Comparisons of Performance
In this section, we compare the performance of different droop controllers established in the previous sections with and without the proposed supplementary controller. Tables 2-4 list the energy curtailed in each house using the droop-based techniques and the adaptive method using supplementary ADP. The energy curtailed in each house decrease in both quadratic and exponential droops than in linear droop-based APC method. This points out that changing droop coefficient can reduce unnecessary power curtailments in PV inverters in APC technique. It can been seen that using supplementary ADP in APC technique reduces the energy loss in the system in each of the droop-based technique. The proposed method reduced the total energy curtailment by about 17% with respect to base case using the linear droop, about 15% with respect to base case using the quadratic droop and about 12% with respect to base using the exponential droop-based methods. From a utility standpoint, it can be seen that the proposed method ensures better energy savings than existing droop-based method.

Conclusions
The APC method with both quadratic and exponential functions was able to reduce energy loss in the system, while maintaining the voltage profile within acceptable limits. Results indicate that adaptively changing the droop coefficient can significantly decrease the energy loss in individual houses, and the total energy loss in the overall system. Adaptively changing the droop coefficient using ADP reduced PV curtailment by 12-17%. Note that the proposed approach requires simple communication infrastructure for exchanging system parameters and control values within the houses at same POC. With the combination of exponential droop-based APC with ADP (total energy curtailment of 10.86 kWh) , the results show excellent improved energy curtailment of approximately 50% less that in linear droop-based APC method (total energy curtailment of 22.72 kWh). The next step in this research would be to investigate the long-term impact on voltage regulation and PV curtailment. A detailed techno-economic analysis of the proposed control will be performed in this long-term impact study. Furthermore, the other area that can be explored is to incorporate reactive power compensation with combination of the proposed adaptive droop control to extend the applicability in LV distribution feeders.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: