Energy Storage System Event-Driven Frequency Control Using Neural Networks to Comply with Frequency Grid Code

: As the penetration of renewable energy sources (RESs) increases, the rate of conventional generators and the power system inertia are reduced accordingly, resulting in frequency-stability concerns. As one of the solutions, the battery-type energy storage system (ESS), which can rapidly charge and discharge energy, is utilized for frequency regulation. Typically, it is based on response-driven frequency control (RDFC), which adjusts its output according to the measured frequency. In contrast, event-driven frequency control (EDFC) involves a determined frequency support scheme corresponding to a particular event. EDFC has the advantage that control action is promptly performed compared to RDFC. This study proposes an ESS EDFC strategy that involves estimating the required operating point of the ESS according to a speciﬁc disturbance through neural-network training. When a disturbance occurs, the neural networks can estimate the proper magnitude and duration of the ESS output to comply with the frequency grid code. A simulation to validate the proposed control method was performed for an IEEE 39 bus system. The simulation results indicate that a neural-network estimation o ﬀ ers su ﬃ cient accuracy for practical use, and frequency response can be adjusted as intended by the system operator.


Introduction
Recently, the power systems of the world have been changing significantly, as the penetration of renewable energy sources (RESs) has accelerated. Among the countries across the world, China has made the largest investment in RESs, and it plans to generate at least 35% of its electricity with RESs by 2030 [1]. Additionally, the European Union has increased its target for RESs from 27% to 32% by 2030 [2], and similar energy policies are being implemented in many countries. Because most RESs are inverter-based resources (IBRs) and not having rotating masses, the system inertia has been decreased consistently. Coupled with the variability of RESs, the low inertia has become a serious problem for system-frequency stability [3,4].
The battery-type energy storage system (ESS) is considered as one of the solutions to cope with the fluctuations caused by RESs.To complement the remaining synchronous generators (SGs) in the system, research on frequency support through the ESSs has been actively conducted. Droop control, which is the inherited control strategy of governors, has been applied to ESSs [5]. However, this technique cannot make a meaningful contribution to the inertia of the system, and further adjustments are required after the frequency reaches a steady state. To increase the system inertia, many researchers have studied control strategies in which ESSs emulate the dynamics of conventional generating units; (1) Event-driven strategy is applied for ESS control.
(2) Whether ESSs should perform a control action to comply with the grid code under a specific disturbance can be judged. (3) With relatively low peak-power rated ESSs, the frequency grid code can be complied. (4) With reduced ESS energy consumption, the frequency grid code can be complied.
The remainder of this paper is organized as follows. In Section 2, the concepts of RDFC and EDFC are explained in detail and compared. In Section 3, the frequency dynamics are analyzed via a model-based approach to elucidate the frequency response in a case where the ESS power output suddenly stops. In Section 4, the organization of the neural networks and the overall control strategy are described. Additionally, the data-selection technique for good performance of the neural networks and the consideration for generator outage situations are involved. In Section 5, the proposed control method is validated with the neural-network estimation performance test and simulation results.

ESS Frequency Support
In response to the penetration of the RESs, the installation of ESSs has been increasing. There are various ESSs with different energy-storage methods, and most types are featured in flexible and fast power control. They are utilized to support the power system in a variety of ways. The general applications of ESSs are peak shaving, frequency regulation, microgrid, and renewable integration.
Among the applications, frequency-regulation ESSs are used to keep the system frequency within the stable region. Maintaining the system frequency is important, because excessive frequency deviation from the standard can adversely affect the power system equipment. For this reason, most countries have their own grid code associated with a frequency [22]. Since the inertia of power systems is presently decreasing as the conventional generators with inertia are replaced with IBRs, the importance of frequency-regulation ESSs is increasing.

RDFC vs. EDFC
In this study, ESS frequency control methods are divided into two: response-driven and event-driven. In the former, the ESS adjusts the output according to the measured frequency. ESSs and other power-electronic equipment with frequency control normally use response-driven method. The latter can be described as customized control. When a disturbance occurs, control is performed with the operating point precalculated or estimated. The scheme of the RDFC and proposed EDFC methods are shown in Figure 1. F meas represents the measured frequency and P represent the output of the ESS. In RDFC, each ESS is individually controlled by a local frequency. In proposed EDFC, ESSs send equipment status information, such as SoC, to the central control system, which determines the operating point of each ESS according to the status and the event that occurred. This allows several ESSs to operate as one large ESS. RDFC can be divided into droop control and synthetic inertia (SI) control, as shown in Figure 2. The parameters in Figure 2 are as follows: f represents the system frequency, f re f represents the reference frequency of the control, T m represents the time constant of the measurements, k dr represents the droop-control gain, k si represents the SI-control gain, P 0 represents the reference power without RDFC, and P re f represents the practical reference power. The inputs of the droop control and SI control are the frequency error and ROCOF, respectively. In both controls, the output of the ESSs changes in proportion to the input. Here, the swing equation is reviewed for comprehension of the SI control dynamics. The difference between the mechanical power and electrical power of the SG is given by the change in the kinetic energy of the SG rotor. The power absorbed by the rotor can be expressed in per unit magnitude as follows: where ∆P m represents the mechanical-power change, ∆P e represents the electrical-power change, and H represents the inertia constant. When the ESS can change the power output in proportion to the ROCOF instantaneously, it can act as a synthetic rotor. Clearly, the SI-control gain k si has the same dimension as the inertia constant H. Since RDFC uses the local measured frequency, it has the advantage that communication is not necessary; however, ESSs cannot cooperate with each other globally. Additionally, ESSs operate unnecessarily even under disturbances that do not violate the grid-code criteria, and the selection of the control parameters can be complex. EDFC based on credible data can alleviate these problems.
For EDFC to show the desired performance for a particular event, the following information is needed: the extent of the event and corresponding operating point of the ESSs to comply with the grid code. However, it is difficult to obtain the information accurately owing to the nonlinearity of the system frequency characteristics. In this case, a data-based neural network can be an appropriate solution. Only with short-time frequency data, the neural networks can estimate the information needed for EDFC. If the estimation is appropriate, the ESSs of the system can respond to the event immediately and optimally. The estimation method using neural networks is discussed in detail in Section 4.

System Frequency Response (SFR) with EDFC
In this section, the frequency response is analyzed via a model-based approach. Although the prediction of disturbances is impractical with a model-based approach, this analysis provides insight into the frequency change resulting from a disturbance as the ESS performs the proposed EDFC. The frequency response is approximated by a linear SFR model [23], as shown in Figure 3a. In Figure 3, ∆P ESS represents the ESS output, ∆P D represents the disturbance power, t du represents the duration of the ESS control action, R represents the droop constant of the governor, F H represents the high-pressure turbine ratio, and T R represents the generator output time constant. All the variables are in per unit values. The MVA base is equal to the total rating of all the generating units in the system. In this study, the damping factor is ignored, and the disturbance power ∆P D is always positive. It is assumed that the disturbance and the ESS control action occur simultaneously, because the EDFC process takes a relatively short time compared to the frequency dynamics. Additionally, it is assumed ESS stops the control action after a certain period of time (t du ). To make the state variable of the SFR model clearer, Figure 3a is modified as Figure 3b. When the state variables are frequency change (∆ f ) and x in Figure 3b, the differential equation of the SFR is as follows: Since the input (∆P input ) before and after the ESS stops the control action is different, the time needs to be divided into two intervals for time-domain analysis. The inputs in the first and second intervals are ∆P ESS −∆P D and −∆P D , respectively. Since the initial values of the state variables are zero in the first interval but not in the second interval, they must be considered when solving the differential equation in the second interval. The development of differential equations is described in Appendix A, and the result is as follows: where ω n represents the natural frequency; ξ represents the damping ratio; ω r represents the damped frequency of oscillation; θ 1 and θ 2 represent the initial phase in the first and second intervals, respectively; ∆ f t du represents the frequency change when the ESS stops the control action; and α and  The two variables that the system operator can handle are the magnitude and duration of the ESS output. According to Equation (4), the FN is dependent on the magnitude of ESS output. In the case of limitless ESS output, the FN can be easily adjusted as intended by the system operator. In contrast, the second nadir (SN), which is affected by both the magnitude and duration, is not easy to adjust. Because the parameters of the second interval change with respect to the initial value of the state variable, the SN is difficult to analyze. However, the fact that state variable x is equal to the frequency change in the steady-state significantly simplifies the second-interval parameter. In a steady state, the parameters of the second interval become equal to those of the first interval, and f t du becomes R(−∆P D + ∆P ESS ). When x is equal to ∆ f at t du , the components of the SN are presented in Figure 4b. Furthermore equation (4) can be simplified as follows: Additionally, the FN and SN can be simplified as follows: The nadir without control (∆P ESS = 0) is expressed as follows: Unless the ESS output exceeds the disturbance, both the FN and the SN are larger than the nadir without any frequency support control. However, as the ESS output increases, although the FN increases, the SN decreases. Since the actual nadir is the smaller of the two values, the optimal nadir is obtained when the FN and the SN are equal. Here, the optimal nadir means the best nadir on the condition that the ESS stops the control action in the middle. When Equations (6) and (7) are identical, the magnitude of the ESS output at the optimal nadir can be obtained as follows: Substituting Equation (9) into Equation (7) yields the optimal nadir: When the ESS output decreases with a sufficient time constant (not a step response), the frequency response can improve. However, the rate of change of the output should be excessively slow, similar to the case of maintaining the output. When the optimal nadir for a particular disturbance is lower than the grid-code criteria, the ESS should maintain the output until the automatic generation control (AGC) intervenes.

ESS EDFC Using Neural-Network Estimation
In this section, an EDFC strategy using neural networks is introduced. The objectives of the neural networks are not only to predict the extent of the disturbance but to determine the corresponding magnitude and duration of the ESS output. The first subsection introduces the neural-network structure and overall control scheme. The second subsection introduces the data-selection method. The third subsection deals with the generator-tripped case.

Neural Network Structure and Control Scheme
As illustrated in Figure 5, two neural networks are used: NN1 and NN2. NN1 has one input and three outputs, and NN2 has three inputs and two outputs. The description of the new parameters related to NNs are as follows: t f represents the time at which the frequency reaches two threshold frequencies, f nd represents the frequency nadir without control, f ss represents the steady-state frequency without control, and FNR and SNR represent the references of the FN and SN.  Figure 6. NN1 estimates the ∆P D , f nd , and f ss from t f . According to Equation (1), the system power difference is proportional to the ROCOF; thus, the ROCOF can be an ideal variable for predicting the extent of the disturbance. However, because the instantaneous ROCOF can be considerably affected by noise during the measurement process, the average ROCOF can substitute it. There are two threshold frequencies in Figure 6: f th1 and f th2 . The average ROCOF can be determined as follows: When the t f is sufficiently small, the difference between the average ROCOF and the instantaneous ROCOF can be ignored. That is, t f involves sufficient information to predict the three outputs.

I/O values are visualized in
Each estimated value of NN1 is used as a control signal. ∆P D is used as the input for the NN2. When predicted f nd does not violate the grid-code criteria, the ESS does not need to perform any control. In this method, unnecessary operation of the ESS can be prevented. The f ss can determine whether to maintain the output of the ESS until the AGC operates. According to Equation (4), the steady-state frequency due to a disturbance can be expressed as: The optimal nadir can be obtained by substituting Equation (12) into Equation (10). When the optimal nadir is lower than the grid-code criteria, the ESS should continue the control action until there is an adjustment by the AGC. Nonetheless, the system frequency would violate the grid code.
NN2 estimates the magnitude (∆P ESS ) and duration (t du ) of the ESS output corresponding to the occurred disturbance (∆P D ) to form the intended nadir (FNR, SNR). The inputs are the predicted ∆P D , FNR, and SNR. Both nadir references must be higher than the grid-code criteria. Since ∆P ESS and the FN are highly correlated, corresponding ∆P ESS can be relatively accurately estimated. However, estimation of t du is more complex due to the relatively low correlation with SN. Therefore, proper data selection is more important for estimation of the duration than for estimation of the magnitude.
The overall control scheme is shown in Figure 7. The time count (t count ) starts after the frequency reaches the first threshold frequency ( f th1 ). When the frequency does not reach the second threshold ( f th2 ) for a long time, it can be judged that the disturbance is not large enough. In this case, the reference frequency ( f re f ) is updated to the currently measured frequency ( f m ), and the time count is reinitialized to zero. In the case where the frequency reaches the second threshold, t f is obtained, and the neural-network estimation is performed with it. In neural network operation, the control signal determination process is shown in Figure 8a. When the frequency reaches a steady state after the disturbance occurs, it is necessary to update the frequency reference, and the time count is reinitialized.

Data Selection for Accurate Estimation
The most important factor for the good performance of neural networks is the training data quality. For data to be high-quality, the output variation caused by the input variation must be noticeable. The correlation between the ROCOF and the disturbance power is obvious. Similarly, there are clear relationships between other inputs and outputs except the SN and t du . When the frequency reaches a steady state during the ESS control, there is no change in the SN, irrespective of the extent to which the t du increases. If the dataset contains many examples where the frequency reaches a steady state before the ESS stops the control action, the neural network encounters difficulty in estimating the duration of the ESS power for the reference SN. To prevent it, the data must be filtered. The data filtering algorithm is presented in Figure 8b.
The process of obtaining the data is as follows. First, the disturbance and ESS output of a random magnitude are determined. There is a constraint that the magnitude of the ESS output is lower than the magnitude of the disturbance. Then, the simulation is performed for the target system iteratively as the duration of the ESS output is incremented. When the SN does not change anymore, it can be judged that the steady state has been reached, and subsequent simulation examples are invalid data. Then, the simulation environment is changed to a new disturbance and ESS output. By this algorithm, a high-quality dataset for accurately predicting the duration can be built.

SG Tripped Condtion
In the case of disturbance accompanied by a generator outage, additional information is needed for a desired performance of the proposed EDFC. Since the estimation of the neural networks is highly dependent on the SG status in the system, the central control system needs to instantly acquire the information about the operating status of the generating units.
The neural-network estimation is based on the frequency characteristics of a particular system. Disturbances such as a momentary fluctuation of the RES power does not affect the estimation of the neural networks, even though the magnitude is significantly large. However, when the inertia and droop dynamics change, the intended frequency response may not be obtained with the original neural networks trained with normal-state data. Therefore, the neural network should be trained for all possible generator outages, and a neural network tailored for a current system status should be utilized.

Simulation Result
The dataset-building process and the case studies was performed for the IEEE 39 bus system by using the Power System Simulator for Engineers. The ESS was modeled as a lumped element with no capacity limitations, and the bandwidth of the active power control was set as 1.d6 Hz. f th1 and f th2 were set to 59.95 Hz and 59.9 Hz. The generator and governor models were the same for all ten generators in the system: GENTPJU1 and IEESGO, respectively. Additionally, for all the generators, the inertia constant and droop constant were 3.0 and 0.05, respectively. Information about the generator rating and operating point is presented in Table 1. It is necessary to analyze the time taken for the proposed control action to be applied in real-time. Measurement of the time for the frequency to reach f th2 is required to estimate the desired operation action of ESSs. When a disturbance that slightly violated the grid code occurred in the system, the time it took for the frequency to reach f th2 was about 150 ms. In addition, the time for the central control system to receive the generator trip information and to send the control signals to the ESSs needed to be considered. In [24], the signal propagation time on a 200-km-long fibre optic cable was assumed to be 1 ms. Eventually, the time from the occurrence of the disturbance to the ESS control actions was assumed to be 300 ms with enough margin in the study.
For building the dataset for NN training, a simulation was performed. While ESS control was involved in the simulation for the NN2 dataset, it was not involved in the simulation for the NN1 dataset. When a random-magnitude disturbance occurs in the system without frequency support control, the frequency change is as the dashed line in Figure 6. P D , t f , f pnd , and f pss constitute one example of the dataset for NN1. In the simulation for the NN2 dataset, the disturbance, the magnitude, and the duration of the ESS output were decided randomly. There were constraints that the ESS power was smaller than the disturbance (∆P ESS ≤ ∆P D ) and the ESS power maintained at least until the FN appears. The frequency change is as the solid line in the Figure 6. ∆P D , ∆P ESS , t du , FN, and SN constitute one example of the dataset for NN2. The disturbances are divided into two types: only a load change and a load change with a generator trip. The neural network was trained for both types.

Neural Network Training Results
The neural network model used in the study was an open-source model provided by MATLAB. The Levenberg-Marquardt training algorithm was employed, and the training parameters are presented in Table 2. All other parameters were default values in MATLAB.
Since the error rates differ by the training attempt, the average value after 10 training attempts was determined as the final neural-network error rate for a particular dataset. The error rates for NN1 and NN2 are presented in Table 3; Table 4. The error rate for the three outputs of NN1 and the magnitude of the ESS out of NN2 is less than 1%. The prediction is highly accurate, and it can be inferred that is a strong correlation between the inputs and the outputs. In contrast, the error rate for the duration of the ESS output is relatively high (2%~3%). However, this is also a sufficiently meaningful prediction result and can be used for control in consideration of the error rate. Table 4 presents the results of the training where data were selected using the dataset filter algorithm described in Section 4.2. Table 5 is the result of the training dataset, generated with a long duration limit of the ESS output in a normal state. The duration limit was set to 12 s. As the duration limit increases, the ESS maintains the output for a sufficient time, and the system is likely to reach a steady state. Since the SN does not change with the longer duration after the frequency reaches a steady state, an invalid dataset is generated, and it leads to a significant error rate for the duration of the ESS power. It can be seen that the error rate for the neural network trained by poor-quality dataset is approximately 10 times the error rate for neural network trained by the filtered dataset. Nevertheless, the error rate of the magnitude of the ESS output does not change. The reason is that the magnitude of ESS output is only correlated to the FN, which is determined independently of the duration of ESS output.

Energy Consumption of the ESS by FN and SN Reference
A key feature of the proposed control method is that the central control system can determine the desired frequency-nadir. A simulation was performed to find the appropriate FNR and SNR values in this subsection. The disturbance was fixed, and the FN and SN values were collected as the magnitude and duration of the ESS output changed. Then, the examples that share the same frequency nadir among the data were chosen and the ESS energy consumption examined with respect to the formation of the FN and SN. The magnitude of the disturbance was set as 400-MW and 600-MW load change. The results are shown in Figure 9. Here, one dot represents one example, and groups of same-color dots represents the examples that share the same nadir. The horizontal axis indicates the arrangement of the FN and SN. Zero indicates that the FN and SN have the same value. A positive value on the horizontal axis indicates that the SN is greater than the real nadir, and a negative value implies the opposite.
A high FN indicates that the magnitude of the ESS output is large and a high SN indicates that the magnitude is small and the duration is long. Because frequency nadir does not change when the duration is sufficiently long, there is a point at which only the energy consumption changes, not the FN and SN. In the graph, when the values of the FN and SN are close, the ESS energy consumption is modest. As indicated by the graph, the FNR and SNR should be set to the same value for achieving energy efficiency. Additionally, because the estimation of duration correlated to the SN has a relatively large error rate, it is necessary to set the SNR with a certain margin.

EDFC Performance with Different Nadir References
Here, the frequency-nadir criterion that the system must comply with is set to 59.5 Hz. The performance of the proposed control under a disturbance that violates the criterion is examined. However, the severe disturbance which forms the lower nadir than the optimal nadir of the proposed control is not considered. ESS EDFC was performed for three different nadir references: (59.5 Hz, 59.5 Hz); (59.6 Hz, 59.5 Hz); and (59.5 Hz, 59.6 Hz). A 700-MW load change occurred at 1 s, and the control action of the ESSs started at 1.3 s. The results are shown in Figure 10. The nadir decreases to approximately 59.4 Hz when the ESS do not support the frequency. When the EDFC is applied, the nadir is improved and formed near the reference value. In particular, the FN is nearly identical to the reference in all three cases where the EDFC is applied. However, for the SN, there is a small error, indicating a trend similar to that described in Section 5.1. The output of the ESS is presented in Figure 11. In consideration of the error rate of the duration of the ESS output, it is desirable to set the SNR to a higher value than the FNR with a margin.

Comparison with Conventional Methods
Since the ESS EDFC is a noble strategy that has not existed before, the proposed method is compared with the conventional ESS RDFC methods: droop control and SI control. Each method is described in Section 2.1. For comparison of the three methods, the RDFC parameters were adjusted to form the same nadir for the same disturbance. The control gain and deadband range of the RDFC methods are presented in Table 6. As the previous case study, a load change of 700 MW occurred. Frequency and ESS output for three different methods are shown in Figure 12.  All methods form the same frequency nadir for the same disturbance, but the corresponding ESS outputs are different. In droop control, it can be seen that the ESS maintains the output even in a steady state. In the SI control, the maximum output of the ESS is relatively high. In the proposed method, the energy consumption is reduced by stopping the control action in the middle, and the maximum output is also small.
When a modest disturbance which does not violate the grid code occurs, the results are shown in Figure 13. The disturbance was a 200-MW load change. The frequency nadir is formed near 59.9 Hz, which is not a severe value even when the ESS do not perform any control action in the proposed method. However, it can be seen that the ESS discharged power in the conventional methods. It can be concluded that the ESS is more efficiently controlled by the proposed method in the case study.

Generator-Tripped Case
A simulation was performed for the case where the generator is tripped. The disturbance magnitude is 700 MW, the same with the simulation described in Section 5.3. However, the power of G#34 is included. The results for the normal-state neural networks and the customized neural networks for the contingency (G#34 trip) are compared. The nadir references of the normal NN and the customized NN are the same: (59.5 Hz, 59.5 Hz). The result is presented in Figure 14. Although the magnitude of disturbance was the same with the previous case study (700 MW), the nadir is much lower when frequency support is not applied, owing to the SG trip, resulting in an inertia decrease. Additionally, the estimation of neural networks trained with the normal-state dataset is not accurate, and the frequencies violated the frequency grid code (59.5 Hz). In contrast, the estimation of the tailored neural networks trained with the dataset for the contingency is accurate. It indicates that the neural networks need to be trained for each possible contingency, and generator trip information is needed for a desired performance.

Conclusions
In this paper, a noble ESS EDFC strategy using neural networks was proposed and verified using the IEEE 39 bus system. The neural networks were trained with a dataset consisting of numerous examples according to the disturbance in the system. After the training, the neural networks predicted the severity of the occurred disturbance and estimated the customized operating action of the ESSs. The test results indicated the estimation accuracy of the neural networks was sufficient to be used for practical use. The validate simulation was performed in PSS/e using the control action of the ESS provided by the neural-network estimation. The results showed the frequency nadir with the EDFC was nearly identical to the reference nadir set by the operator-in other words, estimating the required action of the ESS for grid code compliance.
The proposed control prevents the ESSs from operating unnecessarily under a small disturbance which does not violate the grid code. Furthermore, if there is a more severe disturbance that violates the grid code, ESSs can use minimal energy to comply with the grid code by stopping the control action by the signal of the central control system in the proposed EDFC. This feature is a huge advantage compared to the RDFC. The proposed strategy is expected to be utilized in the optimal design and operation of frequency-regulation ESSs in a highly variable future system with large-scale RESs. In the absence of an initial value, the Laplace transform of differential Equations (2) and (3) (4) when t ≤ t du . The parameter of Equation (4) when t ≤ t du can be expressed as follows by using the governor and generator parameters: To determine the time at which the nadir occurs, the first minimum of the transient component must be obtained. The transient component of Equation (4) when t ≤ t du is as follows: ∆ f transient (t) = αe −ξω n t sin(ω r t + θ 1 ) Differentiating Equation (A5) yields where θ X = θ 1 + tan −1 ( The first time Equation (A6) becomes zero is the time at which nadir occurs. It is given as ω n T RS −ξ ). When the ESS power output lasts for a sufficient time and x t du is equal to ∆ f t du , T RS becomes equal to T R and, accordingly, α and β become the same.