Assessment and Commissioning of Electrical Substation Grid Testbed with a Real-Time Simulator and Protective Relays/Power Meters in the Loop

Abstract

Electrical utility substations are wired with intelligent electronic devices (IEDs), such as protective relays, power meters, and communication switches. Substation engineers commission these IEDs to assess the appropriate measurements for monitoring, control, power system protection, and communication applications. Like real electrical utility substations, complex electrical substation grid testbeds (ESGTs) need to be assessed for measuring current and voltage signals in monitoring, power system protection, control (synchro check), and communication applications that are limited by small-measurement percentage errors. In the process of setting an ESGT with real-time simulators and IEDs in the loop, protective relays, power meters, and communication devices must be commissioned before running experiments. In this study, an ESGT with IEDs and distributed ledger technology was developed. The ESGT with a real-time simulator and IEDs in the loop was satisfactorily assessed and commissioned. The commissioning and problem-solving tasks of the testbed are described to define a method with flowcharts to assess possible troubleshooting in ESGTs. This method was based on comparing the simulations versus IED measurements for the phase current and voltage magnitudes, three-phase phasor diagrams, breaker states, protective relay times with selectivity coordination at electrical faults, communication data points, and time-stamp sources.

1. Introduction

Modern electrical utility substations have protective relays, power meters, and communication devices that are commissioned by electrical engineers [1,2] to assess the appropriate measurements from intelligent electronic devices (IEDs). Then, like a real electrical utility substation, complex electrical substation grid testbeds (ESGTs) need to be assessed before running experiments for monitoring, control (synchro check), power system protection, and communication applications because an uncommissioned ESGT could provide different experimental results. Commissioning provides the ability to test highly sensitive electronic systems contained in electrical substations, solar farms, wind farms, and generation systems [3] based on protection systems and power grid interconnection standards [2,4].

In electrical substations and power grids, the aim of the site commissioning test is to identify whether the operation of the protection devices, power meters, and stability control devices is consistent with the designed scheme under actual working conditions [5]. The measurements at the electrical substations may provide vital information to support decisions on service needs and grid control. However, power meters and protective relays can measure the phase voltages and currents on buses with different error percentage limitations depending on the applications. Although the requirements for the electrical substation measurements on delivered power are rather strict, the precision related to generalized monitoring is less so, and an error margin of up to 5% on the absolute measurements is acceptable [6]. However, for synchro phasor applications, the accuracy limits are based on synchro phasor measurements that must be synchronized with Coordinated Universal Time to meet accuracy requirements [7,8]. In this case, a phase error of 0.01 radians (0.57°) in the synchro phasor measurement will cause 1% total vector error, which is the maximum steady-state error permitted by IEEE Std C37.118 [8].

In the interconnection of distributed energy resources (DERs) based on IEEE Std 1547.1-2020 [4], after each voltage, a new steady state of reactive power and voltage values is measured. As a guideline, at two time points for the open-loop response time setting, the steady-state error is 1% [4]. In addition, instrumentation filtering may be used to reject any variation during the steady-state measurements. A sampling frequency of at least 10 kHz is recommended for measuring high harmonic components. The typical switching frequency of direct current/alternating current converters, usually operated with the pulse width modulation technique, is higher than 3000 Hz [9]. Thus, higher order harmonics can be an important concern in large-scale photovoltaic installations in which converters with voltage notching controls can result in induced noise interferences and current distortions [10]. The computational process for protective relays and power meters for measuring transient events with high-frequency components at the interconnection of inverter-based systems with DERs [11] needs to process the phase voltage and current signals in small time steps [12]. This computational process depends on the IEDs’ sampling frequency related to the Nyquist sampling theorem [13]. The sampling frequency of a signal should be at least twice the bandwidth of the signal to avoid aliasing at measuring the analog signals [13]. Thus, an IED with a sampling frequency of 10 kHz can measure frequencies up to 5 kHz.

In power system protection applications, the commissioning of protective relays is based on measuring the relay operation time and breaker states at different electrical faults with a relay test system [14]. Then, overcurrent protective relays could be set for inverse time overcurrent protection, and the calculated and measured relay times could be compared. In addition, the selectivity coordination between the primary and backup protective devices could be evaluated. The measured minimum coordination time interval (CTI) should be greater cycles than 7.2 for relay–relay devices, and greater than 12.2 cycles for fuse–relay devices [13], working as primary and backup protection devices.

Table 1. Measurement percentage error and minimum CTI limits for power grid applications.

Type of Simulation	Measurement Applications at Electrical Substations and Power Grids		Percentage Error and Minimum CTI Limits
Power flow analysis	Voltage, current, frequency, real and reactive power, and so on	General monitoring [6]	5%
		Synchro phasor [7,8]	1% *
		Interconnecting DERs [4]	1%
Electrical fault analysis (transient event states)	Selectivity between primary and backup overcurrent protection devices	Primary fuse and backup relay [13]	7.2 cycles **
		Primary and backup relays [13]	12.2 cycles **

* Total vector error, ** static relay (microcontroller), 1 cycle = 1 s/60 cycles = 0.01666 s/cycle.

lists the percentage error and minimum CTI limits for different power grid applications.

The real-time simulators (RTSs) of the electric power system reproduce the output (voltage/currents) waveforms with a desired accuracy, and are representative of the behavior of the real power system being modeled [15]. The RTS needs to solve the model equations for one time step within the same time on a real-world clock [16]. Then, the RTS produces outputs at discrete time intervals, where the system states are computed at certain discrete times using a specific time step. The RTSs for electric power systems are based on a technique for the transient simulations using digital-computer time-domain solutions [17]. The power system models are represented by taking advantage of the components available in the library of the software tool using a graphical interface and simulated on a hardware platform employing parallel computation. OPAL-RT Technologies Inc. [18,19,20] was used in this research, which uses the MATLAB/Simulink as the main modeling tool for the simulation, and RT-LAB software to run the real-time simulations.

University and lab research testbeds for power grid use cases are designed according to different applications. The Power Cyber testbed [21] and the Idaho CPS Smart Grid Cybersecurity testbed [22] were deigned to perform different attacks and explore cyber-physical impacts using distributed network protocol (DNP) and IEC 61850 Generic Object Oriented Substation Event (GOOSE) messages. The cyber-physical testbed [23] was built to investigate the potential cybersecurity vulnerabilities of protective relays and the impact of cyber-attacks, and the laboratory testbed [24] was designed for analyzing the fault-detection reaction times of protective relays for different substation topologies. Like real electrical utility substations, complex ESGTs should be assessed based on their application to identify whether the operation of the protection devices, power meters, communication, and control devices is consistent with the designed scheme under the actual working conditions in the testbed.

The electrical domain of this study was focused on real-time simulation of power systems and power “hardware in the loop.” In this research, a novel method to commission an ESGT based on an RTS with HIL is presented. This ESGT has power meters and protective relays as IEDs. It is an original and relevant topic in the field and is based on creating a method for commissioning ESGT with an RTS and HIL. This topic addresses a specific gap given by the need of commissioning an ESGT prior to running the experimental tests. The proposed method is based on using flowcharts to perform the commissioning and problem-solving tasks in an ESGT, comparing the simulations versus IEDs’ measurements for the phase current/voltage magnitudes, voltage polarity/balanced power systems, time-stamp sources, communication dataset points, breaker states, relay times, and minimum CTI between primary and backup protection devices. Figure 1 shows the contributions, measurements, and applications of this proposed method.

2. Materials and Calculations

In this section, the electrical and communication diagrams in the testbed are described. The theory and equations for performing the commissioning of the ESGT with an RTS and protective relays/power meters in the loop are presented. Then, the simulation environment and RT-LAB project for the experimental model are described in detail.

2.1. Electrical and Communication Diagram in the Testbed

In this study, the single-line diagram was based on an electrical substation (Figure 2A) with two transformers of 10 MVA, primary/secondary voltages of 34.5/12.47 kV, and a 12.47 kV radial configuration (Figure 2B). The electrical grid has Schweitzer Engineering Laboratories (SEL) 734/735 power meters and fuses on the outside substation feeders. In the electrical substation, two SEL 451 protective relays were installed. The electrical substation was a sectionalized bus configuration [25]. This arrangement has two single bus schemes tied together with bus sectionalizing breakers. The sectionalizing breakers may be operated open or closed depending on system requirements. This electrical substation configuration enables the removal of a bus or circuit breaker from service to keep service with another circuit breaker and/or bus, if necessary. The sectionalized bus configuration (Figure 2A) allows for flexible operation, higher reliability than a single bus scheme, isolation of bus sections for maintenance, and loss of only part of the substation for a breaker failure or bus fault [25]. The electrical grid was connected to the substation feeders through two breakers, and four power meters measured the phase currents and voltages on the load feeders (Figure 2B). The electrical substation and grid protection schemes were based on using overcurrent relays and fuses, respectively.

The communication system was based on a CGG system using DLT. The CGG system with DLT was applied to measure the data from the protective relays and meters. There, the SEL 451 protective relays and SEL 735 power meters were set with IEC61850 GOOSE protocol, and the SEL 734 power meters were set with DNP. In addition, a synchronized time clock was used to set the DLT devices with a precision time protocol, and the protective relays and power meters with the Inter-Range Instrumentation Group Time Code B (IRIG-B) protocol. Figure 3 shows the CGG system with DLT and communication diagram for the ESGT. In this diagram, the servers were part of the CGG system with DLT, and the workstations were used to monitor and collect data at the ESGT. Figure 4 shows the ESGT in the Advanced Protection lab at the Grid Research Integration and Deployment Center at Oak Ridge National Laboratory.

In this study, a novel method based on using flowcharts was developed to commission and perform the problem-solving tasks in an ESGT with an RTS and HIL. This testbed was developed to create a real framework to study the scalability of the CGG system with DLT, and the vulnerability assessment in an electrical substation with inside and outside substation devices [26]. The main goal of this testbed was to generate different power system use cases, including electrical fault tests.

The installation of the ESGT required the integration of power system protection, control, and communication equipment with different software programs [26]. Figure 5 shows the software steps applied in the ESGT.

MATLAB/Simulink software was used to create the ESGT model. RT-LAB software was used to integrate the ESGT model with the RTS. In addition, RT-LAB software was used to run the power system simulation tests. AcSELerator Quickset software was used to set the protective relays and power meters. AcSELerator Architect software created and downloaded IEC 61850 Configured IED Description (CID) files into the SEL 451 protective relays and SEL 735 power meters for using the IEC61850 GOOSE protocol. The SEL 734 power meters had DNP instead of the IEC 61850 GOOSE protocol. AcSELerator RTAC software was used to create the configuration for the SEL 3530-4 RTAC and the SEL 734 power meters. AcSELerator Diagram Builder software performed the supervisory control and data acquisition architecture for the ESGT. Wireshark software was used to collect and verify the GOOSE messages from the SEL 451 protective relays and SEL 735 power meters, and DNP messages from the SEL 734 power meters. Blueframe software was used to retrieve and store the artifacts from protective relays and power meters. SynchroWAVe software plotted and analyzed the events from protective relays after running the simulations.

2.2. Theory and Calculations

2.2.1. Voltage and Current Gains for the RTS

In the SEL 451 protective relays, the current and voltage gains for the RTS were calculated at the low-voltage level interface using the current and voltage scaling factors from the protective relay’s instruction manual [27]. The analog signals for the protective relays were scaled using the simulated phase currents and voltages with gain blocks. These current and voltage gains were calculated by Equations (1) and (2), respectively [28].

$${CG}_{RELAY}=1/\left({CTR}_{RELAY}\times CSF\right)$$

(1)

where CG_RELAY is the current gain in the RTS for the protective relay, CTR_RELAY is the current transformer ratio of the protective relay (80), and CSF is the protective relay’s current scaling factor (75 A/V) in amperes per volt.

$${VG}_{RELAY}=1/\left({PTR}_{RELAY}\times VSF\right)$$

(2)

where VG_RELAY is the voltage gain in the RTS for the protective relay, PTR_RELAY is the potential transformer ratio of the protective relay (60), and VSF is the protective relay’s voltage scaling factor (150 V/V) in volts per volt.

In the SEL 735 and SEL 734 power meters, the analog signals were not connected directly from the RTS to the IEDs’ low-voltage level interface because the current and voltage scaling factors were not available in the power meter’s instruction manuals [29,30]. In this case, the current and voltage signals were connected to the 5 A/120 V connectors. Then, the power meters were fed by individual 5 A and 150 VAC amplifiers [31], which were connected to the RTS. In this case, the analog signals for the power meters were scaled using the simulated phase currents and voltages with the gain blocks. These current and voltage gains were calculated by Equations (3) and (4), respectively.

$${CG}_{METER}=1/\left({CTR}_{METER}\times CAF\right)$$

(3)

where CG_METER is the current gain in the RTS for the power meter, CTR_METER is the current transformer ratio of the power meter (225), and CAF is the current amplifier factor (0.89 A/V) in amperes per volt.

$${VG}_{METER}=1/\left({PTR}_{METER}\times VAF\right)$$

(4)

where VG_METER is the voltage gain in the RTS for the power meter, PTR_METER is the potential transformer ratio of the power meter (450), and VAF is the voltage amplifier factor (20 V/V) in volts per volt.

The current and voltage gains for the protective relays were 1 V/6000 A and 1 V/9000 V from Equations (1) and (2), respectively. However, the current and voltage gains for the power meters were 1 V/200 A and 1 V/9000 V from Equations (3) and (4), respectively.

2.2.2. Inverse Time Overcurrent Protection

The inverse time current (ITC) curves were used to calculate the protection settings between the protective relays and fuses. The fuse–relay overcurrent coordination between the 50 and 100 T (TCC 170-6-2) fuses [32] with SEL 451 protective relays were calculated by setting the fuses and protective relays as primary and backup protections, respectively. The inverse time overcurrent settings of the protective relays were based on the U3 Very ITC curves [27] represented by Equation (5):

$$T_R=TDS\times \left(K_1+\frac{K_2}{M^{K_3}-1}\right)\times 60=TDS\times \left(0.0963+\frac{3.88}{{\left(I/CTR/I_P\right)}^2-1}\right)\times 60,$$

(5)

where T_R is the calculated relay time in cycles, TDS is the time dial setting in seconds, M is the applied multiple of pickup current, I is the primary input current in amperes, CTR is the current transformer ratio, I_P is the relay current pickup setting in amperes, and K₁ (0.0963), K₂ (3.88), and K₃ (2) are the curve constants for the U3 Very ITC curves [27].

The clearing ITC curves for fuses were created by collecting data from the 50 and 100 T (TCC 170-6-2) fuse data sheets [32] and fitting the collected data with the US curves as a function of the primary current and selecting the F₀–F₅ factors to match the manufacturer fuse curve with Equation (6), which represents the clearing ITC curve of these 7.2 kV fuses.

$$T_F=F_0\times \left(F_1+\frac{F_2}{{\left(I/F_4/F_5\right)}^{F_3}-1}\right)\times 60=6\times \left(0.002+\frac{3}{{\left(I/F_4/1.6\right)}^2-1}\right)\times 60$$

(6)

where T_F is the calculated clearing time of fuse in cycles, I is the primary input current in amperes, and F₀ (6), F₁ (0.002), F₂ (3), F₃ (2), F₄, and F₅ (1.6) are the curve constants for the 50 and 100T fuses. The F₄ curve constant is 70 for the 50 T fuse and 140 for the 100 T fuse.

The ITC curves of the 50 and 100 T fuse-relay coordination are shown in Figure 6A and B, respectively. The electrical fault analysis was conducted at the electrical substation grid circuit (Figure 2), and the fault block was placed at the load feeders.

Using MATLAB/Simulink, the electrical fault currents were simulated and the root-mean-square current magnitudes for the breaker in the protective relay were collected. The maximum electrical fault current was 1134 A for the three-line-to-ground (3LG) electrical fault, and the minimum electrical fault current was 751 A for the single-line-to-ground (SLG) electrical fault. The maximum load current at normal operation was 91.2 A on the breakers. The maximum load current was not intersected by the relay and fuse ITC curves. However, at minimum and maximum electrical fault currents in the load feeders, currents were intersected by the fuse and relay ITC curves, satisfactorily validating the selectivity coordination between the primary (relays) and backup (fuses) protection devices.

For the primary and backup protective devices, the maximum interrupting current was the value at which the protection devices reached the minimum CTI. Based on IEEE Std 242-2001 [13], the minimum CTI for a fuse–relay is 7.2 cycles (0.12 s). Then, the calculated CTIs between the fuse and protective relay for different electrical fault tests were estimated by Equation (7).

$$CTI=\left(T_R-T_F\right)>7.2cycles,$$

(7)

where CTI is the calculated CTI between backup (relay) and primary (fuse) protective devices in cycles, and T_R and T_F are the calculated time of the relay and fuse, respectively, at the electrical fault currents in cycles.

From Equation (7), the CTIs between fuses and protective relays at electrical faults were plotted. The operation time of the relay and fuse was calculated with Equations (5) and (6), respectively. The minimum CTIs for the 50 and 100 T fuses with the SEL 451 relays were 66 and 22 cycles, respectively. Figure 7 shows the CTIs between fuses and protective relays at electrical faults.

2.2.3. Current and Voltage Measured Percentage Errors

The current and voltage measured percentage errors for protective relays and power meters were calculated using the measured phase currents and voltages collected from the RTS and human–machine interface (HMI). The percentage error of the measured n (phase A, B, or C) currents (E_{% In}) was estimated by Equation (8).

$$E_{\%\hspace{0.17em}In}=\left(\frac{I_{n\hspace{0.17em}HMI}-I_{n\hspace{0.17em}RTS}}{I_{n\hspace{0.17em}RTS}}\right)\times 100,$$

(8)

where I_{n RTS} is the n (A, B, or C) phase current collected from the RTS in amperes, and I_{n HMI} is the n (A, B, or C) phase current collected from the IEDs’ HMI in amperes.

The percentage error for measured n (A, B, or C) phase voltages (E_{% Vn}) was calculated by Equation (9).

$$E_{\%\hspace{0.17em}Vn}=\left(\frac{V_{n\hspace{0.17em}HMI}-V_{n\hspace{0.17em}RTS}}{V_{n\hspace{0.17em}RTS}}\right)\times 100$$

(9)

where V_{n RTS} is the n (A, B, or C) phase voltage collected from the RTS in volts, and Vn _HMI is the n (A, B, or C) phase voltage collected from the IEDs’ HMI in volts.

2.3. Simulation Environment and RT-LAB Project

In the ESGT, the real-time simulations and experiment environment were developed by an OP5650 real-time simulator and an OP5607 expansion box, to increase the number of IEDs in the loop. The power system models were implemented with MATLAB/Simulink, and the simulations were run with RT-LAB. The power system simulations used a solver (powergui block) that provided a discrete simulation, 50 μs of sample time, and frequency of 60 Hz.

Table 2. Real-time simulation environment.

Software		Solver (Powergui Block)
Real-Time Simulator	Power System Model	Simulation Type	Sample Time [μs]	Frequency [Hz]
RT-LAB	MATLAB/Simulink	Discrete	50	60

shows the real-time simulation environment.

The RT-LAB project configuration was implemented in the host computer that deployed the RT-LAB project configuration in the target computer (RTS) and ran the simulations. The RT-LAB project configuration had two subsystems: one master block (SM_Master) with the simulated electrical substation and grid circuit, and another block to perform the scope supervision (SC_Console). Figure 8A,B show the SM_Master (electrical substation grid circuit) and SC_Console (scopes) blocks.

Inside the SM_Master block (Figure 8A), the 34.5/12.47 kV electrical substation grid circuit (Figure 9) was set based on the single-line diagram presented in Figure 2. The electrical substation grid circuit includes the utility source, electrical substation, power lines, and power load feeders. As shown in Figure 9A, the protective relays measured the phase currents and voltages from the breaker feeder locations and breaker trip–close signals. As shown in Figure 9B, the power meters measured the phase currents and voltages from the fuse feeders. The fault block (Figure 9D) can set the electrical faults at any location of the electrical substation grid circuit. The fault signal circuit (Figure 9C) set the time to start the fault state at the fault block (Figure 9D). Protective relays are typically connected to current measurement transformers that provide the scaled phase currents and voltages, and they are typically wired at the rear side’s connectors of the protective relays. However, the low-voltage level interface located at the front side of the protective relays was used in this case.

The current and voltage interfaces of the protective relays are shown in Figure 10A and B, respectively. The current and voltage signals were limited by a saturation block of +3.3/−3.3 V to protect the low-voltage level interface of the SEL 451 protective relays [27]. The A, B, and C phase currents and voltages from sensor blocks on the electrical substation grid circuit (Figure 9) were scaled into low voltage level signals by the current (1 V/6000 A) and voltage (1 V/9000 V) gain blocks, respectively (Figure 10A,B). These current and voltage gains were calculated by Equations (1) and (2), respectively. In the power meters, the analog signals were feed by current and voltage amplifiers that were connected to the RTS. The current and voltage interfaces of the power meters are shown in Figure 10C,D, respectively. These current and voltage gains were calculated by Equations (3) and (4), respectively.

The breakers of the electrical substation (Figure 9A) were controlled by the SEL 451 protective relays. During the simulations, the breakers were closed at normal operation and tripped at electrical faults. Figure 10 shows the breaker pole state (Figure 10E) and trip–close (Figure 10F) interfaces with the trip–close signal circuit (Figure 10G) for the breakers. The trip and close signals were received from the protective relays in the loop by the trip–close interface (Figure 10F), and these signals were collected by the trip–close signal circuit (Figure 10G). The trip (overcurrent pickup) and close signals were generated by the protective relay’s control outputs that enabled operating the breakers. The circuit breaker models were commanded with one signal (close = 1, open = 0). In the trip–close signal circuit (Figure 10G), the trip and close signals were detected by a hit crossing block, and the J–K flip-flop block was placed to detect the trip and close signals received from the protective relays. In addition, the breaker pole states from the trip and close breaker operations were sent to the protective relay’s control inputs through the breaker pole state interface (Figure 10E) using an external source to detect the three-phase breaker pole states.

Inside the SC_console subsystem (Figure 8B), the OpComm block (Figure 11A) and scopes (Figure 11B–E) were set to supervise the simulations. The OpComm block collected the signals simulated from the SM_Master subsystem (Figure 8A). Then, the scopes were opened during the simulations to supervise the experiments. The scopes for the protective relays measured the phase currents and voltages, and breaker pole state signals. The scopes for the power meters measured the phase currents and voltages.

3. Methodology

In this study, a novel method to commission and assess ESGTs with an RTS and IEDs in the loop is presented based on commissioning the current and voltage signals for power system applications, phase voltage polarity and balanced systems, protective relay time and selectivity with breaker operation, and communication data points for protocols and time sources. Flowcharts for commissioning an ESGT with an RTS and protective relays/power meters in the loop are presented, and the technical and economic aspects of commissioning a real electrical substation versus an ESGT are analyzed.

3.1. Commission of Current and Voltage Signals for Power System Applications

The commission of current and voltage signals for different power system applications is crucial, because—like real electrical utility substations—complex ESGTs need to be assessed. Figure 12 shows the flowchart to commission the current and voltage signals for a specific power system application. Initially, the ESGT was run for a balanced load power system simulation. Then, the measured current and voltage signals from the RTS and HMI were collected from the host computer and IEDs, respectively. The percentage errors for the measured n (A, B, or C) phase currents (E_{% In}) and voltages (E_{% Vn}) for IEDs were calculated by Equations (8) and (9), respectively. Then, the power system application was selected. If the percentage errors for the measured n (A, B, or C) phase currents (E_{% In}) and voltages (E_{% Vn}) were higher than the current and voltage measurement percentage errors for the selected power system application, then the current and voltage gains in the RTS for the IEDs were recalculated, up to the current and voltage measurement percentage error limits for the selected power system application.

In Figure 12, the adjusted current or voltage gain at the n (A, B, or C) phase in the RTS for the IEDs was calculated by Equation (10).

$$G_{ADJ\hspace{0.17em}n}=G_{CALC\hspace{0.17em}n}\times \frac{M_{RTS\hspace{0.17em}n}}{M_{HMI\hspace{0.17em}n}},$$

(10)

where G_{ADJ n} is the n (A, B, or C) phase current or voltage adjusted gains in the RTS for the IEDs, G_{CALC n} is the n (A, B, or C) phase current or voltage calculated gains in the RTS for the IED based on Equation (1), (2), (3) or (4), M _{RTS n} is the n (A, B, or C) phase measured current or voltage collected from the RTS in amperes or volts, and M _{HMI n} is the n (A, B, or C) phase measured current or voltage collected from the HMI of the IEDs in amperes or volts.

3.2. Commissioning of the Phase Voltage Polarity and Balanced Power System

Commissioning of the phase voltage polarity and balanced power system was performed by running a test with the RTS and IEDs in the loop. The test was a three-phase balanced load simulation without electrical faults. Figure 13A shows the flowchart to commission the phase voltage polarity/balanced power system. The measured phase voltage and current angles from the IEDs’ HMI were collected. Then, the phasor diagrams of the three-phase current and voltage components for the ABC rotation (Figure 13B) from the IEDs were assessed as a balanced power system. If an imbalanced power system was observed, the possible troubleshooting to be verified includes the (1) signal labels and/or polarities of phase current and voltage sensors on the MATLAB/Simulink model, (2) polarity of phase current and voltage signals from the analog cards at the RTS, (3) polarity of the phase current and voltage signals at the IED’s low-voltage level interface, (4) polarity of the 5 A current and 150 VAC voltage amplifiers at the high voltage–current level interface of IEDs, and (5) ABC phase rotation setting at the IEDs.

3.3. Commission of the Protective Relay Time, Selectivity Coordination, and Breaker Operation

The commissioning of the protective relay time, selectivity coordination, and breaker operation was based on observing the behavior of the ESGT at different electrical faults. In this case, the power system simulation was assessed to transient events generated during the electrical fault states, and the operation of protective relays and breakers were measured. Figure 14 shows the flowchart of commissioning the overcurrent relay settings in the ESGT with the RTS and protective relays in the loop.

In Figure 14, the calculated time of the relay and fuse were estimated with Equations (5) and (6), respectively. Then, the electrical fault (SLG, line to line ground [LLG], line to line [LL], and 3LG) tests were run to observe as the protective relay tripped the breaker. Figure 15 shows the relay event for the SEL-734_ABCG_FAULT test represented by a 3LG electrical fault set at the SEL 734 power meter location. The phase currents, trip and breaker pole state signals, and relay event data are shown in Figure 15A–C, respectively.

After running the electrical fault tests, the phase currents events were collected from the protective relay and plotted (Figure 15). Then, from Equation (11) and Figure 15, the measured relay time was estimated.

$$T_{Rm}=T_{{TRIP}_m}-T_{{IF}_m},$$

(11)

where T_Rm is the measured relay time in cycles, T_TRIPm is the measured relay trip time in cycles, and T_IFm is the measured initial fault state time in cycles.

The protective relay was also assessed by calculating the percentage relay time error for the electrical fault tests. The percentage relay time error was estimated by Equation (12).

$$E_{\%RT}=\left(\frac{T_{Rm}–T_R}{T_R}\right)\times 100,$$

(12)

where E_%RT is the percentage relay time error, T_R is the calculated relay time in cycles, and T_Rm is the measured relay time cycles.

Then, the measured CTI was calculated by Equation (13), after Equation (11).

$${CTI}_m=T_{R_m}-T_F=\left[T_{{TRIP}_m}-T_{{IF}_m}\right]-T_F,$$

(13)

where CTI_m is the measured CTI between backup (relay) and primary (fuse) protective devices in cycles, and T_F is the clearing time of the fuse in cycles.

In the flowchart (Figure 14), the protective relay must trip, the measured CTIs between the primary fuse and backup relay must be greater than 7.2 cycles [13], and the percentage relay time error should be smaller than 10%. If those desired conditions were not available, then troubleshooting was performed.

3.4. Commissioning of the Data Points for Protocols and Time Source

The commissioning of the data points for protocols was based on assessing the messages transmitted from the IEDs. Figure 16A shows the flowchart to commissioning the communication data points of the IEDs in the ESGT. In this case, some devices were set with IEC61850 GOOSE and others with DNP. The GOOSE and DNP maps (Figure 16B,C) were collected from the CID files and IEDs, respectively. The load-balanced power system simulation was run for 100 s with the RTS and the IEDs in the loop. Then, the simulated phase current and voltage magnitudes from the IEDs’ locations at the electrical substation grid from the host computer were collected. Wireshark software was used to collect the messages and verify the protocols of communication from the IEDs. In addition, the message data points for the phase voltages and currents were verified by plotting the phasor diagrams for the simulated balanced load power system.

The power meters and protective relays were set using the high-accuracy IRIG (HIRIG) protocol because the precision time protocol was not available on these IEDs. The time source for the IEDs was verified from their device displays based on the manufacturer’s manual [27,29,30]. The steps to observe the time source for the protective relay are shown in Figure 17, and the HIRIG source was observed on the protective relay’s display (Figure 17C).

3.5. Technical and Economic Aspects of Commissioning a Real Electrical Substation versus an ESGT

In electrical substations, the cost of performing the tests versus the gain of having the results need to be considered [33], because the relay test systems, technical training, and appropriate personal to develop the commissioning tasks could represent an extra cost. In addition, the electrical substation commissioning tasks are usually performed in electrical hazards areas such as electrical substation yards or electrical panels. However, in the commissioning of the ESGT, the commissioning of power meters and protective relays will be performed in a lab space, that represents a more secure site and with a lower operation cost than commissioning a real electrical substation.

The goal of the commissioning tests is to identify whether the operation of the protection devices, power meters, and stability control devices are consistent with the designed scheme under actual working conditions [5]. Electrical utilities need to develop a testing plan, prior to the start of the acceptance tests of electrical equipment in the electrical substations [33]. Similarly, the commissioning of the ESGT needs a plan for ensuring the necessary tests at the specific power system applications for being simulated, based on the flowcharts presented in Figure 12, Figure 13, Figure 14 and Figure 16. The method to commission an ESGT with an RTS and IEDs in the loop presented in this study was based on commissioning the current and voltage signals for the power system applications; phase voltage polarity and balanced systems; protective relay time and selectivity with breaker operation; and communication data points for protocols and time sources.

In electrical substations, a perfect startup and commissioning is important, because a functional failure in a main electrical substation is unacceptable [34]. The costs of such a failure can easily exceed the initial costs of the substation and cause significant process equipment damage [34]. In addition, having identical protective relays and power meters in the field (electrical substations) and lab spaces present a great advantage for electrical utilities, because some tests like cyber-attacks and/or electrical faults can be performed in an ESGT. Then, the electrical substation operation risk cost can be minimized by reducing the possibility of protective relay misoperations, electrical equipment damages, electrical hazards, and undesired blackouts.

4. Results

The ESGT was applied for general monitoring, and the commissioning of the analog signals for IEDs was performed satisfactorily. The measured phase currents and voltages from the RTS versus protective relays and power meters were compared. The percentage errors for the A, B, and C phase currents and voltages of IEDs were calculated using Equations (8) and (9), and plotted in Figure 18A,B, respectively.

The GOOSE and DNP messages for the protective relays and power meters were assessed. Wireshark software was used to collect the messages from the devices and collect the magnitudes and angles of the phase currents and voltages. The three-phasor diagrams for the phase currents and voltages were plotted to observe the simulated three-phase balanced power system that was generated by the RTS. The balanced phasor diagram and ABC rotation checked the wiring connection for the positive and neutral connectors of the IEDs. The phasor diagrams for the phase currents and voltages of IEDs are shown in Figure 19 and Figure 20, respectively, verifying the measured phase balanced power system from the GOOSE and DNP messages. In the GOOSE messages, the data points were expressed in 08-hexadecimal values; then, the phase currents and voltages were calculated by converting the 08-hexadecimal values into decimal values. In the DNP messages, the data points were expressed in decimal values; then, the phase currents and voltages were calculated by multiplying these decimal values by a factor of 0.1 to calculate the magnitudes, and 0.01 to calculate the angles.

The AG, AB, ABG, and ABC (or ABCG) electrical fault tests were assessed satisfactorily, and the simulated electrical fault tests at the ESGT were evaluated to the selectivity coordination between the fuses and protective relays working as primary and backup protection devices, comparing the simulated and measured minimum CTIs (Figure 21A). The calculated and measured minimum CTIs were calculated with Equations (7) and (13), respectively. For the primary and backup protective devices, the maximum interrupting current was the value at which the protection devices reached the minimum CTI. Based on IEEE Std 242-2001 [13], the minimum CTI for a fuse–relay is 7.2 cycles (0.12 s). In this study, the calculated minimum CTI for the fuse–relay protective device was greater than 7.2 cycles, as shown in Figure 21A. The percentage relay error time and breaker states for the electrical fault tests are plotted in Figure 21B. The percentage relay time errors were estimated with Equation (12). The percentage relay time errors were less than 10%, and the selectivity coordination between the fuses and protective relays was obtained.

5. Discussion

In the ESGT, the measured phase currents and voltages may provide vital information to support decisions on service needs and grid control. Although the requirements for the electrical substation measurements on delivered power are rather strict, the precision related to generalized monitoring is less so. Often, an error margin of up to 5% on the absolute measurements is acceptable [6]. The percentage errors of the measured phase voltages for the protective relays and power meters were smaller than 0.5% (Figure 18B). In addition, the percentage error of the measured phase currents for the protective relays also demonstrated good performance. However, the percentage error of the measured A and B phase currents for some power meters indicated an error margin greater than 5% (Figure 18A). This situation occurred because the power meters were connected through 5 A current amplifiers instead of using the low-voltage level interface, and the use of amplifiers can generate this behavior. Furthermore, amplifiers have low cutoff frequency and lack of linearity. In this case, the error was reduced by adjusting the current gains of the A and B phase currents at the interface circuits of the MATLAB/Simulink model (Figure 10C), because adjusting these current gains the measured magnitude and sinusoidal shape were improved by using Equation (10). Figure 22 shows the measurement of phase current signals for two SEL 735 power meter using different current gains at the RTS. In Figure 22A, the current gains were set at 1 V/2000 A, and the measured phase currents resulted in a distortional analog signal because the current injected into the power meters were given by 0.02 A. In Figure 22B, the current gains were set at 1 V/200 A to increase the amplifier currents injected into the power meter; and the distortional analog signal for the phase currents disappeared at the power meter’s displays (Figure 22B).

Therefore, using the low-voltage level interface to connect IEDs with RTSs is recommended. If the labeled pinouts or voltage and current scaling factors are not available for IEDs, the low-voltage interface of IEDs can be found [35]. In addition, if IEDs at an ESGT were used with high-precision current and voltage amplifiers [36], that would represent a costly alternative to build a testbed because 12 amplifiers for each IEDs would be needed to measure three phase currents and voltages at both sides of the breaker. Another important commissioning task is to assess the polarity for the measured phase currents and voltages at IEDs in the ESGT. The polarity for the analog signals of IEDs could depend on possible wrong cable connection scenarios at the MATLAB/Simulink model, analog cards, current/voltage amplifiers, and IED high- or low-voltage level interfaces, as shown in Figure 23.

In ESGTs with HIL, it is important to select IEDs with high sampling frequency because the transient events with high-frequency harmonics components could be available at electrical faults and operation of inverter-based systems with DERs [11]. In addition, high–sampling frequency IEDs could be connected to the low-voltage level interface instead of using current and voltage amplifiers that have low cutoff frequencies. The commissioning of the ESGT with HIL provided the ability to test IEDs connected at the RTSs. Then, complex ESGTs can be assessed before running experiments for monitoring, control (synchro check), power system protection, and communication applications to avoid an uncommissioned ESGT with HIL providing different experimental results.

In this study, a detailed assessment of an ESGT with HIL was performed, and the power flow and electrical fault simulations were run to perform the commissioning and define the problem-solving tasks.

Table 3. Commissioning and problem-solving tasks.

Power Flow Simulations	Areas	Commissioning Tasks	Problem-Solving Tasks
	Analog signals	Compare the measured current and voltage magnitudes of the HMI’s relays and meters versus the simulations.	Adjust current/voltage gains. Adjust setting of CTs or PTs at relays and/or meters.
		Compare the measured phase sequence of voltages and currents of the HMI’s relays and meters versus simulations.	Connect the positive, negative, and/or ground wires of analog signals for the meters, relays, amplifiers, and RTS.
		Observe the measured sinusoidal shape of currents and voltages from meter displays.	Adjust current/voltage gains.
	Breakers	Close and trip the breakers from the HMI or push buttons of the protective relays.	Measure the trip–close signal voltage levels. Revise the trip–close logic circuit.
	Time source	Compare the measured time stamps of protective relays and meters at device displays versus the digital clock.	Connect the correct IRIG-B cable to the meters and protective relays.
	Messages	Compare the measured Wireshark data set points for the IEC 61850 and DNP messages versus the data set maps set on the IEDs.	Revise and set the downloaded dataset maps of meters and protective relays.
Electrical Fault Simulations	Breakers	Observe that the breaker was tripped by the protective relay at the electrical fault tests.	Revise the logic circuit, voltage source, and/or control output’s relay setting for the trip signals.
	Overcurrent relay settings	Collect the relay event for the electrical fault test to compare the measured and calculated relay times.	Revise the overcurrent settings of the protective relays. Revise the calculated relay time with the inverse current–time curves.
	Selectivity coordination	Verify that the primary protective devices trip faster than the backup protective devices for maximum fault current, minimum CTI > 7.2 cycles (relay-fuse), and minimum CTI > 12 cycles (relay-relay).	Recalculate the theoretical relay time. Confirm the overcurrent settings of the protective relays. Compare the fault currents at the relay event versus simulation.

shows the commissioning and problem-solving tasks for the power flow and the electrical fault simulations at the ESGT with HIL. The problem-solving tasks showed possible solutions to correct and/or improve the results from the commissioning tasks. In the power flow simulations, the commissioning tasks were segregated into the analog signals, breakers, time sources, and messages. In the electrical fault simulations, the tests were performed for different electrical faults along the power grid to observe how the primary and backup protective devices were working together, that the breakers were tripped by protective relays, and good performance of the selectivity coordination. The commissioning tasks were segregated into the breakers, overcurrent relay settings, and selectivity coordination.

In his proposed framework, the commissioning of an ESGT with an RTS and IEDs (power meters/protective relays) in the loop can present technical limitations, based on hardware and software constraints:

• Number of IEDs: the number of power meters and protective relays that can be wired with the real-time simulator will be limited by the number of available analog and digital cards on the system. The actual system can connect a maximum number of teen IEDs in the loop. However, adding extra expansion boxes can increase the number of IEDs in the loop.
• Sampling frequency: protective relays and power meters have a sampling frequency. The sampling frequency of a signal should be at least twice the bandwidth of the signal to avoid aliasing at measuring the analog signals [13]. Therefore, IEDs with high sampling frequencies can measure and consequently assess algorithms for high harmonics scenarios, such as electrical faults and operation of inverter-based systems with DERs [11].
• Low-voltage interface vs. amplifiers: power meters and protective relays should be connected to the low-voltage level interface instead of using current and voltage amplifiers that have low cutoff frequencies. Sometimes, current and voltage scaling factors are not provided by IED’ manufactures, and the low-voltage level interface is not an option. However, the current and voltage scaling factors and unknown pinouts of low-voltage interfaces for IEDs can be obtained by a practical method [35].
• MATLAB/Simulink models: power system models in MATLAB/Simulink are a great tool to use with power systems testbeds in laboratories. However, these power system models are based on a simulation program that has been developed, and they can have some capabilities and limitations [37]. However, the broad use of MATLAB/Simulink have increased, and the power systems model have been continually being improved by students, professors, and researchers.

In complex ESGT with simulated DERs and multiple electrical utilities use case scenarios, the need of assigning permissions to users on different roles with protective relays and/or power meters is a possibility to be explored. This situation is based on role-based access control (RBAC) that emerged as a solution to establish a robust access control for numerous users [38]. The transition from the existing static RBAC policies to dynamic ones can change over time to provide security and compliance guarantees [39]. Then, several use case scenarios with system failure situations based on the RBAC policies could be studied with ESGTs that could be commissioned before introducing advanced techniques to overcome possible cyber-threats.

In electrical substations, engineers need to develop a testing plan prior to the start of the acceptance tests of electrical equipment [33]. Similarly, the commissioning of an ESGT needs a plan for ensuring the necessary tests. The presented method to commission an ESGT with an RTS and power meters/protective relays in the loop was based on comparing the simulation values versus IED measurements. The current and voltage signals for the power system applications; phase voltage polarity and balanced systems; protective relay time and selectivity with breaker operation; communication data points for protocols and time sources were assessed based on the flowcharts in Figure 12, Figure 13, Figure 14 and Figure 16. In addition, having identical protective relays on the field (electrical substation) and laboratories could be very important for electrical utilities, because commissioning tasks related about electrical fault tests are usually not allowed to be performed in a real infrastructure for safety concerns (risks of having protective relay misoperations, damaged of equipment and electrical hazards). Then, a commissioned ESGT could represent a safety and assessed platform to evaluate power system protection systems before being installed in a real electrical grid infrastructure.

The specific improvements regarding this methodology are part of the results of this study. These improvements are based on the novel contributions (Figure 1) of this new method for commissioning an ESGT with RTS and protective relays/power meters in the loop. This methodology could be a great tool to assess the operation of IEDs prior to perform the research tests. In addition, it could evaluate the stability for control devices (protective relays), to observe if their behaviors are consistent with the working research conditions. This methodology could assess a priori if the measurements from protective relays and meters provide vital information to support the research’s needs. This methodology could verify that the IEDs’ measured voltages and currents magnitude limits match the specific power system application (monitoring, power system protection, synchro check, communication, etc.). This methodology provides a detailed list of problem-solving tasks to perform in an easy way the commissioning of an ESGT with protection and power meters in the loop. However, limitations of this methodology could be related to advanced power system applications with DERs and microgrids.

6. Conclusions

An ESGT with an RTS and IEDs in the loop was set in the Advanced Protection Lab at the Grid Research Integration and Deployment Center at Oak Ridge National Laboratory. A novel method to assess and commission an ESGT was presented based on using flowcharts and comparing the simulations with the IEDs’ measurements. The commissioning tasks were based on assessing the phase current and voltage magnitudes and phasor diagrams for a balanced load power system, ABC or CBA phase rotation, protective relay times, breaker states, inverse time overcurrent selectivity coordination, communication data points, and time-stamp sources.

Commissioning the ESGT with an RTS and IEDs in the loop using this method enabled the assessment of the percentage error of the measured phase voltages and currents for IEDs (adjusting the current and voltage gains if necessary). This was used to verify the GOOSE and DNP messages from the IEDs (plotting the voltage and current phasor diagrams), to check the breaker trip states at electrical fault tests, to compare the calculated and measured relay times at electrical fault currents, to verify the percentage relay time errors for validation of the protection units, and to corroborate the selectivity coordination for the minimum CTIs (>7.2 cycles) between the primary (fuse) and backup (relay) protection devices.

In this study, commissioning and problem-solving tasks are listed. The proposed method was created to assess an ESGT with an RTS and IEDs in the loop for monitoring, power system protection, control (synchro check), and communication applications that are limited by small percentage errors. In the future, this research scope will be expanded. This commissioned ESGT with an RTS and power meters/protective relays in the loop will be assessed for other research applications, such as algorithms to detect faulted phases, monitor power quality limits with a CGG system using DLT, study the behavior of protective relays during cyber-attacks, and assess protective relay misoperations.