Distributed power hardware-in-The-loop testing using a grid-forming converter as power interface

: This paper presents an approach to extend the capabilities of smart grid laboratories through the concept of Power Hardware-in-the-Loop (PHiL) testing by re-purposing existing grid-forming converters. A simple and cost-effective power interface, paired with a remotely located Digital Real-time Simulator (DRTS), facilitates Geographically Distributed Power Hardware Loop (GD-PHiL) in a quasi-static operating regime. In this study, a DRTS simulator was interfaced via the public internet with a grid-forming ship-to-shore converter located in a smart-grid testing laboratory, approximately 40 km away from the simulator. A case study based on the IEEE 13-bus distribution network, an on-load-tap-changer (OLTC) controller and a controllable load in the laboratory demonstrated the feasibility of such a setup. A simple compensation method applicable to this multi-rate setup is proposed and evaluated. Experimental results indicate that this compensation method signiﬁcantly enhances the voltage response, whereas the conservation of energy at the coupling point still poses a challenge. Findings also show that, due to inherent limitations of the converter’s Modbus interface, a separate measurement setup is preferable. This can help achieve higher measurement ﬁdelity, while simultaneously increasing the loop rate of the PHiL setup.


Introduction
Sustainable energy needs for the future are driving the increasing adoption of Renewable Energy Sources that have altered the makeup of the traditional power grid. The increasing complexity and scale of the power system requires tools that can carry out large scale simulations to study its interactions and interoperability with newer hardware and novel control schemes and develop advanced assessment methods. This entails a radical shift from static, offline load flow simulations towards dynamic, online real-time simulations with higher-fidelity. With the help of specialised and powerful computational tools, such as OPAL-RT and Real-time Digital Simulators (RTDS), real-time simulation and Hardware in the Loop (HiL) testing has emerged as an attractive option to study complex power system configurations in detail [1][2][3][4]. However, the capability of existing single real-time simulator-based infrastructures is still limited. Simulation of large-scale power grids with detailed distributed energy resource models and control systems is a major challenge. Furthermore, HiL-based testing has a major limitation in scalability, as only limited number of components or devices can be tested or investigated in a single scenario. Thus, scaling real-time power system simulations with HiL testing for large models is a challenging task [5]. Most importantly, the availability of adequate hardware interface of power HiL (PHiL) testing in the form of fast linear amplifiers is limited due to a high cost. These challenges can be addressed by Geographically Distributed Real-time Simulation (GD-RTS) [5,6], Geographically Distributed Power Hardware in the Loop (GD-PHiL) testing and reducing dynamic fidelity requirements on hardware in test platforms [7]. While a conventional PHiL setup consists of a digital real-time simulator (DRTS), power interface/amplifier and a hardware under Test (HuT), GD-PHiL extends it by a real-time virtual gateway, which handles data exchange between the two sites, as shown in Figure 1. These advanced concepts enable the ability to interface and test larger systems through distributed experiments, giving rise to 'Power System in the Loop' based testing [8,9], which is also the objective of this work.
In this paper, we demonstrate the application of commercial grid-forming converter hardware for PHiL experiments, using the example of an ABB ship-to-shore converter, which overcomes the unavailability of a dedicated power amplifier. In contrast to purpose-built amplifiers for this application, a grid-forming converter cannot be controlled with instantaneous voltage signals but is rather controlled by a set of Root Mean Square (RMS) voltages and frequency. This limitation may pose challenges in the context of ensuring fidelity of PHiL setup with such amplifiers and requires evaluation. In addition, these types of converters usually lack digital or analog interfaces for fast and direct update of control parameters or the acquisition of measurements. However, laboratories may have grid-forming converters in their facilities, which could be utilised for PHiL experiments without the hefty costs of purchasing a new dedicated PHiL amplifier. Due to their comparatively slow interfaces and built-in protection measures, the PHiL integration cannot be dynamic, but instead is considered 'quasi-static'. On the other hand, most of these commercial converters are relatively safe to operate and exhibit a reasonable robust control behaviour, which gives the opportunity to integrate complex multi-device systems with multiple power components and software controls in a Power System-in-the-Loop (PSiL) setup, such as to emulate a distribution feeder with multiple distributed energy resources.
GD-RTS and GD-PHiL environments enable the coupling of different software and hardware subsystems through discrete-time communication, time-varying delays and packet loss in the communication channel between two or more subsystems can result in artificial energy [10]. This is either stored or generated at the interface and may give rise to artificial energy artifacts in the system dynamics. Consequently, due to energy generation, these co-simulation systems can become unstable. Thus, it is advisable to observe this artificial energy generated at the interface and use it as metric, as well as possibly in the future online compensation. In [11], a non-iterative energy-conservation-based co-simulation algorithm is investigated using residual energies for adaptive step size control and evaluating the stability of the co-simulation, by which the accuracy and efficiency of the co-simulation is remarkably enhanced as compared to constant step sizes. A monitoring framework is proposed in [12] to keep track of the excess energy and then a dissipation method is implemented to eliminate that energy. Thus, artificial energy monitoring and compensation can help co-simulations achieve stability in spite of artificial interface energy. However, thus far, applications of dissipative co-simulation interfaces are limited to automotive and mechanical systems. Sources of fidelity degradation in real-time co-simulations using non-iterative solvers and interface algorithms differ from the factors that deteriorate fidelity in GD-PHiL, where communication delays, sampling rates of measurement and control signals and dynamic response of PHiL amplifiers are the main issues. However, the objective is in both applications to address sources of fidelity degradation by achieving energy conservation at the interface. To this end, a metric based on artificial energy at the interface is considered in this work for fidelity evaluation.
The key contributions of this paper are as follows: 1. Experimental demonstration of a Quasi-static PHiL (QsPHiL) power interface using a grid-forming converter, in a geographically distributed multi-rate PSiL setup. 2. Identification of limiting factors for the use of converters as interface hardware in QsPHiL experiments. 3. A compensation method that takes advantage of the different timescales to accelerate convergence of the iterative quasi-static interface, to make up for inherent delays in the QsPHiL setup. This was experimentally verified by a reduction in voltage overshoot and convergence time. 4. First application of a novel Energy-Based Metric (EBM) in GD-PHiL.
The rest of this paper is organised as follows. Section 2 discusses the important aspects of remote coupling and QsPHiL experiments. Metrics for fidelity evaluation are covered in Section 3, while Section 4 delves into the implementation details and applied interface algorithm along with compensation method. Further, Section 5 presents and discusses the key findings of this work, while, in Section 6, key conclusions are drawn.

Remote Coupling and Quasi-Static PHiL
The conventional PHiL setup leverages a simulator that performs electromagnetic transient simulation in real time and a power amplifier controlled by instantaneous values of voltage waveform provided by the simulator at sampling rates similar to the real-time simulator, such as 10 kHz to 20 kHz, which allows investigations of transient phenomena and hardware response to, e.g., anomalous or disturbed system states, such as protection response or fault-ride-through behaviour. By contrast, testing needs in smart energy systems also include long-term behaviours relevant, e.g., for the study of load behaviour connected in complex distribution networks, voltage support and flexibility through coupling to other energy domains [7]. Fidelity requirements here apply to sampling rates of 10 Hz and below. Thus, simplified forms of the conventional PHiL can be considered, with reduced model granularity, lower sampling rates for simulator and interfaces, and simplified dynamics. These simplifications allow for increased scalability, reduced demands on equipment and model accuracy, thus generally reduced testing cost. To accurately describe the method and impact of results, this section introduces the QsPHiL concept, terminology and related literature on simplified and mixed-fidelity PHiL setups, and then focuses on power interfacing issues.

QsPHiL and Other Reduced-Fidelity PHiL Configurations
The QsPHiL approach introduced and analysed here refers to a power interface (converter, communications and interface algorithm) that aims at fidelity for stationary RMS frequency and RMS voltage levels, corresponding to a time scale of 0.1 Hz and slower, and thus reducing requirements at the power interface [7]. Quasi-static here refers to ability to emulate the global energy balance and stationary power flows using stationary RMS frequency and voltage parameters. Alternate reduced-fidelity approaches achieve quasi-dynamic fidelity, meaning that the power interface and simulation are accurate and responsive such that RMS voltage and frequency dynamics can be accurately represented, i.e., above 1 Hz, sufficient to model frequency dynamics for inertia and primary frequency control [9]. Mathematically, quasi-dynamic setups reduce the represented dynamic states as compared to fully dynamic setups; quasi-static setups eliminate dynamic states and are only accurate as stationary representation. In a quasi-static interface setup, convergence of interface parameters is an iterative process where the test casts the number of iterations and time between stationary (equilibrium) states.
A quasi-static fidelity can thus also be considered when power-flow simulation models are applied in a PHiL setup. Quasi-static and quasi-dynamic PHiL setups therefore have many precursors in the literature (see, e.g., [13][14][15]). To classify a PHiL, in terms of dynamic fidelity, we consider: (a) the intra-emulator fidelity (at what resolutions are the emulated dynamics accurate?); and (b) the inter-emulator fidelity (at what rate is the mutual representation of the other emulator through the interface accurate?).

Reduced-Fidelity PHiL in the Literature
The authors of [13] referred to quasi-static time series (QSTS) based models in their setup, operating with a 2 s time step in the simulated systems, which is also employed in [14,15]. A quasi-static power system setup can be applied in a quasi-dynamic multi-domain setups, where fidelity for dynamic states, e.g., in the heat domain, is preserved [16]. In [17], the reduced fidelity setup operates with a model using PowerFactory "quasi-dynamic" simulations and thus their setup is referred to as "quasi-dynamic" PHiL. Note that PowerFactory definition of "quasi-dynamic" is only quasi-dynamic for frequency, but quasi-static for voltage regulation at distribution grids, such that definitions may overlap.
While conventional PHiL setups pose high challenges to be implemented in GD-PHiL manner as instantaneous values of waveforms must be exchanged between simulator and power amplifier, QsPHiL setups are suitable to be considered for GD-PHiL application [7]. A remote control hardware in the loop test, which remotely couples a distribution system simulated in RTDS located in Greece with an actual On-Load-Tap-Changer (OLTC) controller in Spain, is implemented in [18]. This test uses a virtual platform called JaNDER for data exchanging in real-time between two laboratories with the latency lower than 300 ms. A trans-pacific quasi-static closed-loop system test is conducted for a GD-PHiL setup, as presented in [13]. The work uses a power distribution network simulator with physical PV/battery inverter in the loop at the National Renewable Energy Laboratory in Golden, CO, USA and a physical PV inverter at power at the Commonwealth Scientific and Industrial Research Organisation's Energy Centre in Newcastle, NSW, Australia.
A common challenge in quasi-static co-simulation and PHiL setups where either dynamic simulators (as here) or a highly dynamic power interface is involved, as, e.g., in [14], is that a difference of sampling rates needs to be accommodated by a suitable interface model and algorithm.

Mixed-Fidelity PHiL and PSiL
The present setup is characterised as:

•
Subsystem A: RTDS @ 10 kHz • Subsystem B: SYSLAB @ 1 Hz to 10 Hz (measurement limitations) • Inter-emulator A → B: approx. 1 Hz: converter set point updates • Inter-emulator B → A: 10 Hz: measurements Considering typical power system dynamics, accurate representation of both frequency and voltage dynamic parameters cannot be represented, even though both parameters can be set at the interface: the response of the Power Interface hardware is in the order of 2.5 Hz for voltage and several seconds, 0.3 Hz, for frequency, returning measurements of converter current and phase (RMS P/Q values) at a rate of 16 Hz. Several iterations of A → B and B → A are required for convergence of parameters at the interface, leading to an expected interface-based fidelity at about 0.5-1 Hz for voltage events and 0.1 Hz for frequency events.

Interface Fidelity of GD-PHiL Setups
In energy systems simulation, it is essential that energy is preserved across simulator interfaces. However, for most configurations of interface variables, this quality cannot be achieved in practice within a single step: e.g., when an interface is defined by transfer of P/Q information, package loss equates to loss of energy or simply due to the communication delay or inaccurate synchronisation of two simulations. It is possible, however, to log and recover the energy loss incurred through lost packages as in this example, using a corrective mechanism. Therefore, the multi-step interpretation of the exchange quality is important. With the dynamic of exchange in consideration, the convergence of interface variables and stability of invariant qualities within tight error bounds and a deterministic convergence horizon are key qualities to be achieved. An invariant, such as the conservation of energy, can thus be re-formulated as a residual value. This residual can be monitored for possible divergence. In a second step, this monitoring approach can be employed to correct the exchange variables, enabling a stabilisation of the coupling. For further study, this approach should be employed to first log, and then possibly improve the low-rate and multi-rate coupling between Co-simulation and hardware in the loop (QsPHiL) and co-simulation interfacing DRTS .
In addition to the measures for synchronisation, it is also critical to consider the qualities of the simulation interfaces and exchange variables. When sub-systems (simulations) are coupled at a physical layer, algebraic loops arise (cyclic dependencies). As the state of the exchange variables is computed independently in each simulator, in principle, only an iterative and explicit coupling would allow for accurate simulation. In all real-time HiL and SiL, one necessarily degrades the overall system state coherency to allow for the simulation to take place without such iterations. Under this notion of synchronisation, there is an inherent trade-off between the degree of coherency of the overall system state, i.e., to which extent it approximates a monolithic simulation, and the idle time allowed by the choice of simulator computational cycle time T. To wit, the greater one chooses T, the lower the degree of state coherency.
As a consequence, a concern is to preserve essential properties of the global system state as much as possible. To manage this trade-off, it is possible to identify qualities of global exchange variables to be maintained in spite of said coupling losses.
In [19], an integrated test setup for a remote coupling test is examined. The test couples a full-scale hybrid electric vehicle power system simulated in real time at the U.S. Army TARDEC simulation laboratory with a hybrid power train located at the another laboratory in San Jose, CA, USA. This application uses bi-directional real-time communications over the open Internet using Army assets of two different laboratories 2450 miles apart. In this study, leaked energy is calculated for both sites to evaluate how closely the model matches the behaviour of the real hardware and then proposes robust control techniques that compensate for asynchronous Internet communication delays for the closed loop operation.
However, the aforementioned work is missing a generic compensation method to account for different timescales and inherent delays in any remote QsPHiL setup. Hence, this is one of the key research contributions of this work.

Fidelity Evaluation and Monitoring Approach
GD-RTS and GD-PHiL are naturally limited by delays incurred by geographical distances. Thus, phenomena of interest on timescales shorter than these delays are generally not presentable in a GD-RTS implementation. Such experiments include fast transients in transmission networks, ripple due to power electronic switching or effects due to harmonics. Experiments in GD-RTS and GD-PHiL instead target facets of the power system described by the RMS-values of the system, including voltage and frequency response, energy exchange tracking or controller operation.
Due to this focus of GD-RTS and GD-PHiL compared to conventional PHiL, evaluation of the fidelity of a GD-RTS implementation can only to a limited extent apply methods of validation commonly used for PHiL systems, e.g., comparison of voltage and current waveforms during transient response with the objective to achieve high fidelity for a wide range of frequencies, including harmonics [20]. Instead, validation metrics in GD-RTS and GD-PHiL should focus on consistency of RMS values in the emulated grid across boundaries of emulators, and on longer-term drift of associated values, e.g., integrated complex energy exchange.
Another challenge in GD-RTS and GD-PHiL is the evaluation of simulation fidelity itself, as a key performance indicator. In the context of GD-RTS, fidelity can be defined as the degree of similarity of a system response in the GD-RTS environment to the response of a monolithic simulation model. A common approach for fidelity evaluation is to utilise a monolithic simulation to obtain reference results. However, in a real-world GD-RTS or GD-PHiL application scenarios, the monolithic model would typically be unavailable.
The lack of reference results means comparison, both between runs of the same implementation and across implementations, of model outcomes is the sole indicator of fidelity available. Metrics of comparison for these outcomes should be based on the target metric of the experiment, and they should be informed by any artefact of the interface that may impact this metric.
Our purpose here is then threefold: 1. Establish a general set of metrics under which GD-PHiL interfaces can be evaluated even when a reference signal is unavailable. 2. Provide sets of observables and parameters that allow characterisation testing of a GD-PHiL interface. 3. Develop a GD-PHiL interface that directly targets these metrics to demonstrate this characterisation.
We proceed in this section to discuss Points (1) and (2) in general terms, with Point (3) treated in Section 4.

GD-PHiL Interface Models as Two-Port Networks
A co-simulation interface can be modelled and analysed as a 2-port network, as illustrated in Figure 2. Here, a voltage measurement at the primary side is transmitted through an interfacing actuator (I A 1 ) to the secondary side across a communication channel (Comm.) and subsequently translated to be actuated by a voltage source on the secondary interfacing actuator I A 2 . Similarly, a current measurement is transmitted from the secondary side and actuated on the primary side via a current source. Note that, as far as the interface is concerned, there is no difference if the voltages and currents are due to a simulation in an DRTS or represent physical flows due to a hardware component. The treatment in this section does not distinguish if either side is physical or simulated, only that they represent a power flow. For an ideal interface, voltage and current on primary and secondary sides match at all times, and thus in particular instantaneous powers match at all times: When dealing with non-ideal interfaces, Equation (1) will be modified to account for, e.g., communication delay and jitter, intermittency of operation, multi-rate operation of primary and secondary loops and calibration offsets. As an example, suppose there is a consistent delay τ between v 1 occurring on the primary side and v 2 being actuated on the secondary side, with the same delay τ between i 2 and i 1 . Neglecting other sources of error and assuming no compensation, the power balance now writes which clearly does not respect energy balance for the two-port system in general.

Fidelity Evaluation Metrics
Given the ideal behaviour in Equation (1), a metric to examine an interface's fidelity can be derived as: which compares the difference in instantaneous power between Ports 1 and 2 over a time horizon, with measurements i 1 [k], . . . sampled isochronously at rate 1/T s . In experiments where a system event causes the system to go to a steady state, one can use the new steady state of the system as a reference value from which fidelity metrics can be derived. For such an experiment where an event affects the signal v 1 (t) at time t = 0, leading to a new steady state value of v 1 atv 1 , the under-and overshoot of the signal are defined as v overshoot where t is the first crossing time where v 1 (t ) =v 1 . Further, the stabilisation time under tolerance d is defined as and the RMS deviation is defined as In applying these formulations to signals which respond to the event through the interface, i.e, v 2 , i 2 and i 1 , one takes t = 0 as the first time step for which the event translates to these variables.
Suppose more than one event is included in the experiment at times 0 < τ 2 < τ 3 < . . ., with all variables reaching steady states for an event before the next event occurs. Then, the steady-state is defined for each event separately, and all formulations above span over the intervals [0, τ 2 ), τ 2 , τ 3 ), etc. instead of the interval [0, ∞). Note that the times of each event occurring should be taken according to when each variable responds.

Hypotheses
To conduct tests of a GD-RTDS interface, the testing regime must take into account both the action of the system under test and the action of the test bed on which that system is implemented. Considerations thus naturally divide along two axis: On the first axis, one distinguishes parameters, which are decision on free experimental variables made prior to the experiment, and observables, which are recorded during or calculated after the experiment. On the second axis, one distinguishes intrinsic aspects, which concern the signals transported by the interface, and extrinsic aspects, which concern the interface's handling of those signals.
The parameters which can be altered for a characterisation test can be divided into intrinsic parameters, which affect the signals transported by the interface, and extrinsic parameters, which affect the interface's handling of those signals.
Altering the intrinsic parameters reveal how well the interface is able to represent the electrical system connected at either side, i.e., to which extent the interface is able to maintain transparency of electrical operation. Examples of such parameters include: The size of voltage step on primary side; and the size of a current/impedance step on secondary side. In general, altering intrinsic parameters only affects intrinsic observables, as defined below.
In contrast to intrinsic parameters, extrinsic parameters seek to trace how the interface performs under performance degradation. By artificially degrading the performance of the interface in a quantifiable way, it may be possible to extrapolate observables to those that would be obtained using an ideal interface. Examples of such parameters include: Adding artificial delay on the primary or secondary side by inserting sleep commands in the loop; or reducing the rate of sending and actuating by only applying every n = 1, 2, . . . set points. Altering the extrinsic parameters of the system will affect both the intrinsic and extrinsic observables of the system, as defined below.
For both types of parameters, one can define analogous intrinsic and extrinsic observables for the interface; intrinsic observables relate to the signals transported, which are discussed in detail in Section 3.1, while extrinsic observables relate to the timing of the messages carrying these signals.
Pertinent extrinsic observables include: 1. the time between each message being sent from the primary side (primary side sending rate); 2. the delay between a voltage occurring on the primary side and a message being sent reflecting this voltage (primary side sending delay); 3. the delay between a message being sent from the primary side and the message being received at the secondary side (primary side communication delay); 4. the delay between a message arriving at the secondary side and the content of that message being reflected on the physical system (secondary side actuation delay); and 5. the equivalent timings for messages originating at the secondary side.
Taken together, these parameters and observables form a complete set of dimensions for characterisation. For a given interface between given electrical systems at the primary and secondary side, and for a given set of intrinsic and extrinsic variables, the ability of the interface to form a transparent interconnection is given by the intrinsic and extrinsic variables.
With these general considerations in mind, the following section relates the development of a compensation interface aiming to improve each intrinsic metric compared to an uncompensated interface.

Implementation in Distributed Environment
To further study the QsPHiL concept, a case-study was implemented and demonstrated in a real lab setup. This section describes system architecture and interfacing details while Section 5.1 introduces the test grid.
In this case-study, a RTDS real-time simulator at DTU Lyngby and a grid-forming Back-to-Back power converter (B2B) at SYSLAB were interconnected to a simple dump load. This setup poses two challenges to its implementation: 1. Latencies and jitter due to geographical distance between simulator and converter 2. Limited controllability and observability of the converter due to the Modbus interface of the converter The objective is to achieve identical or at least comparable results in the distributed and monolithic environment without exchanging large packets with signal vectors. Here, the distributed environment was represented by the case-study using the SYSLAB laboratory. The monolithic environment was an equivalent model of the lab setup, which was purely simulated in the DRTS at Lyngby as a reference case.
This section describes the architecture and techniques used to achieve this goal. These include the exchange of GPS synchronised timestamps and sequence numbers, asynchronous programming and a fine tuning of the converters Modbus communication. Figure 3 shows the layout of the GD-PHiL setup of this study. The distribution system was simulated in RTDS located in the Lyngby campus of DTU while the hardware part belonged to the SYSLAB laboratory situated in DTU Risø campus, which is approximately 40 km away. The locations are interconnected by the public Internet.  A VILLASnode instance located in DTU Risø campus operates as a gateway for real-time data exchange between the RTDS simulator and the SYSLAB hardware. VILLASnode is a gateway for co-simulation interface data [21]. It converts various protocols and (de-)multiplexes signals to and from multiple sources. The gateway supports over 18 different protocols and interfaces, which include general purpose broker-based messaging protocols such as Advanced Message Queuing Protocol (AMQP) and Message Queuing Telemetry Transport (MQTT), as well as industry standard protocols such as IEC 61850, Web protocols like WebSockets and HTTP/REST and Interfaces to Simulators and custom Field Programmable Gate Arrays (FPGA). It collects statistics about the exchanged data such as packet loss, one-way delay and jitter and provides a network emulation to simulate the conditions of a geographically distributed co-simulation. VILLASnode is a C/C++ application executed on a Real-time optimised Linux machine.

RTDS
Data exchange between RTDS and the VILLASnode gateway is performed via a GTNET card and using the UDP protocol. Communication between the VILLASnode gateway and the SYSLAB loop uses the ZeroMQ publish/subscribe protocol due to firewall restrictions which forbid direct UDP communication between SYSLAB and RTDS. Lastly, the SYSLAB loop communicates with the B2B converter via serial Modbus. The SYSLAB loop was implemented in Python and relies heavily on asynchronous programming (asyncio) to realise non-blocking forwarding of measurements and set points [22].
SYSLAB is an experimental facility at DTU Risø campus, designed as a test bed for advanced control and communication concepts for intelligent distributed power systems. Figure 4 shows a section of the 400 V, three-phase grid which, with a total of 16 bus bars and 119 automated coupling points serves as the electrical backbone of the facility and allows a large variety of different grid topologies. All equipment on the grid is automated and remote-controllable. Each unit is supervised locally by a dedicated computing node which acts as a communication gateway. Nodes contain measuring and network equipment, data storage, backup power and an interfaced computer, which can be used for hardware and software platform test. The whole system can be run and monitored remotely and as such is a good test bed for the (Power) System-in-the-Loop testing objective of this paper. As PHiL interface, a B2B converter is used. Built by ABB, the PCS100 Static Frequency Converter is designed for ship-to-shore operation and operates at 50 or 60 Hz on low voltage distribution grids and is rated for a nominal output power of 125 kVA. Figure 5 shows the simplified electrical layout of the B2B converter. Its internal inverter is controlled by the set points provided from the simulation model in the DRTS in the form of RMS voltage (V rtds ) and frequency ( f rtds ). Measured power at the converter output terminals P b2b and Q b2b is monitored and fed back into the simulation where they are injected by a controlled current (see Section 4.4). The B2B converter is a three-phase device. However, it is only controllable in a balanced mode by a single set of RMS voltage and frequency set point. Voltage measurements are provided individually for each phase, whereas power measurements are aggregated for all phases. This restricts the test system to balanced three phase grids. The required modifications to satisfy this constraint are described in Section 5.1. For the purpose of Power System in the Loop (PSiL) testing, the paper follows the terminology of Heussen et al. [23]: System under Test (SuT) is the IEEE benchmark system, the SYSLAB laboratory and OLTC within the simulation. Object under Investigation (OuI) in the context of this paper is the power interface, namely the GD-PHiL setup, the B2B converter, Interface Algorithm and the Compensation. Hardware under Test (HuT) is the hardware portion of the SuT, namely the cables and Dump Load, but not the B2B converter as it belongs to the OuI.

Synchronisation and Time Stamping
The sequence diagram in Figure 6 shows the communication of signals and timestamps between simulator and SYSLAB loop. Both sites of the setup provide their own time reference via either a dedicated GPS synchronised GTSYNC card for RTDS or an NTP server in the case of SYSLAB. While the NTP server exhibits an inferior accuracy in contrast to the GTSYNC card, it is still obtains its time reference via a direct attach GPS receiver (Stratum 0). The resulting time stamping precision is estimated in the range of a few hundred µs and still adequate for this setup as other sources of uncertainties in the system are around 2-3 magnitudes larger (jitter by B2B controller and ZeroMQ publish/subscribe). The exchange of timestamps alongside the signal value allows for the calculation of a round-trip time (RTT) and an estimated one-way delay (OWD).  Initial tests revealed that our experiment is dominated by the response time of the Modbus connection on the inverter t modbus . The average response time of the converter is t modbus = t pq2 − t v2 ≈ 20 ms for each measurement or set point update. The total round-trip time between simulator and converter is estimated as t rtt = t 3 − t 1 + t update /2 ≈ 5 ms which results in an OWD of t owd ≈ t rtt /2 ≈ 2.5 ms that is around a magnitude lower than the Modbus request time. Therefore, the influence of the geographical distance in this experiment is minor in comparison to the peculiarities of the B2B converter due to its Modbus interface and internal control.

RTDS loop
The time of a blocking Modbus request is the main limiting factor for a faster execution of the SYSLAB loop, as the execution of each request is strictly sequential. Ideally, the requests would be directly synchronised to signals received by the DRTS, which for a stable interconnection over the public internet should not exceed 1 kHz to 5 kHz [6]. To reach such high update rates, the number of Modbus requests must be reduced the minimum required for the HiL test and subsequent analysis (set points and measurements). A total of n reg = 3 Modbus requests is used to exchange the following signal with satisfactory timing for the application.
The signals are listed in the order of their respective Modbus requests: • RMS Voltage set point for the converter outputs • RMS Voltage measurements for each phase of the converter V b2b,abc (RMS) • Active/reactive power of the converter P b2b , Q b2b These requests are continuously communicated in a tight loop with the B2B converter. This results in an an average loop cycle time t loop = n reg × t modbus ≈ 60ms or f loop = t −1 loop ≈ 16 Hz, which is at least a magnitude larger than the targeted update rate by the DRTS.
To simplify the experiment setup, all signals are collected by the simulator resulting in a single file per run. In post processing, timestamps for signals originating from the laboratory are corrected by −t owd in the case of P b2b , Q b2b and −(t owd + t modbus ) in the case of V b2b,abc .

Multi-Rate Interfacing
When coupling discrete systems with differing rates, issues with lack of system state information arise. In the present case, a fast system (DRTS rate of 20 kHz) is coupled with a slow system (SYSLAB loop rate of ≈ 16 Hz). Due to the limited sampling rate of the SYSLAB loop, the simulation lacks information about the present current drawn by the DL + Lines in the SYSLAB environment which are needed for controlling the current injections in the model. Similarly, voltages from the DRTS (V rtds ) are sent at a rate faster than the converter can actuate due to the bottleneck of the Modbus interface.
As described in the previous subsection, the SYSLAB loop is running without synchronisation to the DRTS in order to maintain the maximum loop rate. To avoid congestion of the communication network between DRTS and SYSLAB loop, the DRTS should avoid sending update for every simulation time step as this would result in 1/∆t = 20 kHz network packets per second. To avoid this, the communication rate from DRTS to SYSLAB loop was reduced to 100 Hz, which provides the SYSLAB loop with a new set point every 10 ms. The B2B converter will pick up this new set point on average with a 5 ms delay, as the time offset between arrival of a new set point and the next Modbus request can be expected to be uniformly distributed. In contrast, the measurements of the B2B converter are forwarded to the simulator after the respective Modbus read-register requests have been fulfilled. As shown in Figure 6, two Modbus requests are required for reading voltage and power measurements. The SYSLAB loop always performs both requests and sends the combined measurements to the DRTS. Consequently, the measurements are taken with a slight offset of t pq2 − t v2 which are corrected later on.
One way to address this issue is to build a virtual model to represent what the slow system would do if it could respond quickly enough. For instance, given a voltage set point u f from the fast system, the interface would report a current value i f = f s (u f ), using a function f s inferred from measured values of the slow system. Provided the characteristics of the slow system change slowly, f s will provide a good estimate of the actual current response of the slow system. If, however, parameters of the lab setup are changing, the virtual model must be capable of detecting these changes and adapt accordingly. An interface algorithm satisfying these requirements is introduced in Section 4.4.

Interface Algorithm
The PHiL setup in this work consists of a DRTS, which performs electromagnetic transient simulation, and a power converter, which amplifies the reference voltage signal received from the DRTS and electrically interfaces the HuT, in this case DL and lines (DL + Lines). While simulation time step of DRTS is 50 µs resulting in a 20 kHz, SYSLAB measurements at power converter are only available at a rate three magnitudes lower on average. Therefore, it is not feasible to apply Interface Algorithms (IA) based on instantaneous values of voltage and current waveforms.
Instead, the IA is based on RMS of voltage, frequency and power quantities. On DRTS side, the RMS of the voltage waveform is calculated at the fundamental frequency and forwarded to the power converter along with estimated system frequency. The power that is withdrawn or injected by DL + Lines is measured at the power converter terminals and forwarded to the DRTS. The IA in DRTS calculates current waveform based on the received power measurements and local voltage measurement. IA is illustrated in Figure 7. The RMS voltage V rtds (t) at the DRTS terminal of the interface is sampled with a frequency of 100 Hz and available at B2B controller with time delay t owd . Response time of B2B converter t b2b results in the delayed voltage V b2b (t) = V rtds (t − t owd − t b2b ) at B2B terminal with respect to the voltage measured at DRTS terminal of the interface. Power measurements at the B2B terminal, P b2b (t) and Q b2b (t), represent HuT response to the delayed voltage, which results in P b2b (t) = f (V b2b (t)) = f (V rtds (t − t owd − t b2b )). These power measurements and voltage measurement V b2b (t − t modbus ) are sampled with f loop ≈ 16 Hz and communicated with the DRTS where they are available with additional t owd time delay. Therefore, power measurements of DL + Lines that are available in DRTS are delayed for communication time delay t owd and for B2B time response t b2b since they represent DL + Lines response to the voltage at B2B terminal that is delayed with respect to DRTS voltage. Calculation of the current reference value i re f (t + T S ) is conducted in direct-quadrature (dq) synchronous reference frame. The calculation is based on the power reference P re f (t + T S ) and requires a Phase Locked Loop unit to track the voltage phase at the interface to the simulated electrical grid. Figure 8 illustrates the calculation method. Note that i re f (t + T S ) represents a sample of the current waveform and it is updated at every simulation time step T S .

Compensation Method
Limitations in terms of the Modbus communication rate and response dynamic of the converter represent the main factors of fidelity degradation. As it is not feasible to alter the response dynamic of the B2B converter, the compensation method utilised in this work aims at leveraging measurements available in the DRTS to compensate time delays. Power and voltage measurements of the B2B converter are utilised in the DRTS to estimate impedance of DL + Lines, which is used along with local voltage measurements in RTDS to correct power reference for current source in the DRTS.
The compensation method described above relies on following equations: In Equation (9), the conductance G DL+Lines of the HuT is estimated based on received measurements P b2b and V b2b . The reference for the current source for the next time step in the DRTS is corrected by ∆P(t + T S ) and calculated based on Equation (10).

Test System
The test system used in this study is a modified version of the IEEE 13-bus feeder, which was modelled in RSCAD and executed on a PB5-based RTDS simulator. It was interfaced with the HuT at the node 634 by the B2B converter connected to a controllable DL through a long electric cable as shown in Figure 9. To accommodate limitations of the laboratory setup, the original IEEE 13-bus feeder was modified as follows: 1. All loads of the system were balanced to account for the missing controllability of individual B2B converter phases. 2. The frequency of the main voltage source in the system was adjusted to 50 Hz to account for the frequency limits of the B2B converter. 3. A simple OLTC tap change control scheme was implemented based on [24]. 4. The load at branch 634 was replaced by an ideal current injection controlled by the IA in Figure 7. 5. Current injections based of measurements from the real-world DL were scaled by a factor of 40.
The simulation model contains an OLTC between busses 650 and 632. This OLTC controls the voltage in the distribution network to compensate voltage drops caused by varying the DL in the lab setup. As a constant impedance load, the DL in turn reacts on voltage step caused the tap change event. Hence, the OLTC and DL are a suitable to demonstrate the interactions between simulation and lab setup by a sequence of events propagating between the two sides.
The simple OLTC control was implemented inside the test system with a voltage bandwidth BW = 0.05 pu, a step size ∆V = 0.025 pu and a delay time t delay = 2 s [24].

Test Scenario
A scenario of a load step increase from 0 kW to 20 kW of the DL (0 kW to 0.8 MW in RTDS simulation) was investigated in this test. The DL's maximum rated power was specified with 78 kW. Even when operated at its maximum rating, the load change of the DL will not produce a significant disturbance in the grid. Therefore, a scaling factor of 40 was used for the power of load in RTDS to overcome the hardware capacity limitation. Firstly, open loop tests were implemented to characterise connection timing and hardware components (DL and B2B converter). Then, the proposed compensation method was evaluated via a closed loop.
This test scenario triggers a series events of which only this step change of the DL set point originates from the real-world lab setup. The step change of the DL causes a voltage dip in the test system and leads to the activation of the OLTC in order to restore the voltage level in the distribution network. The OLTC controller delays the activation of the tap change for about 2 s after the initial transient produced by the DL. The tap change operation produces as a consequence another transient, which now originates from the simulation and propagates to the real-world lab setup.
This test scenario was chosen as it contains events both originating from the lab setup as well as from the simulation. Hence, the the compensation method described in Section 4.4 could be evaluated for both cases.

Test Procedure
The target test case for this study reviewed the suitability of a grid-forming converter for QsPHiL by studying the closed-loop interaction between a real-world DL and a simulated OLTC in the test model. At the same time, it targeted the evaluation of the introduced compensation method.
For this test case, many test runs were performed to demonstrate the repeatability of the test setup. In addition, parameters such as the measurement sampling rate were varied to study their impact.
Before the closed-loop test, two open-loop tests were performed to characterise the B2B converter and DL step responses. For the remaining closed-loop tests, the following procedure was used:

Comparison Metrics
To evaluate the the fidelity and accuracy of the PHiL setup, several metrics were selected to quantify the improvement by the compensation method.
V overshoot a voltage overshoot which can be observed after the DL step. V undershoot a voltage undershoot which can be observed after the DL step. T s a settling time until convergence of interface quantities (voltage/power). max(∆E bal ) the energy balance before and after the event. It provides an indicator if the interface injects or absorbs energy. RMSD V the root mean square deviations (RMSD) between bus voltage at the interface V rtds and the reference case of a monolithic simulation of the full setup (no PHiL). RMSD P the root mean square deviations the active power measurement at the interface P rtds and the reference case.
For comparison, a monolithic simulation solely in the DRTS was performed. For this purpose, the laboratory was modelled by two constant impedance loads which were toggled between the on and off state of the DL and the respective steady state values seen by the converter in the real-world case. Initial open loop tests showed that these lines load the B2B with a significant amount of reactive power (≈ 1 kvar). The use of real-world steady state measurements instead of the known impedance of the DL for the monolithic simulation model was therefore motivated by the fact of other unknowns such as lines parameters connecting B2B and DL in the lab.
For the analysis, the monolithic simulation was used as the reference case for comparing the above mentioned metrics. Other metrics were compared only against other PHiL runs with different parameters such as the measurement sampling rate or (de)activated compensation.

Open-Loop Characterisation of Laboratory Hardware
To characterise the B2B converter, an open-loop test for a step-change in voltage from 400 V to 370 V with and without DL was examined. As can be seen clearly in Figure 10, the voltage response experiences various step changes before reaching the reference, which takes roughly t b2b ≈ 400 ms. This transient is represented by only 6-7 samples due to the limited sampling rate of the converter measurements. Similar results were reached for the inverse step as well as different step sizes. Overall, the voltage response of the B2B converter can be qualified as acceptable for the use in this study, due to the Quasi-stationary properties of the PHiL setup. Repeating similar tests with changes of the frequency set point show that the B2B converter reveals a slower response to frequency adjustments than voltage. Since the present scenario does not exhibit any frequency transients, the frequency of the B2B converter was fixed to f = 50 Hz. This simplification is permissible due the frequency invariance in the distribution grid caused by an ideal voltage source at the PCC. Besides, the lack of frequency set point updates saves an additional Modbus request and hence speeds up the SYSLAB loop.
Similarly, an open-loop test with a step change in the DL power from 0 kW to 20 kW at a constant voltage was investigated to characterise the DL. It takes approximately 100 ms for the DL relay response to hit the targeted set-point, which can impact the voltage response and compensation method in the closed loop. The results show that the B2B reacts immediately to this load change with only a negligible voltage dip. In addition, the constant impedance behaviour of the DL was verified during this open-loop test, as described in Section 4.
Especially for the fast DL step response, the sampling rate achievable over the B2B Modbus interface is hardly sufficient to reproduce the transient. These initial results necessitate the optimisation of the SYSLAB loop's Python code for a more efficient readout of measurements via the Modbus interface.
The internal design of the DL described in Section 4 suggests that the response of the DL should be an ideal step. However, Figure 11 shows two different variants of the DL step response which could be traced back to small timing differences in the switching behaviour of individual relays which activate the resistors. Out of 10 runs, five exhibit one-step response, while the others show the two-step response. This non-deterministic behaviour falsifies the closed-loop test runs. Therefore, subsequent test runs in which a two-step behaviour was detected were discarded from further analysis.
Another undesirable side-effect which was detected during open-loop characterisation tests is a an internal droop-control of the B2B converter. This droop control was subsequently disabled by reprogramming the converter control. The tests also showed internal limits for the voltage and frequency set points, which when exceeded trigger a safety shutdown of the converter. To avoid these shutdowns, signal limiters were added to the simulation model (48 Hz < f < 52 Hz, 360 V < V rms < 440 V). As a result of the converter's frequency limit, the IEEE 13-bus distribution network was modified to a 50 Hz system.   Step: 0 -> 0.7 MW

Closed-Loop Characterisation of Compensation Method
Step: 0 -> 0.7 MW Step: 0 -> 0.  A full version of the results can be found in Figure A1 in Appendix A. The scenario generates two different types of transients. The first transients induced by the DL step (t = 0 s ) propagates from the hardware setup to the simulation, whereas the two OLTC events (t = 2 s, t = 4 s ) propagate in the opposite direction. To discuss these two events, the results are represented as inset graphs and only OLTC tap change event following t = 2 s is shown.

DL Step Change
The initiating load change of the DL causes a voltage dip in the feeder, as shown in Figure 12.
The results show that the voltage response was improved by the compensation on the RTDS side ( vs.
). At the same time, the compensation caused no degradation of the response on the B2B side ( vs.
). Similarly, the overshoot in the power response on the RTDS side was improved by the compensation, as shown in Figure 13 ( vs. ). However, the respective undershoot in the power response on the RTDS side differs from the B2B side, and hence leads to a residual energy imbalance between the two ports. This energy imbalance is a natural consequence of improving power response on one side, but not the other.
In the case of the power response, the compensation failed to improve the initial DL step reaction, as the compensation distorts the current waveform. However, in the case of the OLTC tap changes, the compensation is capable of reducing the settling time (see Figure A1 t = 2 s and t = 4 s, compensated vs. uncompensated ).

OLTC Tap Change Event
In contrast to the DL step event, the OLTC tap change originates from the RTDS simulation as the simulated OLTC control gets activated by the lasting under-voltage contingency on the feeder. The response of voltage and power was improved by the compensation in the same way as for the preceding DL step response. In addition, the power response on B2B side was also improved by compensation.

Energy-Based Metric
In Figures A1 and 13, the energy balance on PHiL interface is shown as E bal . The energy balance accounts ingress and egress energy on both ports of the interface.
Notably, the DL step event results in a negative energy balance ∆E bal across the interface while the subsequent OLTC events produce a positive contribution ∆E bal and thereby absorb the generated energy of the DL step resulting in a more balanced outcome. However, this fact is only a coincidence caused by the present scenario. In the notation used here, a negative balance is caused by the interface injecting additional energy into the system, while a positive balance indicates absorption of energy (see Equation (11)).
The constant decline of E bal is an indicator for a small steady state error between simulation and laboratory signals. The step at t = 0 s shows the absorption of energy during the transient.

Impact of Reduced Hardware Sampling Rate
To further investigate the influence of the measurement sampling rate, a series of tests was conducted with reduced rate. For the decimation factor n ds , values 1, 2, 5, 7 and 10 were chosen, which reduces the sampling rate by a factor of n ds . Figure 15 shows the impact of the sampling rate reduction using the same scenario as in the previous sections. All runs were aligned at t = 0 to the drop of the simulated bus voltage V rtds . This choice for alignment may be debatable, as we do not align the results to the DL step which triggers the event as in Section 5.6. However, with higher decimation ratio, the the sampling instant of the measurements differ more and more from the time of the DL step. Therefore, runs are not comparable due to the random alignment of DL step and sampling. With reduced sampling rates, we see a larger time lag before the first response in the simulation. For example, for n ds = 10, the effective sampling rate is reduced to around 1.6 Hz, which in the worst case could delay the propagation of the DL step for up to 0.625 s. Step: 0 -> 0.7 MW Step: 0 -> 0.7 MW Step: 0 -> 0.7 MW Step: 0 -> 0.7 MW Step: 0 -> 0.7 MW Step: 0 -> 0.7 MW Step: 0 -> 0.7 MW Step: 0 -> 0.7 MW Step: 0 -> 0.7 MW Step: 0 -> 0. 7 Figure 15. Voltage response of DL step at PHiL interface with varying sampling rates. Table 1 lists the key metrics for the different decimation rates. The results reveals two trends: • The compensated runs show better metrics for all decimation ratios. • For most runs, a higher decimation ratio yields worse results.
The convergence time T s is around 1.4 s to 1.5 s. No correlation of n ds or the compensation is visible.

Conclusions
The results demonstrate the feasibility of Geographically Distributed Power Hardware in the Loop experiments using a commercial grid-forming ship-to-shore converter. A first PHiL experiment was conducted in DTU's SYSLAB smart-grid laboratory with support of a RTDS real-time simulator located at a 40 km distant DTU campus.
A set of open-loop tests characterises the controllable dump load (DL) and converter for the application in the GD-PHiL setting.
The Modbus interface of the converter was optimised for a high throughput of measurement readouts and set point updates. However, the Modbus interface remains the main bottleneck for more frequent updates between simulator and the internal converter control. The maximum update rate of 16 Hz contributes significantly to the overall delay of the loop and directly limits the sampling rate for acquisition of interface quantities.
For similar low-cost PHiL setups, the authors suggest the enhancement of existing power converters with dedicated measurement equipment to achieve a higher sampling rate and reliable time stamping. A dedicated measurement setup would in turn allow for more frequent set point updates in the converter, as the Modbus interface is relived from the pressure of retrieving measurements. At the same time, set point updates of the converter could be synchronised to the real-time simulator.
The compensation method used in this study attained a fidelity improvement in the interface signals by reducing voltage over-and undershoot. However, after evaluation of the energy balance, it becomes evident that it failed to improve the conservation of energy across the interface. The residual energy imbalance indicates that a complementary compensation (e.g., based on an Integral controller) may be able to improve response.
Eventually, the chosen compensation method depends on the objective of the PHiL experiment. In the present study, the fidelity of the voltage signal is prioritised, while other studies may focus on more long-term phenomena in which an accurate reproduction of the exchanged energy is of capital importance (e.g., battery energy storage systems). The studied power system in this paper is a common multi-bus low voltage distribution grid feeder. Modern distribution grids are however increasingly characterised by the presence of renewable energy generation (REG). Future work should therefore target new test-cases demonstrating the penetration of REG using the presented QsPHiL coupling methodology. New REG components offering new control functionalities requires improved management of distribution grids to ensure power quality and avoid stability issues. Thorough testing of such components must go beyond the common component level PHiL testing and could be realised by PSiL testing. An example for such a test case could be multi-level voltage control or efficient system integration of REG for electric vehicle (EV) charging.
The test case in this paper is characterised by a uni-directional power flow at the PHiL interface, in which the B2B converter only sources power to the Hardware-under-Test (HuT). Future test-cases might also result to a reversed power-flow at the PHiL interface. The employed power interface in the present laboratory setup could support this scenario due to its back-to-back topology.
Tests with a reduced sampling rate showed a reduced effect of the compensation method with lower sampling rates. Hence, it is inferred that higher sampling rates will yield an increasing improvement by the compensation. Funding: This work received funding from the European Regional Development Fund (EFRE-0500029) and the European Union's Horizon 2020 research and innovation programme under grant agreement No 654113.

Acknowledgments:
The authors would like to thank the ERIGrid Transnational Access exchange program for enabling this work by providing access to DTU's Research Infrastructure and technical support.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript: Eintf Eintf (comp.) Figure A1. Closed-loop test results.