Evaluation of the Communication Delay in a Hybrid Real-Time Simulator for Weak Grids

Abstract

Real-time Simulation (RTS) is one of the effective means via which to study device level or system level dynamics, such as power converter online testing, evaluation, and control, and power system stability analysis. The RTS -enabled design-chain offers a time -effective, low-cost, and fail-safe development process. As the penetration of renewable energy is becoming higher, the demand in hybrid system real-time simulation becomes imperative, where fast-dynamic device level power converters and slow -dynamic large -scale power systems are simulated at the same time. This paper introduces a novel hybrid real-time simulation architecture based on the central processing unit (CPU) and the field-programmable gate array (FPGA). Compared with the off-the-shelf power system real-time simulation system, it offers both wide time scale simulation and high accuracy. The multi-time scale model can perform electromechanical electromagnetic transient hybrid simulation, which can be applied to the research of power systems penetrated with power converters. In the proposed simulation platform, the communication delay is introduced when different RTS platforms exchange real-time data. The communication delay should be considered in the stability analysis of the grid-connected inverters in a weak grid environment. Based on the virtual impedance characteristic formed by the control loop with and without communication delay, the impedance characteristics are analyzed and inter-simulator delay impacts are revealed in this paper. Theoretical analysis indicates that the communication delay, contrary to expectation, can improve the virtual impedance characteristics of the system. With the same hardware simulation parameters, the grid-converter system is verified on both the Typhoon system alone and the Typhoon-dSPACE-SpaceR hybrid simulation platform. The THD value of grid current in a weak grid environment that works in the Typhoon system is 4.98%, and 2.38% in the Typhoon-dSPACE-SpaceR hybrid simulation platform. This study eventually reveals the fact that the inter-simulation delay creates the illusion that the control system built in the novel hybrid real-time simulation is more stable under weak grid conditions.

1. Introduction

Nowadays, energy demand and environmental problems are becoming more and more compelling. This gives tremendous momentum in renewable energy technology development. With the increasing penetration of renewable energy, the power quality of renewable power generation has received extensive attention, which brings about higher requirements for accurate power regulation and faults diagnosis capabilities [1]. grid-tie inverters are the main equipment used to connect distributed generation systems with the conventional utility grid [2,3]. Therefore, in order to accurately control output power, it is necessary to improve the efficiency of renewable energy grid-connected devices and the quality of grid-connected currents.

Real-time simulation is an important tool used to study power electronics and power systems, which enables graphical modeling, efficient emulation, and friendly parameter tuning. Through RTS -based debugging and observation, researchers can quickly and intuitively obtain the electromagnetic transient behavior characteristics of power electronic equipment and power systems under steady-state and transients, which is helpful in expediting control algorithms and protection strategy verification [4,5,6]. Off-the-shelf real-time simulation of power systems is mostly based on high-performance computers or computer clusters, where the computing unit mostly consists of CPU clusters. With the expansion of microgrid and the penetration of power electronic equipment in the power system, the computation power of CPUs does not match the requirements of the high-frequency switching action in power electronic simulation [7,8]. Compared with a CPU, field-programmable gate array (FPGA) has strong computing power and low communication delay. It has been extensively used in power electronic real-time simulation and has attracted more and more attention [9,10,11].

Presently, commercial and research real-time simulator computation cores include the following implementations. The first category is CPU -based; The featured commercial simulators include RT-LAB [12] and HYPERSIM [13] developed by OPAL-RT company, ARENE [14] developed by EDF (Electricite De France), NETOMAC developed by Siemens, and ADPSS developed by China Electric Power Research Institute. The other category is FPGA based; The benchmark product developed by Typhoon Inc. and MIT belongs to this category and is widely used in switching converter real-time simulation.

It can be observed from

Table 1. Evaluation of different hardware architectures.

Hardware Architecture	Advantages	Disadvantages
CPU	Flexible solving algorithm Complex solver Easy coding	Large I/O latency Low sampling rate
FPGA	Low I/O latency High sampling rate Flexible parallelism	Weak in sequential computing algorithms (math operations) Inherent fixed-point numerical representation Large coding effort and longtime bitstream generation
CPU-FPGA	Efficient computing resource partition Benefit from both sequential computing and high parallelism	Large communication latency Model numerical stability issue

that real-time simulators with CPU as the only computing unit offers a flexible, nonlinear solving capability and easy coding. While the switching power converter is connected to the power grid, it will inject high -frequency voltage/current perturbation to the power system, and will bring a huge computational burden to the CPU -based platform. FPGA can fill the gap of real-time simulation in such scenarios. The CPU-FPGA hybrid real-time simulator combines the advantages of the two, and it also induces the communication delay into the model. Therefore, it is urgent to launch a hybrid real-time simulator based on CPU and FPGA in the market, and to evaluate the difference between the performance of the simulation model and the real plant.

The research on hybrid simulation technology of power systems started more than 40 years ago. The first hybrid simulation technology was NETOMAC, developed in Germany by Siemens in the 1970s [15]. Parallel algorithms are of great significance for accurate real-time simulation of a large-scale power system. Therefore, network segmentation methods of various computer cluster parallel algorithms have been produced. In [16], cluster parallel technology is used to analyze the sensitivity of the transient process, and the Jacobian matrix obtained from the simultaneous iterative solution of differential equations by Newton method is used to reduce the simulation calculation time; Ref. [17] uses the time division method to assign different time points or time steps to different computers for calculation at the same time; Ref. [18] uses the conjugate gradient iterative method to solve the linear equation in parallel, and the operation effect is good; Ref. [19] uses the Newton GMRES method to carry out parallel computing on heterogeneous computers with strong expansibility, and uses adaptive variable order prediction and multi-step coordination method to improve the efficiency of parallel computing. The research team, led by Snider and Kevin from Hong Kong Polytechnic University, researched hybrid simulation technology and realized the docking of SVC electromagnetic transient simulation program with conventional electromechanical transient simulation programs in [20]. Ref. [21] proposes a parallel interface data interaction protocol. In [22,23,24], the problems of parallel computing and real-time in a hybrid simulation are analyzed. Ref. [25] focuses on the hardware-in-the-loop (HIL) real-time simulation system of variable speed hydro-electric plants (VS-HEP), and a hydraulic modeling method using mathematical optimization programs to analyze and obtain the hydraulic static model is introduced. Ref. [26] introduces the latest developments of RTDS Technologies Inc., including the growth of high-performance processing space and the research of high-end processors in embedded design. For large-scale power system simulation, Ref. [27] introduces a hybrid real-time simulation tool combining real-time simulators and transient stability programs. The simulation results are similar to those of a large-scale real-time simulator, which verifies its feasibility. Ref. [28] introduces the latest improvement of an open-source real-time simulator DPsim, which can combine the characteristics of traditional vector and electromagnetic transient simulations.

Inspired by the above idea, a hybrid real-time simulator architecture is proposed in this paper, which combines the CPU-based and FPGA-based simulators. In the simulation system, the communication delay is introduced when different RTS platforms exchange real-time data. The communication delay cannot be ignored on the simulator during the stability analysis of the grid-connected inverters in a weak grid environment. The authors have investigated a grid-connected inverter control that uses grid voltage feedforward and PLL control method in this simulation platform, and verified the negative effects of the communication delay on the stability of the system [29].

In this paper, the impedance characteristics of a grid-tie inverter will be investigated in a proposed hybrid real-time simulation platform. Both theoretical and experimental investigations will be carried out to evaluate the impact of the inter-simulator delay to the control design of the inverter. Potential design pitfalls when using a hybrid simulation platform will also be revealed.

2. The Real-Time Simulation Configuration

The study of power electronics penetrated power systems demands mixed small time step switching systems and large complex electric networks simulations, thus it is often necessary to use multi-core CPU or FPGA hardware with strong computing power for calculations. The simulation scale and computing power of the real-time simulation system depend on the amount of hardware resources of the simulator. In this section, a mixed simulation platform is proposed.

2.1. Platform Framework

The CPU-based RTS system SPACER, a real-time simulation system with enhanced computation power, is shown in Figure 1. The platform can directly load the mathematical model of the dynamic system established in MATLAB/SIMULINK for simulation, control, and testing. Each RTS has up to 12 CPU cores, realizing multi-task parallel processing. Meanwhile, the target machine has an integrated PCI-E expansion Bus for expanding multiple I/O expansion boxes.

Another RTS system is based on FPGA with a dedicated model library; this system is called Typhoon (a Hardware-In-Loop system for power electronics), as shown in Figure 2. It offers a complete set of solutions for engineering projects, such as power electronics system design, verification, real-time simulation, and testing.

Table 2. Performance Parameters of Real-Time Simulator.

	SPACER	Typhoon HIL402
Computing unit	CPU	FPGA
Processor	12 cores	4 cores
Analog I/O	16 Analog inputs	16 Analog inputs
	16 Analog outputs	16 Analog outputs
Digital I/O	32 Digital inputs	32 Digital inputs
	32 Digital outputs	32 Digital outputs
Resolution	16 bit	16 bit
Connectivity	Ethernet	USB/Ethernet
Sample rate	1 MSPS	1 MSPS
Real-time simulation step	20 μs	0.5, 1 or 2 μs

shows the specifications of SPACER and Typhoon. As demonstrated, the SPACER only uses the CPU as the computing unit, and, thus, large time scale power grid models can be built into this simulator. Power electronics simulation models are built in the Typhoon HIL. The dSPACE RTS is a real-time development and testing platform of control systems, which is seamlessly connected with MATLAB/Simulink. The architecture of the proposed simulator is shown in Figure 3.

2.2. Data Interaction between the Platform

In the proposed simulator, there are two data interaction modes: the ethernet data communication and the I/O port communication, as shown in Table 2.

(1)

Ethernet data communication

Ethernet is a baseband LAN technology. Its maximum data transmission rate is 1GBIT/s, which cannot support sustainable network data transmission. Its communication protocols include TCP and UDP, and their characteristics are shown in

Table 3. Specification comparison between TCP and UDP.

	TCP	UDP
Definition	Transmission Control Protocol	User Datagram Protocol
Advantages	Connectionless Low resource overhead Fast transmission speed	Reliable transmission No error Mass data transmission Flow control
Disadvantages	Unreliable, easy to loss packets No flow control	Slow transmission speed High resource overhead

.

In this study, the TCP protocol is adopted when testing the data communication function. The single simulator data transmission test results are shown in Figure 4, where the data are transmitted without omission. The results show that the data cannot be well transmitted between two simulators (SPACER and Typhoon HIL). The primary reason for this can be that the simulators are produced by different manufacturers with protocols that differ from the TCP protocol, thus, the performance and reliability of the real-time simulator is significantly reduced.

(2)

I/O port data communication

Analog port communication is much simpler. As shown in Table 2, both Typhoon HIL and SPACER have 16 analog output ports and 16 analog input ports, and have the same analog signal transmission range of −10~10 V. Since the signal resolution is 16 bits, the theoretical error can be calculated. As can be seen in the experimental verification result shown in Figure 5, the actual signal error is about 0.1%, no data are missing, the waveform is not distorted, and it can meet the engineering requirements. However, the analog port communication will bring about a new type of communication delay at the microsecond level.

After the above comparison of the data error caused by two different data interaction modes, analog port communication is applied in this paper for experiments, and a series of theoretical research and experiments are carried out to verify the impact of communication delay.

3. Influence of Communication Delay in Weak Grid

3.1. Modeling of Grid-Connected Inverters

This proposal focused on the analysis of a three-phase LCL-typed grid-connected inverter, as shown in Figure 6. the control system of a grid-connected inverter usually adopts grid voltage feedforward [30,31,32] to enhance the dynamic response performance of the grid-connected inverter under the situation of grid voltage disturbance, and to suppress the grid current harmonics caused by grid voltage harmonics. The phase-locked loop (PLL) is applied to obtain the real-time grid voltage phase. Here, the LCL filter includes the converter side filter inductance L₁, filter capacitor C, and the grid side filter inductance L₂. U_dc is the DC-link voltage. U_pcc is the PCC voltage, which is the synchronous reference voltage of the PLL. Based on this structure, a weak grid can be denoted by the Thevenin equivalent circuit, where an ideal voltage source U_g is connected in series with the grid impedance L_g. Grid current I_g is under close-loop control, and the current reference is directly given by i_dqref. The capacitor-current-feedback for active damping is applied to suppress the LCL filter resonance peak, where K_c is the feedback coefficient of capacitive current.

Figure 7 shows a typical control structure of grid current feedback. U_pcc can be replaced by $u_g+s{i}_g{L}_g$. According to the mason transformation rule, Figure 7a can be transformed into Figure 7b through a series of equivalent transformations and simplifications. The detailed transformation process is presented in Appendix A. The open-loop transfer function of the system can be calculated as Equation (1):

$$G_o(s)=\frac{G_c(s)G_d(s)}{s(L_2+L_g)(s^2{L}_{1}C+s{L}_1/Z_{eqc}+1)-s{L}_g{G}_d(s)G_f(s)G_{td}(s)+s{L}_1},$$

(1)

where, G_c(s) is the inductor current controller, also known as the proportional-integral (PI) controller, which can be expressed as:

$$G_c(s)=K_p+\frac{K_i}{s},$$

(2)

where K_p and K_i is proportional gain and the integral gain respectively.

G_d(s) is the controller delay, and G_td(s) is the communication delay caused by real-time data interaction between the simulators. They can be expressed as [10]:

$$G_d(s)=e^{-1.5T_{s}s},$$

(3)

$$G_{td}(s)=e^{-T_{td}s},$$

(4)

Overall, 1.5T_s is total delay time caused by sampling and calculation process of PWM while T_td is data interaction time on the platform.

3.2. Virtual Impedance Characteristic of Control Loop with Communication Delay

3.2.1. Virtual Impedance Characteristic of Capacitive Current Feedback Control

As shown in Figure A1b of Appendix A, capacitive current feedback control forms a virtual equivalent impedance Z_eqc in parallel with the filter capacitive C. It can be formulated as:

$$Z_{eqc}=\frac{L_1}{K_{c}C{G}_d}=\frac{L_1}{K_{c}C}{e}^{1.5T_{s}s}=R_{eqc0}(\cos (1.5T_s\omega )+j\sin (1.5T_s\omega )),$$

(5)

where R_eqc0 is the virtual equivalent resistance of capacitive current feedback without digital control delay, as

$$R_{eqc0}=\frac{L_1}{K_{c}C},$$

(6)

When there is a digital control delay, the Z_eqc can be expressed as the virtual equivalent resistance R_eqc and reactance X_eqc in parallel.

$$R_{eqc}=\frac{R_{eqc0}}{\cos (1.5T_s\omega )},$$

(7)

$$X_{eqc}=\frac{R_{eqc0}}{\sin (1.5T_s\omega )},$$

(8)

3.2.2. Virtual Impedance Characteristic of Grid Voltage Feedforward Control

The actual voltage feedforward is the U_pcc, which contains the ideal grid voltage u_g and capacitance voltage u_c, and can be calculated as follows,

$$U_{pcc}=\frac{L_g}{L_2+L_g}{u}_c+\frac{L_2}{L_2+L_g}{u}_g,$$

(9)

Notably, as shown in Figure 6, the novel simulator platform proposed in this paper causes a communication delay G_td. Thus, it can be found that the capacitor voltage part of the feedforward quantity in Figure 8a is equivalent to forming an equivalent virtual impedance Z_eqv in parallel with the output filter capacitor as follows,

$$Z_{eqv}=-\frac{s{L}_1(L_2+L_g)}{L_g{G}_f{G}_d{G}_{td}}=X_{eqv0}(-\sin \left[(1.5T_s+T_{td})\omega \right]+j\cos \left[(1.5T_s+T_{td})\omega \right]),$$

(10)

In which, X_eqv0 is the equivalent virtual reactance formed by grid voltage feedforward control when there is no digital control delay and the communication delay, and its calculation result can be expressed as,

$$X_{eqv0}=-\frac{\omega L_1(L_2+L_g)}{L_g{G}_f},$$

(11)

When there is a delay of digital control and transmission line, Z_eqv can be expressed as a virtual equivalent resistance R_eqv and a virtual equivalent reactance X_eqv in parallel, and can be respectively calculated as follows,

$$\begin{array}{l}{R}_{eqv}=-\frac{X_{eqv0}}{\sin \left[(1.5T_s+T_{td})\omega \right]}\end{array}$$

(12)

$$X_{eqv}=\frac{X_{eqv0}}{\cos \left[(1.5T_s+T_{td})\omega \right]},$$

(13)

It can be seen from the above expression that Z_eqv is affected by the grid inductance L_g. Under the condition of a strong power grid, Z_eqv tends to be infinity, thus, the impact on system stability caused by Z_eqv can be ignored. However, under the condition of a weak current grid, Z_eqv is a finite value, and the impact cannot be ignored. Therefore, this research method is only suitable for weak power grid conditions.

3.2.3. Virtual Impedance Characteristic of Control Loop

Based on the analysis above, the virtual impedance characteristics of capacitive current feedback and grid voltage feedforward are all affected by the digital control delay, however only the virtual impedance characteristics of grid voltage feedforward are affected by the communication delay. For further analysis, the effects of digital control delay and communication delay on the virtual impedance characteristics will be studied separately.

Assuming that the communication delay can be ignored, the virtual impedances of grid voltage feedforward and capacitive current feedback are:

$$Z_{eqv}=X_{eqv0}(-\sin (1.5T_s\omega )+j\cos (1.5T_s\omega )),$$

(14)

$$Z_{eqc}=R_{eqc0}(\cos (1.5T_s\omega )+j\sin (1.5T_s\omega )),$$

(15)

Therefore, the equivalent virtual impedance Z_eq0 without digital control delay can be expressed as:

$$Z_{eq0}=\frac{R_{eqc0}{X}_{eqv0}}{R_{eqc0}+X_{eqv0}},$$

(16)

From Equation (16), the equivalent virtual impedance is determined not only by resistance R_eqc0, but also by reactance X_eqv0. The X_eqv0 is capacitive reactance, which has a low-frequency offset effect on the system resonant frequency and is related to grid impedance L_g and coefficient of grid voltage feedforward G_f.

Then, the equivalent virtual impedance Z_eq with digital control delay can be expressed as an equivalent resistance R_eq and an equivalent reactance X_eq in parallel, calculated as follows,

$$Z_{eq}=\frac{Z_{eqc}{Z}_{eqv}}{Z_{eqc}+Z_{eqv}}=\frac{R_{eq}{X}_{eq}}{R_{eq}+X_{eq}},$$

(17)

$$R_{eq}=\frac{-R_{eqc0}{X}_{eqv0}}{R_{eqc0}\sin (1.5T_s\omega )-X_{eqv0}\cos (1.5T_s\omega )},$$

(18)

$$X_{eq}=\frac{R_{eqc0}{X}_{eqv0}}{R_{eqc0}\cos (1.5T_s\omega )+X_{eqv0}\sin (1.5T_s\omega )},$$

(19)

By taking the communication delay in consideration, the virtual impedances of grid voltage feedforward and capacitive current feedback are:

$$Z_{eqvtd}=X_{eqv0}(-\sin ((1.5T_s+T_{td})\omega )+j\cos ((1.5T_s+T_{td})\omega )),$$

(20)

$$Z_{eqc}=R_{eqc0}(\cos (1.5T_s\omega )+j\sin (1.5T_s\omega )),$$

(21)

Thus, the equivalent virtual impedance Z_eq can be expressed as an equivalent resistance R_eq in parallel with an equivalent reactance X_eq as,

$$Z_{eqtd}=\frac{Z_{eqc}{Z}_{eqvtd}}{Z_{eqc}+Z_{eqvtd}}=\frac{R_{eqtd}{X}_{eqtd}}{R_{eqtd}+X_{eqtd}},$$

(22)

$$R_{eqtd}=\frac{-R_{eqc0}{X}_{eqv0}}{R_{eqc0}\sin ((1.5T_s+T_{td})\omega )-X_{eqv0}\cos (1.5T_s\omega )},$$

(23)

$$X_{eqtd}=\frac{R_{eqc0}{X}_{eqv0}}{R_{eqc0}\cos ((1.5T_s+T_{td})\omega )+X_{eqv0}\sin (1.5T_s\omega )}.$$

(24)

The frequency response of Z_eq and Z_eqtd is shown in Figure 9, where f_s is the sampling frequency. The frequency characteristics of equivalent resistance and reactance are affected by grid impedance L_g, grid voltage feedforward coefficient G_f, capacitor current feedback coefficient K_c, digital control delay 1.5T_s and communication delay T_td.

The communication delay on the frequency characteristics of equivalent resistance and reactance will be illustrated as follows. It can be seen from the frequency response results (Figure 9) that after introducing communication delay, the boundary frequency and amplitude of equivalent resistance and reactance change. The boundary frequency of X_eqtd is shifted to low frequency by 25 Hz; The positive resistance range of R_eqtd becomes smaller, which can render the system unstable.

It can be concluded that the communication delay has an impact on the system stability whilst the impact caused by the grid impedance is strong. Meanwhile, it can be seen from Figure 10 that the weaker the power grid, the larger the damping range of the equivalent virtual resistance of the control loop, and the resonant damping effect of the control loop can be effectively improved.

4. Simulation and Experimental Verification

To verify the feasibility of the virtual impedance characteristic analysis above, the simulation models, as shown in Figure 4, are built into the proposed hybrid real-time simulation system. Meanwhile, the switching frequency of the inverter is 10 kHz. A filter with a cut-off frequency of 500 Hz is selected. Based on the design equation as in (25), the filter inductance and capacitance are selected to be 8 mH and 10 uF, respectively.

$$10f_o\le f_c\le f_{sw}/6,$$

(25)

The main parameters are shown in

Table 4. Specifications of the Grid-tie Inverters.

Parameter	Symbol	Value
Grid phase-to-phase voltage	Ug	380 V
Grid frequency	fg	50 Hz
DC-link voltage	Udc	700 V
Inverter side inductance	L1	8 mH
Filter capacitance	C	10 μF
Grid impedance	Lg	0.5 mH
Switching frequency	fsw	10 kHz
PI controller of PLL	Kp_PLL	100
PI controller of PLL	Ki_PLL	3200
PI controller of current control	Kp_Curr	0.018
PI controller of current control	Ki_Curr	1

.

The simulation results of communication delay are shown in Figure 11. The model established in Typhoon HIL obtains the power grid current waveform under both strong grid and weak grid conditions. As can be observed in Figure 11, when the grid impedance becomes larger, the grid current waveform worsens.

To compare the quality of the grid-connected current, Figure 12 shows the harmonic distortion rate of the grid-connected current. The fast Fourier transform (FFT) value with L_g = 1 mH is 4.98% and with L_g = 0 mH is 3.27%; here, the former is obviously larger than the latter.

In order to compare these results with the simulation results in Typhoon HIL, where only control delay T_d persists, the model is re-implemented in the proposed hybrid real-time simulator. The platform is shown in Figure 13. The power grid model is built by SPACER, while Typhoon HIL provides the inverter model. the simulation results of the inductor current waveform with and without grid impedance are shown in Figure 14a and Figure 14b, respectively. The FFT value with L_g = 1 mH is 2.36%, and with L_g = 0 mH is 6.94%, as shown in Figure 15a and Figure 15b, respectively. Here, the former is obviously smaller than the latter. The result is completely opposite to the previous results obtained on a single simulator, as shown in Figure 12.

Therefore, the above theory can be successfully verified by comparing the simulation results on different platforms. The influence of communication delay between platforms cannot be ignored, and the model of the hybrid real-time simulator under a weak grid can improve the power grid current waveform. the weaker the power grid, the smaller the current waveform harmonic, and the greater the quality of the waveform.

5. Conclusions

A hybrid real-time simulation platform is proposed in this paper, combining the computation engines of CPU and FPGA, which solves the challenges of co-existing simulation time scale and simulation accuracy. The communication delay of the hybrid real-time simulator impacts the system design. Through theoretical analysis and experimental verification, it is revealed that, in the real-time simulation environment, the communication delay will, surprisingly, improve the stability of grid-connected inverters connected to a weak grid. Therefore, designers who works with hybrid simulators without a synchronized system timing/clock can pay attention to this illusive phenomenon and take corresponding measures to ensure seamless design transition from RTS to physical plants.