Novel Neural Control of Single-Phase Grid-Tied Multilevel Inverters for Better Harmonics Reduction

Xingang Fu 1,*, Shuhui Li 2, Abdullah Al Hadi 1 and Rajab Challoo 1 1 Department of Electrical Engineering and Computer Science, Texas A&M University-Kingsville, Kingsville, TX 78363, USA; abdullah_al.hadi@students.tamuk.edu (A.A.H.); Rajab.Challoo@tamuk.edu (R.C.) 2 Department of Electrical and Computer Engineering, The University of Alabama, Tuscaloosa, AL 35401, USA; sli@eng.ua.edu * Correspondence: Xingang.Fu@tamuk.edu; Tel.: +1-361-593-4575


Introduction
The distributed power generation system [1,2] has proven to be a very useful way of using renewable sources in small and large-scale energy production.Grid-tied inverters are widely used in renewable energy integration, e.g., solar power, fuel cell, and wind energy.With the increasing interest in renewable energy generation system, the demand for high power integration to the grid from these renewable energy sources increases.However, the voltage ratings of the currently available power electronic devices are still not sufficient for medium and high voltage grid applications.For example, a 6.5 kV rated device is currently available in the market while the 10 kV rated SiC IGBT and MOSFET are still under development of the research facilities [3].Hence, the conventional two-level design of the converter is not feasible for medium to high voltage applications.
Multilevel converters have been shown to be a very promising and reliable solution to connect medium/high voltage level distribution grids by eliminating the transformer and reducing the system's cost.Moreover, the high switching frequency feature helps to reduce the harmonic content of the inverter waveform.The converter power ratings can be increased by adding the number of output levels without increasing the power or voltage ratings of the switching devices which is not possible with conventional two-level topology due to the voltage and current limitation of each individual power device.Furthermore, as the multilevel converters generally offer lower Total Harmonic

Grid-Tied Multilevel Inverter
The schematic diagram of a grid-tied multilevel inverter is shown in Figure 1.In order to keep the harmonics as low as possible, an LCL filter is placed between the inverter and the grid.
A cascaded H-bridge (CHB) voltage source converter was adopted to design the grid-tied multilevel converter.Two identical H-bridge converters are connected in series to generate five-level voltage waveforms.Figure 2 shows the schematic diagram of the single-phase five level cascaded H-bridge (CHB) converter, in which the two legs of the upper bridge are denoted as Leg 1 and Leg 2, and the two legs of the lower bridge are denoted as Leg 3 and Leg 4. V DC1 and V DC2 represent the DC voltages across two capacitors, respectively.V DC stands for the combined CHB DC voltage.In general, for a number (N) of series-connected H-bridge modules, (2N + 1) number of output voltage levels will be generated.One individual module of the H-bridge converter generates three different voltage outputs: V DC /2, 0, and −V DC /2.When two H-bridge modules are connected in series, the combination of those two modules gives output voltages on five levels, which are V DC , V DC /2, 0, −V DC /2, and −V DC .

Imaginary Circuit
For a single-phase grid-tied inverter, an imaginary circuit is needed to create another phase for the implementation of the d-q vector control.The created imaginary circuits have the same amplitude as the real circuits with π/2 phase shift in all quantities.The real circuit and imaginary circuit together constitute the α-β frame of the LCL system.The system equations in the α-β frame can be further transferred into the d-q domain by a transformation matrix, T, as shown in Equation (1) [18]: Also, the inverse transformation matrix from the d-q frame to the α-β frame is just inverse of matrix T [18].

Mathematical Model
Using the current direction defined in Figure 1, the system Equation (2) of the real circuit describes the system equations of the single-phase LCL filter based grid-tied inverter system, where R g and L g stand for the resistance and inductance of the grid-side inductor; R i and L i represent the resistance and inductance of the converter-side inductor; C and v cv specify the parallel capacitor and the capacitor voltage; and v g , v i , i g , and i cc define the grid voltage, the converter voltage, the grid current, and the converter-side current, respectively.The equations described in Equation ( 2) correspond to the α-axis quantities.Accordingly, the Equation (3) describing the imaginary circut are built into the β-axis domain: where, v * g , v * c , v * cv , i * g , and i * cc stand for those corresponding real circuit quantities in the imaginary circuit, respectively.Specifically, , and i * cc = i cc e −π/2 .

PI-Based Vector Control
For an LCL filter-based grid-tied inverter system, one simple method is to omit the parallel capacitance, and thus the LCL filter will be simplified into the L filter [19].
By omitting the capacitance impact and using the α-β frame, Equations ( 2) and (3) of the single-phase LCL grid-tied inverter system are simplified in Equation (4): Further, applying the transformation matrix T to Equation (4) gives the Equation ( 5) in the d-q frame for the design of the current loop controllers: The tuning of PI controllers is based on v d and v q in Equation ( 5) for the dand q-axis control loops with all the other terms considered to be the compensation items.The corresponding relationships between the α-β domain and the d-q domain are listed as follows: i g , i * g ↔ i d , i q , v g , v * g ↔ v d , v q , and v cv , v * cv ↔ v d1 , v q1 .However, this simplification utilizes an imprecise description of the system and could cause potential oscillatory and/or unstable dynamic behavior.To overcome this problem, the LCL filter or the designed controller needs to be properly damped [20,21].
The PI controller for current loop control is tuned by the plant transfer function, 1/[(R i +R g ) + (L i + L g )s].The control structure for the closed-loop model is shown in Figure 4. To tune the PI controller, the phase margin was selected as 60 deg with the bandwidth set as 1500 rad/s.This selection tended to generate the best results considering the Pulse Width Modulation (PWM) saturation constraints in our tests.In order to solve the resonance phenomenon associated with the LCL filter, an active damping method was adopted in this paper [22] as such methods do not consume extra power like passive damping methods.A low-pass filter was added to the output of the current-loop controllers [23] at the resonance frequency, as shown in Equation ( 6) [24]: The outer DC-link voltage control loop was designed based on the principle of power balance [25] and the tuning process for the outer voltage controller had the similar process as that for the current-loop controllers.

Modulation Technique
Different modulation techniques are compatible for different converter topologies.PWM exhibits one of the most promising for multilevel converters by also applying some changes in carrier generation.
Multilevel converters require a multicarrier technique for the PWM method since these converters have more legs than conventional two-level converters.
For phase-shifted modulation, a CHB multilevel converter containing N cells will need a carrier phase shift of 180 • /N [26].For five-level converters, the total number of cell required is 2, which indicates that a phase shift of 180 • /2 = 90 • is required for phase shifting carrier signal modulation.Carrier signals are shifted 90 • from one bridge to another.

Current-Loop Neural Network Controller
In contrast to using the NN for classification [27] and forecasting [28], the NN was adopted as the controller directly in the closed-loop inverter control systems in this paper.The proposed NN controller has an input preprocessing block and a feed-forward network as shown in Figure 7.To avoid input saturation, the input preprocessing block regulates the inputs into the range [−1, 1].The outputs of the input preprocessing block are tanh( − → e dq /Gain) and tanh( − → s dq /Gain2), in which − → e dq is the error term and is defined as As shown in Figure 7, the feed-forward NN has two hidden layers with six nodes in each hidden layer.The NN outputs two d-q voltage control signals, −→ v * dq1 .The hyperbolic tangent functions (tanh) are adopted as activation functions for all neurons.A two-hidden-layer NN generally yields a stronger approximation ability [29] than a one-hidden-layer NN, and the training of a two-hidden-layer network is relatively easier than that for a network with three or even more hidden layers.The number of neurons in each hidden layer was selected by a trial-and-error method [16].
If expressing the NN controller in Figure 7 in a function form, −→ v * dq1 =R( − → e dq , − → s dq , − → w ), and considering the pulse-width-modulation (PWM) gain, k PW M [30], of the converter system, the control action, −→ v dq1 , can be represented as

Training Neural Network Controller
The principle of Bellman's optimality is widely employed [10] in Dynamic Programming (DP) and is a useful method for solving optimal control problems [12].
The DP cost function for the NN training is defined as in which, γ is a discount factor, and the range of γ is 0 < γ ≤ 1. U(•) is often called the utility or local cost function.The DP cost function, C dp , is referred to as the cost-to-go of initial state, − → i dq (j), in which j > 0. The goal of the NN training is to minimize the DP cost, C dp , in Equation (8).
To train a moderate number of network parameters, the Levenberg-Marquardt (LM) algorithm appears to be the fastest convergence algorithm [31], and usually, can achieve better convergence results compared with the backpropagation through time (BPTT) algorithm especially for an RNN [32].
However, in order to utilize the LM algorithm, the cost function, C dp , in Equation ( 8) has to be expressed in a sum-of-squares format.When γ = 1, j = 1 and k = 1, • • • , N for a limited number of sequences, the cost function, C dp , can be rewritten as Equation ( 9) through a simple transformation: and the gradient ∂w can be expressed in a matrix product form: where the Jacobian matrix Equation ( 12) gives the weightupdate formula [31,33,34] for training a neural network (NN) controller : In a closed-loop control environment, the NN will be a Recurrent Neural Network as the outputs of a CHB system are fed back to the NN as the inputs at the next time step, as illustrated in Figure 8.The system state-space equations in the d-q domain serve as the feedback confections for the NN controller.Further, Equation (13) describes the system equation of an LCL filter-based GCC in the d-q domain [35], which was used for NN training, as shown in Figures 7-9.
where ω s is the angular frequency of the grid voltage, and all other symbols are consistent with those used in Equations ( 2) and ( 3).The corresponding relationships of all the variables between the d-q domain and the single-phase circuit domain are the following: To calculate the Jacobian matrix, every trajectory needs to be expanded forward through time, with the Forward Accumulation Through Time (FATT) algorithm illustrated in Figure 9, and the general BPPT algorithm for RNN training does not apply in this case [32].

Simulation Results
MATLAB SIMULINK was used for design verification and comparison in this paper.Figure 10 shows the Simulink model for the grid-tied CHB inverter.Both the proposed novel RNN control and conventional PI-based vector control were implemented in the shown Simulink model.Two modulation techniques, that is phase-shifted modulation and model-shifted modulation, were adopted in the simulation.Table 1 specifies the parameters of a single-phase CHB grid-tied inverter system.Most system parameters were taken from references [36,37]; the LCL filter values were selected to provide better attenuation results.

Five-Level CHB DC Voltage
Two cascaded-connected H-Bridge converters generate combined five-level converter voltages.Figures 11 and 12 show the generated five-level CHB converter voltages under conventional PI-based vector control with phase-shifted modulation and level-shifted modulations, respectively.Figures 13 and 14 shows the generated five-level CHB converter voltages under novel RNN control with phase-shifted modulation and level-shifted modulations, respectively.
Compared with one H-Bridge-based grid-tied inverter, the five-level converter systems can generate more close to sinusoidal voltage waveform and less Total Harmonic Distortion (THD).This advantage will definitely help better and high-quality integration of renewable sources to be obtained.Exactly the same voltage loop PI controllers were used in all simulations.The CHB combined DC voltages in four scenario simulations were all able to stabilize at 200 V, as expected.comparison between conventional PI-based vector control and novel RNN control indicated that the proposed RNN control has fewer oscillations and a fast response in d-q current waveforms, especially at the starting point, which can be recognized from the d-q current plots in Figures 19-22    From Figures 23 and 24, it can be seen that the THD of the inverter voltage, V g , is very small-around 0.01%.This indicates very good harmonic reduction ability of the LCL filter under conventional PI-based vector control with both phase-shifted modulation and level-shifted modulation.

Current Tracking in the d-q Domain
Figures 25 and 26 show the calculated Total Harmonic Distortion (THD) of the filtered inverter voltage, V g , under novel RNN control with phase-shifted modulation and level-shifted modulation, respectively.From Figures 25 and 26, it can be seen that the THD of the inverter voltage, V g , is even smaller and actually less than 0.01%.This indicates a much better harmonic reduction ability under novel RNN control with both phase-shifted modulation and level-shifted modulation.The comparisons between Figures 28 and 30, and 32 and 34 demonstrate that novel RNN control yields smaller THD than conventional PI-based vector control with both phase-shifted modulation and level-shifted modulation.

Conclusions
This paper focused on a control system design with different carrier signal generation for the PWM of a grid-tied multilevel converter.The proposed RNN control system can be easily extended for higher level systems, i.e., seven, nine, eleven or more.In sum, this paper demonstrates the complete simulation process with the RNN-based closed-loop control strategy for both phase-shifted and level-shifted PWM inverters.The effectiveness and correctness of DC bus voltage balancing capability have been successfully validated through MATLAB simulation results.To further evaluate the feasibility of the NN technique, testing the proposed NN controller in a complete hardware-in-loop experiment with a microcontroller board and a solar panel is planned for future research.

Figure 1 .
Figure 1.The schematic of a five-level grid-tied inverter.

Figure 2 .
Figure 2. The schematic of a five-level grid-tied inverter.

Figure 3
demonstrates the tuning processing of the current-loop PI controllers.

Figure 3 .
Figure 3.The tuning process of the PI controllers.

Figure 4 .
Figure 4.The PI-based vector control for the single-phase grid-tied inverter system.
Figure 5 demonstrate the phase shift modulation technique where Cr leg1 , Cr leg2 , Cr leg3 , and Cr leg4 represent the carrier signals for H-Bridge 1 and H-Bridge 2.

Figure 5 .
Figure 5.The phase shifted carrier signal with modulation.

Figure 6 .
Figure 6.The level shifted carrier signal with modulation.

Figure 8 .
Figure 8.The NN controller training in a closed loop control system.

Figure 9 .
Figure 9. Illustration of the proposed Forward Accumulation Through Time (FATT) algorithm for training an NN controller.

Figure 10 .
Figure 10.The developed Simulink model of the Recurrent Neural Network (RNN) control & PI-based vector control for a five-level grid-tied converter.

Figure 11 .
Figure 11.Five-level CHB voltage under conventional PI-based vector control with phase-shifted modulation.

Figure 12 .
Figure 12.Five-level CHB voltage under conventional PI-based vector control with level-shifted modulation.

Figure 13 .
Figure 13.Five-level CHB voltage under novel RNN control with phase-shifted modulation.

Figure 14 .
Figure 14.Five-level CHB voltage under novel RNN control with level-shifted modulation.

Figures 15 and 16
Figures 15 and 16 show the CHB combined DC voltage, the voltage, V DC1 , of capacitor DC1, and the voltage, V DC2 , of capacitor DC2 under conventional PI-based vector control with phase-shifted modulation and level-shifted modulations, respectively.

Figure 15 .
Figure 15.CHB DC voltage tracking under conventional PI-based vector control with phase-shifted modulation.

Figure 16 .
Figure 16.CHB DC voltage tracking under conventional PI-based vector control with level-shifted modulation.

Figures 17 and 18
Figures 17 and 18 show the CHB-combined DC voltage, the voltage, V DC1 , of capacitor DC1, and the voltage, V DC2 of capacitor DC2 under novel RNN control with phase-shifted modulation and level-shifted modulations, respectively.

Figure 17 .
Figure 17.CHB DC voltage tracking under novel RNN control with phase-shifted modulation.

Figure 18 .
Figure 18.CHB DC voltage tracking under novel RNN control with level-shifted modulations.

Figures 19 and 20
Figures 19 and 20  show the CHB-combined DC voltage, d-q current tracking and the grid current, i g , under conventional PI-based vector control with phase-shifted modulation and level-shifted modulations, respectively.

Figure 19 .
Figure 19.CHB combined DC voltage tracking, d-q current tracking, and grid current i g under conventional PI-based vector control with phase-shifted modulations.

Figure 20 .
Figure 20.CHB-combined DC voltage tracking, d-q current tracking, and grid current i g under conventional PI-based vector control with level-shifted modulations.

Figures 21
Figures 21 and 22  show the CHB-combined DC voltage, d-q current tracking, and grid current, i g , under novel RNN control with phase-shifted modulation and level-shifted modulations, respectively.

Figure 21 .
Figure 21.CHB-combined DC voltage tracking, d-q current tracking, and grid current, i g under novel RNN control with phase-shifted modulations.

Figure 22 .
Figure 22.CHB-combined DC voltage tracking, d-q current tracking, and grid current, i g , under novel RNN control with level-shifted modulations. .

5. 4 .
Figures 23 and 24 show the calculated Total Harmonic Distortion (THD) of the filtered inverter voltage, V g , under conventional PI-based vector control with phase-shifted modulation and level-shifted modulation, respectively.

Figure 23 .Figure 24 .
Figure 23.Total Harmonic Distortion (THD) of the inverter voltage, V g , under conventional PI-based vector control with phase-shifted modulation.

Figure 25 .Figure 26 .
Figure 25.Total Harmonic Distortion (THD) of the inverter voltage, V g , under novel RNN control with phase-shifted modulation.

5. 5 .
Figures 27 and 28 show the grid current, i g , and the calculated Total Harmonic Distortion (THD) of the grid current, i g , under conventional PI-based vector control with phase-shifted modulation.

Figure 27 .
Figure 27.Grid current, i g , under conventional PI-based vector control with phase-shifted modulation.

Figure 28 .
Figure 28.Total Harmonic Distortion (THD) of the grid current, i g , under conventional PI-based vector control with phase-shifted modulation.

Figures 29 and 30
Figures 29 and 30  show the grid current i g and the calculated Total Harmonic Distortion (THD) of the grid current, i g , under conventional PI-based vector control with level-shifted modulation.

Figure 29 .
Figure 29.Grid current, i g , under conventional PI-based vector control with level-shifted modulation.

Figure 30 .
Figure 30.Total Harmonic Distortion (THD) of the grid current, i g , under conventional PI-based vector control with level-shifted modulation.

Figures 31 and 32
Figures 31 and 32  show the grid current, i g , and the calculated Total Harmonic Distortion (THD) of the grid current, i g , under novel RNN control with phase-shifted modulation.

Figure 31 .
Figure 31.Grid current, i g , under novel RNN control with phase-shifted modulation.

Figure 32 .
Figure 32.Total Harmonic Distortion (THD) under novel RNN control with phase-shifted modulation.Figures33 and 34show the grid current i g and the calculated Total Harmonic Distortion (THD) of the grid current, i g , under novel RNN control with level-shifted modulation.

Figure 33 .
Figure 33.Grid current, i g , under novel RNN control with level-shifted modulation.