Adaptive Dynamic Programming and Energy Management for Multiple Converters Under Primary Frequency Regulation

Abstract

There are abundant energy resources in remote areas of China, such as photovoltaics and small hydropower. With uncertain factors such as sunshine and climate, hydroelectric and photovoltaic power generation face prominent problems such as large output power fluctuations, unstable energy transmission, and difficulty in multi-converters’ synchronous control for primary frequency regulation. This article proposes an adaptive dynamic programming (ADP) control method and energy management strategies for multi-converters under primary frequency regulation, in order to address the problems of large-scale access to new energy. Firstly, a parameter online optimization design is proposed based on ADP controller to improve the dynamic performance of the system and the power quality of the output currents of multiple converters. Secondly, in order to achieve energy optimization management of multiple converters, a multimodal collaborative optimization control strategy is proposed to achieve energy optimization control and comprehensive management of the entire system. Finally, the effectiveness of the proposed ADP and energy management strategies are verified by simulation.

1. Introduction

To ensure the safety, reliability, environmental friendliness, economic viability, and sustainable development capacity of power supply under a future high-proportion new energy generation scenario, and to advance the overall goals of energy transformation, most countries have emphasized the need to build a new type of power system with “new energy” as its mainstay. This is also essential for adapting to new situations, new formats, and new models, deepening the revolution in energy production and consumption, and promoting the green and low-carbon transformation of the energy industry. According to the World Energy Statistical Yearbook 2025, the power generation from wind and solar energy increased by 16%, with their share in the global total power generation rising from 13% to 15%. Renewable energy is the fastest-growing energy type globally, with a growth rate as high as 9%. Over the past five years, the average growth rate of renewable energy has been about five times the annual average growth rate of global total energy demand [1,2,3,4,5].

To achieve optimal energy distribution and stable operation of a large number of renewable energy systems under primary frequency regulation, researchers have conducted studies on the parallel operation control of multiple inverters [6,7,8,9,10]. In terms of controlling parallel-operated multiple inverters in AC microgrids, collaborative control methods can be broadly classified into peer-to-peer control, master–slave control, and hierarchical control [11,12,13,14,15]. Distributed adaptive droop control strategies incorporating a current regulator to fine-tune the droop coefficient under varying load conditions were proposed in [16,17,18,19]. In [20,21,22,23], distributed collaborative control strategies based on a leader–follower structure were introduced, where the master inverter managed bus voltage and the slave inverter regulated energy storage battery balance. For the parallel operation of multiple inverters, control strategies combining decentralized logic and virtual negative resistance are presented, which reduced three-phase voltage asymmetry caused by unbalanced loads or other issues [24,25,26,27,28]. References [29,30,31] proposed coordinated control strategies for multiple inverters based on multi-agent/multi-level control. This approach divided microgrid control into multiple levels, each handling control tasks at different time scales, thereby facilitating stable system operation and optimized management [32,33]. However, most existing studies have focused solely on the coordination of the same kind of converters, leaving unaddressed both scenarios involving multiple heterogeneous sources and the issue of flexible adaptation among converters in multi-mode scenarios.

In terms of the joint collaborative control and energy management of different kinds of multiple converters under primary frequency regulation, a large number of advanced control methods and energy management strategies have been applied [34,35,36,37,38]. In [39], an upper-level energy management strategy was present, which established a game-theoretic framework through current and voltage sharing strategies by a distributed iterative learning algorithm based on Bellman’s optimality principle, and implemented it using approximate dynamic programming (ADP) techniques. In [40], a novel ADP control method was proposed based on discrete-time multi-agent systems to address the optimal tracking control problem. In [41], a multi-port energy router was investigated with two AC buses, where loads can be switched or shared between them, and proposed a fuzzy-logic-based hierarchical control strategy. In [42], multi-mode operation control strategies were proposed for a hybrid four-port energy router, taking into account the states and energy flow directions of each port. In [43], a systematic control strategy was proposed for a hybrid five-port energy router, classifying system operation into three modes—islanding, grid-connected, and standby—and presenting a fuzzy-logic-based charging/discharging strategy that incorporates energy storage price optimization. In summary, most existing advanced control methods were primarily designed for single converter [44,45,46,47,48] and cannot be directly applied to multi-converter interconnected systems. Furthermore, existing energy management approaches mainly emphasized upper-level energy management while rarely addressing the lower-level adaptation of multiple converters, thus failing to achieve optimal adaptation for a multi-converter system comprising renewable energy sources, photovoltaics, energy storage, and grid integration [49,50,51].

To tackle the issues of renewable energy integration into the power grid, including significant fluctuations in output power, unstable energy transmission, and difficulties in primary frequency regulation, this paper aims to enable dynamic access and flexible control of various renewable energy sources with multiple converters under primary frequency regulation. The main contributions of this paper are as follows:

(1)

Based on optimal control and ADP, a parameter online optimization design method is proposed to improve the dynamic performance of the system and the power quality of the output currents of multiple converters.

(2)

In order to achieve energy optimization management of multiple converters, a multimodal collaborative optimization control strategy is proposed to achieve energy optimization control and comprehensive management of the entire system.

The structure of this article is as follows. In Section 2, basic knowledge for adaptive dynamic programming control is studied and an ADP online optimization design method for multiple converters is proposed to enable online learning and optimization of the controller. In Section 3, the system is divided into multiple operating modes based on the status of each port, and corresponding control strategies are adopted for each converter under different operating modes to achieve energy optimization control and comprehensive management of the entire system. In Section 4, ADP controllers for DCAC converters, ADP controllers for DCDC converters, and the energy management strategies of multiple converters are discussed under primary frequency regulation. In Section 5, The effectiveness of the proposed ADP and energy management strategies are verified by simulation.

2. Adaptive Dynamic Programming Controller for Multiple Converters

The overall structure multi converters for adaptive dynamic programming and energy management under primary frequency regulation are shown in Figure 1. Based on the theoretical foundation of “optimal quadratic regulator”, an ADP online optimization design method is proposed to enable online learning and optimization of the controller, achieving online design of the controller during synchronous operation of multiple converters.

2.1. Basic Knowledge for Adaptive Dynamic Programming Control

According to the theory of optimal quadratic form, the system state equation can be described as:

$$\dot{x}=A_{k}x+B{u}_k$$

(1)

where $x\in R^n$ is the system state fully available for feedback control design; $u\in R^m$ is the control input; $A\in R^{n·n}$ and $B\in R^{n·m}$ are uncertain constant matrices.

To achieve online learning and an iterative solution, a small disturbance signal e needs to be injected into the control input. The control strategy can be expressed as follows:

$$u_k=-K_{k}x+e$$

(2)

where e is also referred to as an incentive. According to Equation (1), we derive the performance metric:

$${\displaystyle \frac{d}{dt}}(x^T{P}_{k}x)=x^T(A_k^T{P}_k+P_k{A}_k)x+{2u_k}^T{B}^T{P}_{k}x=-x^T{Q}_{k}x+{2u_k}^{T}R{K}_{k+1}x$$

(3)

where $Q_k=Q+K_k^{T}R{K}_k$. In (3), $-x^T{Q}_{k}x$ can be obtained through real-time data sampling of the system’s state trajectory, $K_{k+1}$ is an unknown matrix in $R{K}_{k+1}$, and it can be solved with $P_k$. So, by (3), without requiring the state matrices A_k and B of the system, $P_k$ and $K_{k+1}$ can be solved online. Specifically, for any given interval $[t,t+\delta t]$, integrating both sides of Formula (3) yields:

$$x^T(t+\delta t)P_{k}x(t+\delta t)-x^T(t)P_{k}x(t)-2{\displaystyle {\int }_t^{t+\delta t}{(K_{k}x+u_k)}^{T}R{K}_{k+1}xd\tau }=-{\displaystyle {\int }_t^{t+\delta t}{x}^T{Q}_{k}xd\tau }$$

(4)

Thus, after a period of sampling, a set of continuous state data is obtained. By substituting the state data into Formula (4), a system of equations containing $P_k$ and $K_{k+1}$ are derived. By solving this system of equations, $P_k$ and $K_{k+1}$ can be calculated. The more detailed calculations of ADP can be referred to in [52].

2.2. ADP Controller Implementation for DCAC Converters

In Figure 1, state equations of the T-type NPC converter can be written as:

$$\left[\begin{array}{l}{\dot{i}}_{\mathrm{d}}\\ {\dot{i}}_{\mathrm{q}}\\ {\dot{u}}_{\mathrm{dc}}\\ \Delta {\dot{u}}_{\mathrm{dc}}\end{array}\right]=\left[\begin{array}{cccc}-{\displaystyle \frac{R_{\mathrm{s}}}{L}}& \omega & 0& 0\\ -\omega & -{\displaystyle \frac{R_{\mathrm{s}}}{L}}& 0& 0\\ {\displaystyle \frac{-3(S_{\mathrm{d}1}-S_{\mathrm{d}2})}{2C}}& {\displaystyle \frac{-3(S_{\mathrm{q}1}-S_{\mathrm{q}2})}{2C}}& 0& 0\\ {\displaystyle \frac{-3(S_{\mathrm{d}1}+S_{\mathrm{d}2})}{2C}}& {\displaystyle \frac{-3(S_{\mathrm{q}1}+S_{\mathrm{q}2})}{2C}}& 0& 0\end{array}\right]\left[\begin{array}{l}{i}_{\mathrm{d}}\\ i_{\mathrm{q}}\\ u_{\mathrm{dc}}\\ \Delta u_{\mathrm{dc}}\end{array}\right]+\left[\begin{array}{cc}{\displaystyle \frac{1}{L}}& 0\\ 0& {\displaystyle \frac{1}{L}}\\ 0& 0\\ 0& 0\end{array}\right]\left[\begin{array}{c}{e}_{\mathrm{d}}-u_{\mathrm{d}}\\ e_{\mathrm{q}}-u_{\mathrm{q}}\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ \frac{2}{C}\\ 0\end{array}\right]i_{\mathrm{DC}}$$

(5)

where i_d, i_q, u_dc, Δu_dc are the state variables of DCAC converters, that is, the d-axis current, the q-axis current, the DC side voltage, and the DC side midpoint voltage difference. R_s and L are the resistance and inductor of the converter. $\omega$ is the angular velocity. S_d1, S_d2, S_q1, S_q2 are the switching variables of DCAC converters. e_d, e_q, u_d, u_q are the d-axis grid voltage, the q-axis grid voltage, the d-axis converter voltage, and the q-axis converter voltage. i_DC is the DC side output voltage.

From the above equation, it can be seen that i_d and i_q are coupled. If a feedforward decoupling control method is adopted, the control block diagram shown in Figure 2a can be obtained. Here, i_d^ref and i_q^ref are the command values of the converter, PI is the proportional–integral controller of the current loop, and e_q is equivalent to the feedforward compensation for the grid voltage.

When system parameters deviate or are uncertain, it becomes challenging to design controllers. In this paper, to optimize controller parameters online, we propose an adaptive dynamic programming-based method, as illustrated in Figure 2b. Taking the d-axis as an example, current and voltage state values can be adopted online. By solving Formula (4), the required k_d value can be obtained. Therefore, the control law for the d-axis is expressed as:

$$\begin{array}{cc}\hfill u_{\mathrm{d}}& =-[k_{\mathrm{d}1}(i_{\mathrm{d}}^{\mathrm{ref}}-i_d)+k_{\mathrm{d}2}(u_{\mathrm{dc}}^{\mathrm{ref}}-u_{dc}-k_{\mathrm{d}4}{i}_{\mathrm{dc}})+k_{\mathrm{d}3}(x_3^{\mathrm{ref}}-x_3)]\hfill \\ & =-{\mathit{k}}_{\mathbf{d}}\mathit{x}\hfill \end{array}$$

(6)

where ${\dot{x}}_3=u_{\mathrm{dc}}^{\mathrm{ref}}-u_{\mathrm{dc}}$, x₃ represents the integral of voltage difference, which can eliminate the static voltage during steady state. The DC side output i_dc can be feedforward control to improve dynamic performance, or it can be used for droop control to achieve power distribution through parameter k_d4. During system iterative calculation, k_d4 does not undergo dynamic optimization and is set by the upper computer. Meanwhile, the voltage balancing control on the DC capacitor side can be achieved by injecting zero-sequence current. The implementation method of the traditional two-level converter is consistent with the above, and it lacks the voltage balancing control link, making it easier to implement.

2.3. ADP Controller Implementation for DCDC Converters

In Figure 1, state equations of DCDC converters can be written as:

$$\left\{\begin{array}{l}{C}_{11}{\displaystyle \frac{d{u}_{C11}}{dt}}=i_{\mathrm{in}}-i_L\cdot d_{11}\\ C_{12}{\displaystyle \frac{d{u}_{C12}}{dt}}=i_{\mathrm{in}}-i_L\cdot d_{12}\\ C_{21}{\displaystyle \frac{d{u}_{C21}}{dt}}=i_L(1-d_{21})-i_{\mathrm{o}}\\ C_{22}{\displaystyle \frac{d{u}_{C22}}{dt}}=i_L(1-d_{22})-i_{\mathrm{o}}\\ L{\displaystyle \frac{d{i}_L}{dt}}=u_{C11}{d}_{11}+u_{C12}{d}_{12}-u_{C21}(1-d_{21})-u_{C22}(1-d_{22})\end{array}\right.$$

(7)

where C₁₁, C₁₂, C₂₁, C₂₂, L are capacitors and inductors of the DCDC converter. u_C11, u_C12, u_C21, u_C22 and i_L are the state variables of DCDC converters, that is, the capacitor voltages on input and output sides, and the inductor current, respectively. i_in and i_o are the input and output currents. d₁₁, d₁₂, d₂₁, d₂₂ are the duty cycles of different IGBT switches.

The mathematical model of the main circuit of DCDC converters can be derived from Formula (7) as follows:

$$\left\{\begin{array}{l}C{\displaystyle \frac{d{u}_{\mathrm{dc}}}{dt}}=i_L(1-d_2)-i_{\mathrm{o}}\\ L{\displaystyle \frac{d{i}_L}{dt}}=V_{\mathrm{in}}{d}_1-u_{\mathrm{dc}}(1-d_2)\end{array}\right.$$

(8)

Under normal conditions, the DCDC converter is linear in Buck mode and can directly adopt the method described in Section 2.1. When the converter operates in Boost mode, its state equation is expressed as follows:

$$\left\{\begin{array}{l}{\displaystyle \frac{d{x}_1}{dt}}={\displaystyle \frac{1}{C}}\left[x_2-x_{2}u-i_{\mathrm{o}}\right]\\ {\displaystyle \frac{d{x}_2}{dt}}={\displaystyle \frac{1}{L}}\left[-x_1+x_{1}u+V_{\mathrm{in}}\right]\end{array}\right.$$

(9)

where [x₁, x₂] = [u_dc, i_L], u = d₂. According to Formula (9), it can be seen that the converter operating in Boost mode is nonlinear. To implement the adaptive dynamic programming algorithm, sampling data can be taken within the steady-state region D and iterative optimization can be performed. However, for states outside region D, shown the red arrow in Figure 3, due to the presence of strong nonlinearity, the calculated P and K may not be applicable to states within the region. When the initial value is far from the steady-state value (outside Ωi), Formula (7) should not be solved at this time because the system is nonlinear, and the solution obtained would be inaccurate and may even cause system instability. To ensure system stability, the state variables need to enter Ωi before iterative optimization calculations are performed, shown the black arrow in Figure 3.

In summary, the control block diagram of the Buck–Boost converter is shown below:

In Figure 4, the values of controller parameters k₁, k₂, and k₃ can be optimized online by measuring the output voltage u_dc and inductor current i_L. Through state feedback, the control output is obtained as follows:

$$\begin{array}{cc}\hfill u_{\mathrm{d}}& =-[k_1(i_{\mathrm{L}}^{\mathrm{ref}}-i_{\mathrm{L}})+k_2(u_{\mathrm{dc}}^{\mathrm{ref}}-u_{\mathrm{dc}}-k_4{i}_{\mathrm{O}})+k_3(x_3^{\mathrm{ref}}-x_3)]\hfill \\ & =-\mathit{k}\mathit{x}\end{array}$$

(10)

where ${\dot{x}}_3=u_{\mathrm{dc}}^{\mathrm{ref}}-u_{\mathrm{dc}}$, x₃ represents the integral of voltage difference, which can eliminate the static voltage during steady state. During the system operation, k₄ does not undergo dynamic adjustment; instead, it is set by the host computer to achieve power distribution between ports.

3. Energy Management Strategies of Multiple Converters for Primary Frequency Regulation

This part analyzes the energy management strategy of primary frequency regulation, achieving energy optimization control. The energy management strategy adopted in this paper is shown in Figure 5, which is mainly divided into the information collection layer, decision-making control layer, and bottom control layer. The information collection layer uploads data from the bottom devices to the decision-making control layer, such as the average state of charge of energy storage units, the power injected into the DC bus by each unit module, including energy storage, photovoltaic, grid, and new energy; the decision-making control layer receives external dispatching instructions from the distribution network and real-time data about the bottom devices issued by the information collection layer, formulates dispatching control and energy management strategies, divides the modes of multi converters, and determines the control methods of the bottom controllers; the bottom control layer executes the control modes and related instructions issued by the decision-making control layer, and performs overall controller parameter optimization design for each port.

3.1. The Mode Classification in Different Operations

The modal classification of the system is comprehensively determined by considering the SOC state of the energy storage port, the energy access amount P_pv of the photovoltaic port, the energy access amount of other renewable energy port, and the energy output amount of the grid-connected port. Taking into account the characteristics of external sources and grid-connection requirements, the system can be divided into eight modes, as shown in

Table 1. Classification of modes in different operations.

Modes	Different Operations	SOC
Mode 1	Renewable energy is abundant, PV is excess, $P_{\mathrm{pv}}^{\mathrm{MPPT}}>P_{\mathrm{g}}^{\mathrm{ref}}$	SOC ≥ 0.9
Mode 2	Renewable energy is abundant, PV is excess, $P_{\mathrm{pv}}^{\mathrm{MPPT}}>P_{\mathrm{g}}^{\mathrm{ref}}+P_{\mathrm{bat}}$	SOC < 0.9
Mode 3	Renewable energy is abundant, PV is less, $P_{\mathrm{pv}}^{\mathrm{MPPT}}<P_{\mathrm{g}}^{\mathrm{ref}}$	SOC ≥ 0.9
Mode 4	Renewable energy is abundant, PV is less, $P_{\mathrm{pv}}^{\mathrm{MPPT}}<P_{\mathrm{g}}^{\mathrm{ref}}+P_{\mathrm{bat}}$	SOC < 0.9
Mode 5	Renewable energy is storage, battery cannot be charging $P_{\mathrm{pv}}^{\mathrm{MPPT}}+P_{\mathrm{w}}<P_{\mathrm{g}}^{\mathrm{ref}}+P_{\mathrm{bat}}$
Mode 6	Renewable energy is storage, battery is discharging $P_{\mathrm{pv}}^{\mathrm{MPPT}}+P_{\mathrm{w}}<P_{\mathrm{g}}^{\mathrm{ref}}$	SOC > 0.2
Mode 7	Energy is storage $P_{\mathrm{pv}}^{\mathrm{MPPT}}+P_{\mathrm{w}}<P_{\mathrm{g}}^{\mathrm{ref}}$	SOC ≤ 0.2
Mode 8	Renewable energy is storage and shut down $P_{\mathrm{pv}}^{\mathrm{MPPT}}+P_w<P_{\mathrm{g}}^{\mathrm{ref}}-P_{\mathrm{bat}}$	SOC > 0.2

.

Classification of modes in different operations can be described as:

Mode 1: When SOC is ≥0.9 and $P_{\mathrm{pv}}^{\mathrm{MPPT}}>P_{\mathrm{g}}^{\mathrm{ref}}$, the energy storage port operates in idle mode, while the photovoltaic port switches to CVC mode. The output power is supplied by the hydroelectric port unit and the photovoltaic unit. When the energy injected by the photovoltaic unit drops below a certain threshold, that is, $P_{\mathrm{pv}}^{\mathrm{MPPT}}>P_{\mathrm{g}}^{\mathrm{ref}}$, the interaction between the photovoltaic unit and the grid unit may cause a rapid voltage drop on the DC bus. If the detected voltage drop reaches $P_{\mathrm{pv}}^{\mathrm{MPPT}}>P_{\mathrm{g}}^{\mathrm{ref}}$ or $P_{\mathrm{pv}}^{\mathrm{MPPT}}\le P_{\mathrm{g}}^{\mathrm{ref}}$, the multi-converter is switched to Mode 3.

Mode 2: When SOC is less than 0.9 and $P_{\mathrm{pv}}^{\mathrm{MPPT}}>P_{\mathrm{g}}^{\mathrm{ref}}+P_{\mathrm{bat}}$, the energy storage adopts charging mode and the photovoltaic unit adopts CVC mode. When the rising voltage of the DC bus is detected to be 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$ or $P_{\mathrm{pv}}^{\mathrm{MPPT}}\le P_{\mathrm{g}}^{\mathrm{ref}}+P_{\mathrm{bat}}$, switch to mode 4.

Mode 3: When SOC is ≥0.9 and $P_{\mathrm{pv}}^{\mathrm{MPPT}}\le P_{\mathrm{g}}^{\mathrm{ref}}$, the energy storage unit is in an idle state, and the photovoltaic unit is MPPT-controlled. The grid-connected power is provided by the hydropower port unit and the photovoltaic unit. When the detected bus drop voltage reaches 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$, switch to mode 6; when the rising voltage of the DC bus reaches 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$, switch to mode 1.

Mode 4: When SOC is less than 0.9 and $P_{\mathrm{pv}}^{\mathrm{MPPT}}\le P_{\mathrm{g}}^{\mathrm{ref}}+P_{\mathrm{bat}}$, the energy storage unit is in a charging state, the photovoltaic unit is MPPT-controlled, and the grid-connected power is provided by the hydropower port unit and the photovoltaic unit. When the drop voltage of the bus voltage reaches 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$, switch to mode 5; when the rising voltage of the DC bus reaches 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$, switch to mode 2.

Mode 5: When SOC is less than 0.9 and $P_{\mathrm{w}}<P_{\mathrm{g}}^{\mathrm{ref}}-P_{\mathrm{bat}}-P_{\mathrm{pv}}$, the new energy is insufficient to charge the energy storage. The energy storage unit changes from charging mode to idle mode, and the photovoltaic unit switches to MPPT mode; if a drop voltage of 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$ is detected on the DC bus, switch to mode 6 or mode 7 based on SOC; if the rising voltage of the DC bus is detected to be 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$, switch to mode 3 or mode 4 based on SOC.

Mode 6: When SOC is greater than 0.2 and $P_{\mathrm{pv}}^{\mathrm{MPPT}}+P_{\mathrm{w}}<P_{\mathrm{g}}^{\mathrm{ref}}$, the energy storage unit is in discharge mode. If a voltage drop of 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$ is detected on the DC bus, a fault signal will be sent and the system will shut down; if the rising voltage of the DC bus is detected to be 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$, switch the system to mode 5 based on SOC.

Mode 7: When SOC is ≤0.2 and $P_{\mathrm{pv}}^{\mathrm{MPPT}}+P_{\mathrm{w}}<P_{\mathrm{g}}^{\mathrm{ref}}$, the energy storage unit is in idle mode. If a voltage drop of 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$ is detected on the DC bus, a fault signal will be sent and the system will shut down; if the rising voltage of the DC bus is detected to be 0.05 $U_{\mathrm{dc}}^{\mathrm{ref}}$, switch the system to mode 5 based on SOC.

Mode 8: When SOC is greater than 0.2 and the energy supply at the renewable energy ports is severely insufficient. The energy storage unit is in discharge mode, and the renewable energy port unit is shut down. Power scheduling is achieved by charging and discharging the energy storage unit.

In all, the switching process of different modes can be described in Figure 6.

3.2. Energy Management Strategies for Primary Frequency Regulation

There are two types of energy management strategies for primary frequency regulation at the decision-making and control layer:

Energy management strategy 1: The DC voltage regulation control is performed by the new energy port; the energy storage adopts constant current mode or droop voltage regulation mode control, and changes the charging and discharging mode of the energy storage battery and the corresponding charging and discharging current commands according to the average state of charge; the photovoltaic port decides to adopt constant voltage control mode (CVC) or maximum power point tracking (MPPT) mode according to the magnitude of the generated power; the grid-connected port adopts constant power control mode, which can directly achieve precise power control, as shown in Figure 7.

Energy management strategy 2: DC voltage stabilization control through the grid-connected port; energy storage is controlled using either constant current mode or droop voltage regulation mode; the photovoltaic port adopts CVC control or MPPT control mode depending on the magnitude of power generation; the new energy port employs constant power control mode, updating the active power target value based on primary frequency regulation data. As illustrated in Figure 8, the input power of the new energy port is adjusted through a dichotomy control function to achieve precise power control of the grid-connected port.

Due to the dynamic changes in power states at various ports of the system with external conditions and its own physical characteristics in this mode, it is necessary to study the power flow patterns and operational modes under this energy management and scheduling control strategy. Subsequently, research should be conducted to develop control strategies and port power commands for each port under this operational mode, achieving multi-mode collaborative optimization control.

4. Results and Discussion

In order to verify the effectiveness of the proposed ADP control method and energy management strategies, this part conducts simulation verifications on three aspects: ADP controller for DCAC converters, ADP controller for DCDC converters, and energy management strategy of multiple converters for primary frequency regulation.

4.1. ADP Controller for DCAC Converter

The relevant parameters of DCAC converters are set as follows: grid-side inductance L₁ is 0.015 mH, grid-side filtering capacitor C₁ is 150 µF, converter-side inductance L₂ is 0.05 mH, and DC side capacitance C₂ is 2 mF. The range of these parameters are shown in

Table 2. Parameters of DCAC converters.

Parameters	Values
Grid-side inductance L1	0.015–0.05 mH
Converter-side inductance L2	0.05–0.5 mH
Grid-side filtering capacitor C1	150–500 µF
DC-side capacitor C2	2 mF
Switching frequency f	4.8 kHz
The rated power	500 kW

. To verify the effectiveness of the ADP controller, it is compared with the PI controller. The control parameters of the PI controller are: voltage outer loop proportional coefficient K_pv is 1.2, integral coefficient K_iv is 400, current inner loop selected proportional coefficient K_pi is 5, and integral coefficient K_ii is 3000. The operating results are shown in Figure 9.

According to Figure 9, the THD of three-phase AC currents under the PI controller is 1.1%, while the THD of three-phase AC currents under the ADP controller is 0.9%. From this, it can be seen that the three-phase AC current power quality based on the ADP controller is better than that based on the PI controller, and its transient process is analyzed in Section 4.3.

4.2. ADP Controller for DCDC Converter

The parameter range of a DCDC converter can refer to

Table 3. Parameters of DCDC converters.

Parameters	Values
Inductance L	0.1–0.8 mH
Capacitor C	1.5 mF
Switching frequency f	3.6 kHz
The rated power	250 kW

. During simulation, the inductance parameter L is 0.8 mH and the capacitance parameter C is 1.5 mF. The simulation results are shown in Figure 10.

At the beginning of operation, due to an external disturbance, the inductor current i_L and output voltage u_o both oscillate significantly, as shown in Figure 10a,b. Before the online optimization of the ADP controller, due to the deviation of the system parameters in Table 3, the initial values of the controller parameters did not match the system, resulting in a long adjustment time. At 0.02 s, the system injects excitation, as shown in Figure 10e, and begins sampling and learning. At 0.05 s, sampling stops, and the control system uses these data to solve online using Formula (17) to obtain optimized controller parameters K, as shown in Figure 10d. After the controller parameters are updated, the system quickly stabilizes, as shown in the voltage and current waveforms, as shown in the black waveforms in Figure 10a,b, and the control signal duty cycle also quickly returns to a steady-state value, as shown in Figure 10c. According to the simulation, it can be concluded that online iterative optimization design based on ADP controller is feasible and can achieve steady-state control even when system parameters are offset or unknown.

4.3. Energy Management Strategy for Multiple Converters

In order to verify the applicability of the proposed energy management strategy for primary frequency regulation, simulation verifications are conducted on multi converters by the proposed energy management strategies. The measurement points are the outputs of each converter with the DC bus, shown in Figure 1. The bottom layer controller used an ADP controller with online learning iteration, and the upper layer used droop control to achieve energy management. The dynamic optimization of the bottom layer controller parameters was closely combined with the upper layer energy management.

The simulation results by energy management strategy 1 are shown in Figure 11. At the beginning of operation, multiple converters run as a whole in Mode 1 of Table 1, with the grid-connected port operating in constant power mode and an output power of 100 kW. The renewable energy converter and photovoltaic converter are both in constant voltage mode, and the primary frequency regulation droop coefficient is 0.05. The energy storage is in a leisure state. Figure 11a shows the DC bus voltage, which is stable at 700 V. The output current on the DC side of the photovoltaic and hydropower ports is 71.5 A, the DC side current of the grid-connected port is 143 A, and the energy storage is 0 A, as shown in Figure 11b. The AC current and THD of the grid-connected converter and renewable energy converter are shown in Figure 11c–f. It can be seen that the THD is poor during transient state. When the controller parameters are iteratively optimized and the system enters steady state, the power quality is good, with a THD of 1%. At 0.2 s, the system transitions from mode 1 to mode 2, and the energy storage port operates in constant voltage mode. The droop coefficient ratio of the primary frequency regulation for the renewable energy converter, photovoltaic converter, and energy storage converter is 0.05:0.025:0.025, resulting in an output power ratio of 2:1:1 for each port, as shown in Figure 11b. At 0.4 s, the output command on the grid side increased from 100 kW to 150 kW, and the output currents of each port changed. The three-phase AC currents on the renewable energy side and grid side had poor THD during transient state. In order to ensure better power quality and faster dynamic response, the parameters of the two port controllers were optimized. Subsequently, the system entered a new steady-state value, and the THD of the three-phase AC current on both the renewable energy side and the grid side decreased to 1%, thereby achieving optimized control and comprehensive energy management of the multi converters under primary frequency regulation.

The simulation results by energy management strategy 2 are shown in Figure 12. At the beginning of operation, the energy router operates entirely in Mode 1 as shown in Table 1. The grid-connected port operates in constant voltage mode, while the renewable energy converter and photovoltaic converter operate in constant power mode and constant voltage control (CVC) mode, respectively. The energy storage is in an idle state. Figure 12a shows the DC bus voltage, which remains stable at 700 V. Figure 12b presents the DC-side current values of each port. It can be observed that the system successfully achieves power distribution among the ports, and the AC current power quality of both the grid-connected converter and t the renewable energy converter is good, with a total harmonic distortion (THD) of 1%, as shown in Figure 12c–f. At 0.2 s, the system transitions from Mode 1 to Mode 2. The energy storage converter operates in constant current discharge mode. The DC bus voltage experiences a small fluctuation but quickly reaches a new steady state. At 0.4 s, the output power command on the grid side increases from 100 kW to 150 kW. The output currents of all ports change accordingly. During the transient period, the three-phase AC currents on both the hydropower side and the grid side exhibit poor THD. To improve the power quality of the system, the controller parameters of these two converters are optimized. After the parameter update and optimization, the system quickly enters a new steady state. The three-phase AC current power quality on both the hydropower side and the grid side becomes good, with a THD of 1%. Thus, system parameter optimization and comprehensive energy management under energy management strategy 2 are successfully achieved.

Overall, by comparing the above two strategies of multiple converters for primary frequency regulation, it can be observed that regardless of which energy management control strategy is adopted, the ADP controller demonstrates adequate adaptability and enables the synchronous optimal design of multiple controller parameters. Consequently, it achieves the seamless integration and unification of underlying controller parameter adjustment with system-wide energy management.

5. Conclusions

In this paper, an ADP control and energy management method for multiple converters was studied under primary frequency regulation, which closely integrated the dynamic optimization of controller parameters with energy management to achieve flexible adaptation of new energy. Through theoretical analysis and simulation verification, the following conclusions are drawn: (1) In order to achieve energy optimization management of the system, a multimodal collaborative optimization control strategy for multiple converters was proposed. Based on the system state, the system was divided into multiple operating modes, and corresponding control strategies and instructions were adopted for each converter under different operating modes to achieve energy optimization control and comprehensive management of the entire system. (2) When the system parameters of an ACDC converter or a DCDC converter are uncertain or mismatch, using the ADP method for controller parameter optimization design can improve the control performance of the system, achieve fast tracking and control, and optimize power quality. (3) When the mode changes or the system operating state changes, the ADP controller has a certain adaptability and can achieve synchronous optimization design of multiple controller parameters, realizing the integration and unity of parameter adjustment of the underlying controller and the system operating mode.