Design of Robust Adaptive Nonlinear Backstepping Controller Enhanced by Deep Deterministic Policy Gradient Algorithm for Efficient Power Converter Regulation

Abstract

Power converters play an important role in incorporating renewable energy sources into power systems. Among different converter designs, Buck and Boost converters are popular, as they use fewer components and deliver cost savings and high efficiency. However, Boost converters are known as non–minimum phase systems, imposing harder constraints for designing a robust converter. Developing an efficient controller for these topologies can be difficult since they exhibit nonlinearity and distortion in high frequency modes. The Lyapunov-based Adaptive Backstepping Control (ABSC) technology is used to regulate suitable outputs for these structures. This approach is an updated version of the technique that uses the stability Lyapunov function to produce increased stability and resistance to fluctuations in real-world circumstances. However, in real-time situations, disturbances with larger ranges such as supply voltage changes, parameter variations, and noise may have a negative impact on the operation of this strategy. To increase the controller’s flexibility under more difficult working settings, the most appropriate first gains must be established. To solve these concerns, the ABSC’s performance is optimized using the Reinforcement Learning (RL) adaptive technique. RL has several advantages, including lower susceptibility to error, more trustworthy findings obtained from data gathering from the environment, perfect model behavior within a certain context, and better frequency matching in real-time applications. Random exploration, on the other hand, can have disastrous effects and produce unexpected results in real-world situations. As a result, we choose the Deep Deterministic Policy Gradient (DDPG) approach, which uses a deterministic action function rather than a stochastic one. Its key advantages include effective handling of continuous action spaces, improved sample efficiency through off-policy learning, and faster convergence via its actor–critic architecture that balances value estimation and policy optimization. Furthermore, this technique uses the Grey Wolf Optimization (GWO) algorithm to improve the initial set of gains, resulting in more reliable outcomes and quicker dynamics. The GWO technique is notable for its disciplined and nature-inspired approach, which leads to faster decision-making and greater accuracy than other optimization methods. This method considers the system as a black box without its exact mathematical modeling, leading to lower complexity and computational burden. The effectiveness of this strategy is tested in both modeling and experimental scenarios utilizing the Hardware-In-Loop (HIL) framework, with considerable results and decreased error sensitivity.

1. Introduction

DC/DC power converters have been crucial in industrial applications due to their ability to control output voltage with the fewest possible components. The topologies of the Boost and Buck converters chosen here function in step-up-down modes. Nevertheless, a non-minimum phase situation is caused by the Boost converter’s zero in the right half-plane [1,2]. In high-frequency usage, discontinuous results and exceptional non-linearity are documented for these structures, despite their many benefits. This issue leads to tracking control problems by causing the feedback controller to deviate from the reference signal. Additionally, the system’s states may suffer as a result of the internal stability issue. Different kinds of controllers with distinct features have been developed in order to provide gentle switching and smooth operation [3,4]. In the domain of power electronics, particularly in DC–DC converter applications such as Buck and Boost converters, the inherent nonlinearity of the system dynamics—arising from switch-mode operation, time-varying loads, and input voltage fluctuations—demands the deployment of nonlinear control strategies to ensure robust performance under diverse operating conditions.

In contrast to linear control methods that usually cannot ensure stability and tracking error when there are parameter variations and external disturbances, nonlinear controllers can provide a superior dynamic response, superior voltage regulation, and disturbance rejection. Consequently, nonlinear control techniques are becoming more important because of their capacity to guarantee robust regulation of the voltage, high transient response, and stability in the face of parameter uncertainties [5]. Sliding Mode Control (SMC) is one such technique, and it is very robust; however, it is prone to chattering, which can destroy switching components [6,7]. Adaptive Model Reference Control (MRAC) is a parameter tracking controller that can converge slowly or be sensitive to persistent excitation [8,9]. Feedback linearization is precise and highly sensitive to good modeling [10,11], whereas Passivity-Based Control (PBC) is theoretically robust but lacks flexibility and ease of tuning [12,13,14]. Neural networks and fuzzy logic controllers provide intelligent behavior; however, they require significant computational effort or are not scalable [15,16,17,18,19]. Backstepping Control (BSC) has attracted attention because it is a recursive Lyapunov-based control design method that can ensure global asymptotic stability and robustness to matched disturbances [20,21]. Recent works have also used BSC to power converters with good results in voltage tracking and robustness. To further increase dynamic performance, adaptive BSC frameworks have been presented, where subsystems are controlled using nonlinear control laws and adaptive tuning rules, which allow real-time response to parameter changes [22]. These adaptive techniques have greatly enhanced transient response and minimal steady-state error under variable operating conditions. There are, however, two major limitations. First, adaptive BSC is very sensitive to the design of an efficient online adaptation mechanism, which can be computationally demanding and vulnerable to noise. Second, the controller must use gain values that are initially well tuned, which are typically obtained through trial and error or by empirical estimation and therefore influence the convergence speed and overall performance.

The last few years have seen the introduction of Artificial Intelligence (AI) and Machine Learning (ML) approaches, especially Reinforcement Learning (RL), into the controllers of power converters, which can offer the prospect of online optimization, real-time adaptation, and better dynamic response under uncertain circumstances [23,24]. Conventional control techniques have proved to be effective in controlling nonlinear systems; however, they are commonly fixed gain/precisely model-based and lack flexibility. To overcome these limitations, there has been recent interest in integrating RL and classical approaches. As an example, the combination of DDPG with high-order SMC observer has demonstrated improved voltage regulation of DC–DC Boost converters, with reduced chattering and steady-state error [25]. In a similar manner, RL-aided MPC frameworks have been introduced, where RL agents adapt the weighting factors in Finite-Set MPC, decreasing switching frequency and enhancing harmonic performance in power converters [26]. In another work, fuzzy logic-based Q-learning controllers were designed for VSC-HVDC systems, which provide adaptive reference tracking and lessen reliance on the empirical tuning of PI gains [27]. Reinforcement learning was also successfully used in CLLLC converters to optimize performance over a wide range of operating points, thus considerably easing the design task [28]. Furthermore, the authors have paid attention to safety-aware RL implementations, where online agents optimize the control of the converter without unstable learning-induced behavior [29]. Various publications show that it is also possible to directly use RL to regulate voltages and currents. Specifically, deep Q-networks (DQNs) and DDPG agents have been used to control Buck and Boost converters to track voltage, giving better transient response and flexibility compared to a fixed-gain controller [12]. Another model-free DRL approach was developed to stabilize the output voltage of a DC microgrid when using a Buck converter, and it demonstrated dependable performance without modeling the system in detail [30]. Likewise, discrete-time RL-based controllers have been demonstrated to be superior to conventional PID in efficiency and robustness when applied to a practical converter system [31]. The application of RL to nonlinear Boost converter control by Zandi has shown that RL can autonomously learn the optimum duty cycles when the load and voltage vary, and it outperforms manually tuned gains [32]. Combined, these studies highlight that RL can be used both as a controller on its own and as a versatile adaptive layer on top of traditional control architectures, improving their resilience and reactivity in nonlinear and time-varying environments. Some of the notable RL algorithms used include Twin-Delayed DDPG (TD3), Proximal Policy Optimization (PPO), Dueling DQN, and DDPG [33,34,35,36]. Actor–critic algorithms, such as DDPG and TD3, have the advantage of being able to handle continuous control problems and converge better than DQN and Dueling DQN, which makes them suitable for power electronics [34]. In particular, the DDPG algorithm is favorable to be used in continuous-action spaces like power converters, as it is off-policy and sample efficient [37]. As such, this paper suggests the adoption of a DDPG-based RL algorithm into our Adaptive Backstepping Controller (ABSC) framework in order to leverage the Lyapunov-based stability of the controller and the adaptive nature of DDPG. Nonetheless, an outstanding issue has yet to be addressed: the system’s performance is subject to the originally tuned controller gains. Metaheuristic Algorithms (MAs) can be used to address this concern in the optimization process.

MAs have shown significant outcomes in handling complex models, such as high-ordered equations or unpredictable variations [38,39]. They are also beneficial in the modes of exploitation and exploration for both new search areas and existing search areas [40]. Recently, the utilization of metaheuristic algorithms for tuning controllers in power converters and motor drives has been gaining popularity. MAs, such as Particle Swarm Optimization (PSO), Ant Lion Optimizer (ALO), and Genetic Algorithms (GAs), can enhance a controller’s overall performance by optimizing parameters that are not effectively adjusted via traditional methods [41,42,43,44]. The integration achieved has led to improvements in the response of a given system, including enhanced transient response, increased robustness to disturbances, and reduced overshoot. MAs offer flexibility and global search methodologies, enabling the effective optimization of complex, nonlinear, and multi-modal optimization problems. This makes them suitable for controller tuning in power converters and motor drives. The GWO algorithm was developed as a flexible metaheuristic technique inspired by the hunting and mating behaviors of wolves [45]. The GWO algorithm separates individuals into groups, making wolf-like movements to explore and efficiently exploit the search space. It has outperformed traditional optimization methods for complex problems by first searching or exploring the space and then exploiting solutions that improve towards the best solution [46]. Recently published research has shown that the GWO algorithm has the potential to be applied to several optimization problems with maximum performance, notably the optimization problem of control parameters for quantum processors, solving high-dimensional, non-convex problems with state-of-the-art performance from the systems. In association with renewable energy, an enhanced GWO algorithm has been utilized for maximum power point tracking of photovoltaic systems, providing improved convergence speed, accuracy, and greater stability of the systems during partial shading conditions [47,48]. The advantages of this work are improved optimization efficiency, more reliable solutions, and the ability to optimize in complex high-dimensional search spaces. It is essential to keep in mind that the design of the GWO algorithm to control and manage the balance between exploration and sampling is substantial, as it is a practical optimization algorithm for designing controllers of power converters and motor drives optimally, with the potential for improvement in system performance and robustness.

The novelty of this work relies on presenting a robust adaptive backstepping controller enhanced by Lyapunov-based stability, integrating the RL-DDPG algorithm to optimize its control parameters online, leading to better disturbance-rejection behavior and higher adaptation to the working environment. It also benefited from the GWO algorithm to find the best-tuned initial gains for the controller block, leading to faster convergence and lower sensitivity to errors. This work is designed based on a black box strategy to reduce dependency on system modeling, resulting in lower complexity burden and ease of implementation in real-time applications. The primary contributions of this approach areas follows:

• An adaptive backstepping control approach is designed for Boost and Buck converters, which is improved in dynamics using the Lyapunov theorem. This approach can guarantee accurate monitoring by reducing susceptibility to error.
• The ABSC approach is enhanced with the RL-DDPG method to optimize control parameters online to reach better disturbance-rejection behavior and higher adaptability to the working environment.
• The GWO algorithm is adopted as an evolutionary computation part to initially optimize the parameters of the system for faster convergence in the controller block, which leads to better adaptation to the system frequency behavior.
• This is a model-free controller that lowers the dependency on exact mathematical modeling of the system, providing faster dynamics and better disturbance-rejection behavior.
• The use of HIL setups for real-time testing provides a rigorous validation of the system’s performance in both simulation and experimental environments, enhancing the credibility and practical relevance of the results.

2. Mathematical Modeling of the Converters

2.1. Boost Converter

This is a non-isolated step-up power converter widely used in energy conversion systems. It operates by switching a semiconductor device (usually a MOSFET) in two states, controlled through a Pulse Width Modulation (PWM) signal. The basic configuration of this topology is shown in Figure 1a. Looking at the two switching functions in Figure 1b,c, we define the averaged state-space model of the converter [1].

2.2. State Variables and Input Definition

We can define the state vector, input, and output as follows [2]:

$$\begin{array}{cc}\hfill x\left(t\right)& =\left[\begin{array}{c}{i}_L\left(t\right)\\ V_C\left(t\right)\end{array}\right],\hfill \\ \hfill u\left(t\right)& =E\left(t\right)(\mathrm{Input}\mathrm{voltage}),\hfill \\ \hfill y\left(t\right)& =V_C\left(t\right)(\mathrm{Output}\mathrm{voltage}).\hfill \end{array}$$

The Boost converter dynamics can be modeled in the standard state-space form:

$$\dot{x}\left(t\right)=Ax\left(t\right)+Bu\left(t\right),y\left(t\right)=Cx\left(t\right)$$

(1)

2.3. Circuit Equations

2.3.1. Switch ON (Q Closed)

During the ON state (switch is conducting), the inductor is charged and the diode is reverse biased. The circuit equations are as follows:

$$\begin{array}{cc}\hfill {\displaystyle \frac{d{i}_L}{dt}}& ={\displaystyle \frac{E}{L}}\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill {\displaystyle \frac{d{V}_C}{dt}}& =-{\displaystyle \frac{V_C}{RC}}\hfill \end{array}$$

(3)

2.3.2. Switch OFF (Q Open)

During the OFF state (switch is open), the inductor discharges through the diode to the output:

$$\begin{array}{cc}\hfill {\displaystyle \frac{d{i}_L}{dt}}& ={\displaystyle \frac{E-V_C}{L}}\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\displaystyle \frac{d{V}_C}{dt}}& ={\displaystyle \frac{i_L-\frac{V_C}{R}}{C}}\hfill \end{array}$$

(5)

2.4. Averaged State-Space Model

By averaging the dynamics over a switching cycle with duty ratio D, we obtain the averaged state-space equations [1]:

$$\dot{x}\left(t\right)=\left[\begin{array}{c}{\dot{i}}_L\\ {\dot{V}}_C\end{array}\right]=Ax\left(t\right)+Bu\left(t\right)$$

(6)

where the matrices A and B are as follows:

$$A=D\left[\begin{array}{cc}0& 0\\ 0& -{\displaystyle \frac{1}{RC}}\end{array}\right]+(1-D)\left[\begin{array}{cc}0& -{\displaystyle \frac{1}{L}}\\ {\displaystyle \frac{1}{C}}& -{\displaystyle \frac{1}{RC}}\end{array}\right]$$

(7)

$$B=\left[\begin{array}{c}{\displaystyle \frac{1}{L}}\\ 0\end{array}\right]$$

(8)

$$C=\left[\begin{array}{cc}0& 1\end{array}\right]$$

(9)

2.5. Buck Converter

The Buck converter is a step-down DC–DC converter that reduces the input voltage to a lower output level. It operates by rapidly switching a transistor to control energy transfer to the load. The basic structure is shown in Figure 2. Based on the two switching functions in Figure 2b,c, one can define the averaged state-space model of the converter.

2.6. State Variables and Input Definition

We define the system as a two-state system with:

$$\begin{array}{cc}\hfill x\left(t\right)& =\left[\begin{array}{c}{i}_L\left(t\right)\\ V_C\left(t\right)\end{array}\right](\mathrm{state}\mathrm{vector})\hfill \\ \hfill u\left(t\right)& =E\left(t\right)(\mathrm{input}\mathrm{voltage})\hfill \\ \hfill y\left(t\right)& =V_C\left(t\right)(\mathrm{output}\mathrm{voltage})\hfill \end{array}$$

2.7. Mode 1: Switch ON (Q Closed)

The general state-space representation is given by Equation (1). When the switch is ON:

• Diode is reverse-biased.
• Inductor is connected directly to the source.
• Capacitor supplies the load and is charged.

The governing equations are as follows:

$$\begin{array}{cc}\hfill {\displaystyle \frac{d{i}_L}{dt}}& ={\displaystyle \frac{E-V_C}{L}}\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill {\displaystyle \frac{d{V}_C}{dt}}& ={\displaystyle \frac{i_L-\frac{V_C}{R}}{C}}\hfill \end{array}$$

(11)

2.8. Mode 2: Switch OFF (Q Open)

When the switch is OFF:

• Diode conducts.
• Inductor current freewheels through the diode.

The governing equations remain the same

$$\begin{array}{cc}\hfill {\displaystyle \frac{d{i}_L}{dt}}& ={\displaystyle \frac{-V_C}{L}}\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\displaystyle \frac{d{V}_C}{dt}}& ={\displaystyle \frac{i_L-\frac{V_C}{R}}{C}}\hfill \end{array}$$

(13)

2.9. Averaged State-Space Model

Using the duty cycle $D\in [0,1]$, the averaged state-space model in the continuous conduction mode (CCM) is as follows:

$$\dot{x}\left(t\right)=\left[\begin{array}{c}{\dot{i}}_L\\ {\dot{V}}_C\end{array}\right]=Ax\left(t\right)+Bu\left(t\right)$$

(14)

where the matrices are given as follows:

$$A=\left[\begin{array}{cc}0& -{\displaystyle \frac{1}{L}}\\ {\displaystyle \frac{1}{C}}& -{\displaystyle \frac{1}{RC}}\end{array}\right],B=\left[\begin{array}{c}{\displaystyle \frac{D}{L}}\\ 0\end{array}\right]$$

(15)

$$C=\left[\begin{array}{cc}0& 1\end{array}\right]$$

(16)

Based on the state-space matrices defined for both of the converters, we can assume the following second-degree transfer function for the further considerations:

$$G\left(s\right)={\displaystyle \frac{k_1}{s^2+k_{2}s+k_3}}$$

(17)

Equation (17) expresses the simplified second-order transfer function of the Buck and Boost converters, where the dominant dynamics are governed by the inductor current and output voltage. This form is used to define the system order for the backstepping design, without relying on precise converter parameters. The assumption of a second-order system is justified by the averaged continuous-conduction-mode model of Buck and Boost converters, where higher-order effects and parasitics are neglected. This allows the proposed controller to be designed in a black box manner, while the RL agent adapts the gains to compensate for uncertainties and unmodeled dynamics. It should be noted that the state-space equations are only used to establish the system order (second order for Buck and Boost converters), whereas the DDPG agent operates in a black box manner, learning control gains directly from data without requiring explicit converter parameters. The proposed controller is a mode-free controller that can regulate both converters without an exact mathematical model of the systems. The values of the parameters used in both converters are listed in

Table 1. Values and their definitions in Buck and Boost modes.

Component	Definition	Boost	Buck
$V_{out}$	Output Voltage	60–120 V	50–120 V
E	Supply Voltage	10–50 V	120–60 V
L	Inductor	2 μH	2 μH
R	Resistor (Load)	50 $\mathrm{\Omega }$	50 $\mathrm{\Omega }$
C	Capacitor	100 μF	100 μF
f	Switching Frequency	20 kHz	20 kHz

.

3. Controllers

Adaptive Backstepping Technique

To make a state-augmented adaptive backstepping controller requires error equations and an extra condition that is added to the system’s equations based on error integration. In addition, system terms are broken down into subsystems, and the Lyapunov function is used to create a simulated control signal for each subsystem that tries to keep it stable. All of the system equations will be looked at in the last step of creating the backstepping controller, and the control output will be made in a way that the Lyapunov function can guarantee system stability [15].

This stage’s goal is to use the Lyapunov function to stabilize these subsystems. In order to create the control signal based on guaranteeing system stability by the Lyapunov function, all of the BSC design’s equations will be met [15]. Equation (18) is the error equation:

$$e\left(t\right)=V_r\left(t\right)-V_o\left(t\right)$$

(18)

In (18), the error is the product of the output voltage ($V_o$) and the reference voltage ($V_r$). In (19), we introduce three elements: the error itself, the differentiation of the error, and an additional form that employs the weighing integration of the error:

$$\begin{array}{cc}\hfill x_1\left(t\right)& =n{\int }_0^{t}e\left(\tau \right)d\tau ,\hfill \\ \hfill x_2\left(t\right)& =e\left(t\right),\hfill \\ \hfill x_3\left(t\right)& =e(t-1).\hfill \end{array}$$

(19)

To design the control signal for the proposed controller, the discrete transfer function of the system is needed, which has been defined based on Equation (20) using Zero-Order Hold (ZOH) method:

$${\displaystyle \frac{V_o}{u}}={\displaystyle \frac{P_{3}z-P_4}{z^2-P_{1}z+P_2}}$$

(20)

In Equation (20), $P_1\dots P_4$ are coefficients of the system.

The simplification process for the following steps is described by the author in [20] based on the time domain. Next, new up-to-date conditions can be introduced:

$$\begin{array}{cc}\hfill x_1(t-1)& =n{x}_2\left(t\right),\hfill \\ \hfill x_2(t-1)& =x_3\left(t\right),\hfill \\ \hfill x_3(t-1)& =P_3\left(-u(t-1)+ry\right).\hfill \end{array}$$

(21)

Consequently, using the process described in [20], the final control law can be defined in Equation (22).

$$\begin{array}{c}\hfill u(t-1)={\displaystyle \frac{1}{P_3}}[V_r(t-2)-P_1{V}_o(t-1)+\\ \hfill P_2{V}_o\left(t\right)+P_{4}u(t-2)+k_1{x}_3\left(t\right)+k_2{x}_2\left(t\right)+n{x}_2\left(t\right)+\\ \hfill k_3\left(x_3\left(t\right)+{\displaystyle \frac{k_2}{n}}{x}_1\left(t\right)\right)+k_1{x}_2\left(t\right)].\end{array}$$

(22)

In Equation (22), the gains of the controller are shown as $k_1$, $k_2$, and $k_3$. The control law is changed to reflect the adaptation mechanism as follows:

$$\begin{array}{cc}\hfill u(t-1)={\displaystyle \frac{1}{{\widehat{P}}_3}}(& V_r(t-2)-{\widehat{P}}_1{V}_o(t-1)+{\widehat{P}}_2{V}_o\left(t\right)+{\widehat{P}}_{4}u(t-2)+k_1{x}_3\left(t\right)+\hfill \\ \hfill k_2{x}_2\left(t\right)+n{x}_2\left(t\right)& +k_3\left(x_3\left(t\right)+{\displaystyle \frac{k_2}{n}}{x}_1\left(t\right)+k_1{x}_2\left(t\right)\right)).\hfill \end{array}$$

(23)

Moreover, by replacing the control law in the system equations, the error dynamics equations can be formed as follows (24):

$$X(t-1)=AX\left(t\right)+Bu,$$

(24)

$$\left[\begin{array}{c}{x}_1(t-1)\\ x_2(t-1)\\ x_3(t-1)\end{array}\right]=\left[\begin{array}{ccc}0& 1& 0\\ 0& 0& 1\\ -{\displaystyle \frac{k_3{k}_2}{n}}& -k_2-n-k_3{k}_1& -k_1-k_3\end{array}\right]\left[\begin{array}{c}{x}_1\left(t\right)\\ x_2\left(t\right)\\ x_3\left(t\right)\end{array}\right]+\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right]{\widehat{P}}_3{s}^{T}y$$

where

$$s^T=\left[\left({\displaystyle \frac{1}{{\widehat{P}}_3}},{\displaystyle \frac{1}{{\widehat{P}}_3}}\right)\left({\displaystyle \frac{{\widehat{P}}_1-{\overline{P}}_1}{{\widehat{P}}_3}}\right)\left({\displaystyle \frac{{\widehat{P}}_2-{\overline{P}}_2}{{\widehat{P}}_3}}\right)\left({\displaystyle \frac{{\widehat{P}}_3-{\overline{P}}_3}{{\widehat{P}}_3}}\right)\left({\displaystyle \frac{{\widehat{P}}_4-{\overline{P}}_4}{{\widehat{P}}_3}}\right)\right],y=\left[\begin{array}{c}{V}_o(t-2)\\ V_o(t-1)\\ V_o\left(t\right)\\ u(t-2)\end{array}\right].$$

The selected Lyapunov function for analyzing stability is:

$$v\left(t\right)={\displaystyle \frac{1}{2}}{X}^T\left(t\right)PX\left(t\right)+{\displaystyle \frac{1}{2\gamma }}{s}^{T}s$$

(25)

where P is a symmetrical positive definite matrix. Furthermore, differentiating (25) results in the following:

$$v(t-1)={\displaystyle \frac{1}{2}}{X}^T(t-1)PX\left(t\right)+{\displaystyle \frac{1}{2}}{X}^T\left(t\right)PX(t-1)+{\displaystyle \frac{1}{\gamma }}{s}^{T}s.$$

(26)

The third term in Equation (26) originates from the derivative of the cross-term $\frac{1}{2\gamma }{s}^{T}s$ in Equation (25). In the discrete-time expansion, this produces the additional $\frac{1}{\gamma }{s}^{T}s$ contribution. This ensures that the Lyapunov function properly accounts for the adaptive signal energy across successive time steps. Then, consequently, for the Lyapunov function and the parameters of control law, the below equations are presented:

$$\begin{array}{cc}\hfill \dot{\widehat{s}}& =\gamma X^T\left(t\right)PB{\widehat{P}}_{3}y,\hfill \\ \hfill v(t-1)& =-{\displaystyle \frac{1}{2}}{X}^T\left(t\right)QX\left(t\right),\hfill \\ \hfill v(t-2)& =-X^T(t-1)QX\left(t\right).\hfill \end{array}$$

(27)

The process of designing the controller is shown in [20] using a mathematical description and the role of other adopted methods in developing this structure. To optimize the gains of the controller ($k_1$, $k_2$, $k_3$), we have combined the controller with RL to achieve better results for the controller, which is described in the following sections.

4. RL-Based Optimization of ABSC Gains

To enhance both performance and adaptability of the ABSC approach, a model-free reinforcement learning strategy is employed. Specifically, the DDPG algorithm is used to optimize three essential control gains, $k_1$, $k_2$, and $k_3,$ in real time [12].

4.1. Problem Setup

The controller aims to generate an optimal control law using adaptive gains:

$$\mathbf{k}\left(t\right)=\left[\begin{array}{c}{k}_1\left(t\right)\\ k_2\left(t\right)\\ k_3\left(t\right)\end{array}\right]$$

(28)

This task is cast as a continuous control problem modeled using a Markov Decision Process (MDP) with:

• State:

  $$s\left(t\right)=\left[e\left(t\right),\dot{e}\left(t\right),\int e\left(t\right)dt,V_o\left(t\right)\right]$$

  (29)
• Action:

  $$a\left(t\right)=\mathbf{k}\left(t\right)$$

  (30)
• Reward: A function penalizing poor tracking and excessive control effort.

The reward function is defined as follows:

$$r\left(t\right)=-\left[{\alpha }_1{e}^2\left(t\right)+{\alpha }_2{u}^2\left(t\right)+{\alpha }_3{\left({\displaystyle \frac{d{V}_o\left(t\right)}{dt}}\right)}^2\right]$$

(31)

The coefficients ${\alpha }_1$, ${\alpha }_2$, and ${\alpha }_3$ in the reward function were selected to balance tracking accuracy, control effort, and output smoothness. A sensitivity analysis was conducted to determine the most suitable values for these parameters. Specifically, ${\alpha }_1$ was assigned the highest weight to prioritize precise voltage tracking, ${\alpha }_2$ was chosen as smaller to limit excessive switching activity, and ${\alpha }_3$ was selected to penalize abrupt variations in the output voltage, thereby enhancing noise robustness. Based on simulation experiments, the final values were set as ${\alpha }_1$ = 10, ${\alpha }_2$ = 0.5, and ${\alpha }_3$ = 1, which provided the most favorable balance between fast convergence, low steady-state error, and robustness in both simulation and HIL validation.

4.2. DDPG Agent Architecture

DDPG is an actor–critic DRL algorithm intended to work with continuous action spaces. It uses the advantages of deterministic policy gradients, and the policy (actor) can output continuous-valued actions directly with deep function approximation. DPG is an actor–critic model, and as such, has two neural networks: an actor network, which takes states as inputs and outputs actions (e.g., controller gains), and a critic network, which takes pairs of states and actions as inputs and outputs the Q-value of the state–action pair to guide learning. A key benefit of DDPG is that it can learn optimal control policies in real time over nonlinear and high-dimensional systems without the need of an explicit mathematical model of the environment. Also, DDPG has more stable and converging training with the use of experience replay and target networks. These properties render DDPG especially suitable in control applications, including tuning gains in ABSC, where continuous, adaptive learning of policy is necessary in a dynamic context [37]. The DDPG agent includes:

• An actor network $\mu \left(s\right|{\theta }^{\mu })$, which outputs continuous-valued gains $\mathbf{k}$.
• A critic network $Q(s,a|{\theta }^Q)$, which estimates expected return.
• Target networks ${\mu }^{\prime }$ and $Q^{\prime }$ for stable learning.
• A replay buffer for experience replay and sample efficiency.

4.3. Mathematical Formulation

The critic is trained to minimize the Bellman loss:

$$L\left({\theta }^Q\right)={\mathbb{E}}_{s,a,r,s^{\prime }}\left[{\left(Q(s,a\mid {\theta }^Q)-y\right)}^2\right]$$

(32)

with target:

$$y=r+\gamma Q^{\prime }(s^{\prime },\pi (s^{\prime }\mid {\theta }^{{\mu }^{\prime }})\mid {\theta }^{Q^{\prime }})$$

(33)

where ${\theta }^{Q^{\prime }}$ and ${\theta }^{{\mu }^{\prime }}$ denote the parameters of the target critic and actor networks, respectively.

The actor is trained using the deterministic policy gradient:

$${\nabla }_{{\theta }^{\mu }}J={\mathbb{E}}_s\left[{\nabla }_{a}Q(s,a\mid {\theta }^Q){|}_{a=\pi \left(s\right)}{\nabla }_{{\theta }^{\mu }}\pi (s\mid {\theta }^{\mu })\right]$$

(34)

During training, exploration is ensured by injecting noise into the actor’s action:

$$a_t=\pi (s_t\mid {\theta }^{\mu })+{\mathcal{N}}_t$$

(35)

where ${\mathcal{N}}_t$ is generated by an Ornstein–Uhlenbeck process to provide temporally correlated exploration.

The generated adaptive gains $k\left(t\right)$ are injected into the ABSC controller to calculate the following:

$$u\left(t\right)=f\left(e\left(t\right),\dot{e}\left(t\right),\int e\left(t\right)dt;k\left(t\right)\right)$$

(36)

4.4. Learning Loop and Training Behavior

At each time step:

• Observe system state $s\left(t\right)$.
• Actor network outputs gains $k\left(t\right)$.
• Controller computes $u\left(t\right)$ using these gains.
• System returns new state $s(t+1)$ and reward $r\left(t\right)$.
• Transition $(s,a,r,s^{\prime })$ is stored in the replay buffer.
• Critic and actor networks are updated from sampled mini-batches using Equations (32)–(35).
• Target networks are updated with a soft update rule to improve stability.

Training Behavior: In practice, the cumulative reward is increased slowly as training progresses, and this shows some fluctuations in the first episodes due to exploration noise. Initialization of the controller gains with Grey Wolf Optimization (GWO) will significantly reduce unstable exploration at the beginning of training and increase the speed of convergence. As training continues, the actor network stabilizes, and stable adaptive gains are achieved that optimize ABSC performance. The given behavior proves that the DDPG agent has learned a stable control policy.

4.5. Stability Consideration

To maintain stability, the Lyapunov condition is monitored:

$$\dot{V}\left(t\right)={\displaystyle \frac{d}{dt}}\left({\displaystyle \frac{1}{2}}{X}^T\left(t\right)PX\left(t\right)+{\displaystyle \frac{1}{2\gamma }}{s}^{T}s\right)\le 0$$

(37)

The reward function penalizes unstable responses, guiding learning toward safe regions in the control space. Figure 3 shows a general clarification of the RL algorithm and its working process.

In addition to the Lyapunov-based adaptive backstepping structure, the proposed controller relies on a RL agent implemented with deep neural networks. To ensure reproducibility and clarity, the neural network architecture, training procedure, and implementation details of the DDPG agent are described in the following section.

4.6. Neural Network Training and Architecture

The actor–critic networks of the DDPG agent were implemented as fully connected feedforward neural networks shown in Figure 4. The complete architecture is shown in Figure 4a, while the training and regression performance are summarized in Figure 4b. The main hyperparameters and settings used in training are listed in

Table 2. Neural network architecture and training parameters used in the DDPG agent.

Component	Description
Actor network architecture	Input layer: 3 neurons $[e,\dot{e},\int e]$; hidden layer 1: 10 neurons (ReLU); hidden layer 2: 6 neurons (ReLU); output layer: 3 neurons $(k_1,k_2,k_3)$ with tanh activation scaled to safe gain ranges (Figure 4a).
Critic network architecture	Inputs: state vector + action vector; hidden layers: 2 layers of 64 neurons each (ReLU activations); output: single Q-value (linear).
Optimizers	Adam; learning rate ${10}^{-4}$ for actor, ${10}^{-3}$ for critic.
Replay buffer	Capacity: 100,000 transitions; mini-batch size: 64.
Target networks	Soft update with factor $\tau =0.005$.
Discount factor	$\gamma =0.99$.
Exploration strategy	Ornstein–Uhlenbeck noise ($\theta =0.15$, $\sigma$ decays from 0.2 to 0.05).
Training schedule	300 episodes, 6000 steps per episode; warm-up of 5000 random steps before training begins.
Regularization	Gradient clipping (max norm = 1.0) and normalization of state observations.
Initialization	Gains initialized offline using Grey Wolf Optimization (GWO), ensuring stable exploration from the start.
Training outcome	Cumulative reward increased steadily; fluctuations in early episodes due to noise; convergence achieved as actor policy stabilized. Training/test losses converge towards zero (Figure 4b).

.

Though the DDPG policy allows real-time control gain adaptation, convergence performance and quality of learning strongly depend on the initial gain values. Improperly initialized gains may result in unstable exploration, reduced convergence rates, or even the inability to meet the Lyapunov-based stability condition early in learning. An initial policy should thus be well selected to speed up the reinforcement learning and lead it to an optimal solution.

To solve this problem, we incorporate the GWO algorithm as a meta-heuristic pre-training technique to optimize the initial gain values $k_1$, $k_2$, and $k_3$ before feeding them to the DDPG agent. The GWO algorithm is an imitation of the hierarchy of the leadership and hunting process of grey wolves and has proved to be very effective in global search processes in optimization control applications. In the next section, we describe in detail the GWO algorithm and how it was implemented to initialize the reinforcement learning block.

4.7. Grey Wolf Optimization for Initial Gain Tuning

To improve the initialization of the reinforcement learning agent, the Grey Wolf Optimization (GWO) algorithm is employed as a global search strategy to identify near-optimal initial values for the controller gains $k_1$, $k_2$, and $k_3$ before launching the DDPG learning process. GWO is a nature-inspired, population-based metaheuristic that mimics the social hierarchy and hunting behavior of grey wolves. The population is divided into different main wolves. The top three solutions ($\beta$, $\alpha$, and $\delta$) guide the movement of the rest of the population. The search process includes three main phases: encircling prey, hunting, and attacking, all modeled mathematically to update the position of each search agent. One can study detailed benchmarking of this algorithm in [22].

Comparative Analysis of Optimization Strategies

In this study, GWO is applied to minimize a cost function composed of tracking error and control smoothness, which shares the same structure as the reward function used by the DDPG agent. Once the optimal gain vector is identified through GWO, it is used to initialize the actor network parameters of the RL controller, thus improving the training convergence and stability of the policy [12].

In order to ensure a fair comparison between the proposed GWO initializer and other metaheuristic algorithms, the same set of adaptive controller gains $(k_1,k_2,k_3)$ was optimized using identical search ranges and cost function. The algorithms differ only in their search strategies and specific hyperparameters, as summarized in

Table 3. Parameter configuration and selected gains of metaheuristic algorithms for initial controller gain tuning.

Algorithm	Population (P)	Iterations (T)	Dimension (d)	Algorithm-Specific Settings	Selected Gains $({\mathit{k}}_1,{\mathit{k}}_2,{\mathit{k}}_3)$
GWO	30 wolves	100	3	Coefficient vectors $\alpha ,\beta$	(10.2, 5.1, 0.8)
ALO	30 ants	100	3	Roulette wheel selection	(11.4, 5.6, 0.9)
PSO	30 particles	100	3	Inertia $w=0.7$, $c_1=c_2=1.5$	(9.8, 4.9, 0.7)
GA	30 chromosomes	100	3	Crossover = 0.8, mutation = 0.1	(12.0, 6.2, 1.0)

.

All algorithms were applied to the same three adaptive gains $(k_1,k_2,k_3)$ under identical search ranges and cost function. The population size, iteration count, and search dimension were fixed equally across algorithms to guarantee a fair comparison. Algorithm-specific settings (e.g., inertia in PSO, crossover and mutation rates in GA) are reported for reproducibility. All algorithms were executed with the same population size (30), number of iterations (100), and search dimension ($d=3$, corresponding to the three adaptive gains).

To demonstrate the superiority of the GWO-based initialization approach, we conducted a comparative evaluation against other common metaheuristics, including PSO, GA, and ALO. The performance metrics used in the comparison include:

• Convergence speed: Number of iterations to reach 95% of minimum cost.
• Final tracking error: Mean square error after optimization.
• Control effort: Sum of squared control input.
• Stability margin: Evaluated through Lyapunov derivative analysis.

The results in

Table 4. Comparison of optimization methods for controller gain initialization.

Algorithm	Convergence Iteration	Final IAE	Std. Deviation	Param. Sensitivity	CPU Time (s)	Final Fitness
GWO	52	1.62	0.007	1.5%	17.4	0.125
ALO	57	1.73	0.009	1.8%	18.6	0.138
PSO	74	2.38	0.026	3.7%	19.8	0.196
GA	85	2.61	0.033	4.2%	31.4	0.209

clearly demonstrate the effectiveness of GWO in identifying better starting points for the reinforcement learning agent, which contributes to faster convergence and improved stability.

As shown, the GWO algorithm achieves the best overall performance across all metrics. It converges faster, maintains high robustness with minimal variation, exhibits low sensitivity to parameter changes, and produces the lowest cost function (fitness score). This justifies its selection as the optimal metaheuristic algorithm for fine-tuning the RL-ABSC method in this work. The overall schematic diagram of this work is depicted in detail in Figure 5.

Table 5. Computational complexity and real-time feasibility of controllers.

Controller	Online Effort	Memory	Sampling Time (µs)	Exec. Time (µs)	Delay (µs)
ABSC	Few algebraic ops. (fixed)	low	100	10	3
RL–PID	Error calc., sum/diff, clamp	low	100	5	2
RL–ABSC	Actor NN + ABSC algebra	Moderate	100	25	5

summarizes the computational complexity and practical real-time feasibility of the three controllers considered in this study. Both ABSC and RL–PID require only a fixed number of algebraic operations at each sampling instant, resulting in negligible processor load (<10%). The proposed DDPG+GWO–ABSC, while more demanding due to the actor network evaluation, remains computationally efficient with an execution time of approximately 30 µs at a 100 µs sampling period, corresponding to about 25% CPU utilization. The one-time grey wolf optimization is performed offline for initialization and does not impact the hardware loop. These results confirm that all controllers are real-time feasible on the TI LaunchXL-F28379D platform, with the proposed method offering enhanced adaptability at the cost of modestly higher computational requirements.

5. Result of Simulations

This section presents the results of the RL-ABSC across different operating scenarios based on the circuit specifications provided in Table 1. To begin, the controllers are evaluated under nominal conditions, without external disturbances, to assess their ability to maintain stable control. Figure 6 illustrates the tracking performance of the applied control strategies for the Boost converter with a resistive load, shown over a time domain in seconds.

5.1. Case 1: Output Regulation

In the first scenario, the controller’s performance was thoroughly assessed without any external disturbances in an operating range of 50 V supply voltage for the Boost and 100 V for the Buck converters, fed to a resistive load (R = 50 Ω).

Figure 6 shows the efficiency of the RL-PID controller in tracking reference values of 80–120 V in the Boost mode and 50–80 V in the Buck mode, whereas the conventional RL-PID and BSC show significant overshooting and signal distortion. The details in Figure 5 shows that the BSC has reached the reference signal faster, but it suffers from fluctuation and sensitivity to error, making it an undesirable alternative in this case. The RL-PID controller exhibits quicker dynamics and a reduced performance in term of overshooting and undershooting, which are of critical importance in practical applications. Moreover, the RL-ABSC method has significant performance in both convergence speed and lower sensitivity to error, ensuring its robustness in these structures.

In addition to the qualitative waveforms shown in Figure 6, the main quantitative performance indices of the three controllers (ABSC, RL–PID, and the proposed DDPG+GWO–ABSC) are summarized in

Table 6. Quantitative performance metrics of controllers in Buck and Boost modes (corresponding to Figure 6).

Controller	Case	Rise Time (ms)	Overshoot (%)	Settling Time (ms)	RMSE (V)	IAE
ABSC	Buck	6.5	3.8	12.0	0.42	5.1
	Boost	7.2	4.1	13.5	0.47	5.6
RL–PID	Buck	4.2	6.7	10.5	0.36	4.3
	Boost	4.8	7.2	11.0	0.41	4.9
Proposed RL–ABSC	Buck	3.1	2.1	7.8	0.25	3.2
	Boost	3.6	2.5	8.4	0.29	3.5

. These indices include rise time, overshoot, settling time, RMSE, and IAE, providing a numerical comparison for both Buck and Boost modes. The results confirm that the proposed controller achieves the lowest overshoot, fastest rise and settling times, and reduced error.

To confirm the efficiency and robustness of the proposed controller, we experiment with a wide variety of scenarios. In the former case, we varied the output voltage (reference signal). In order to do this, we recreated three varying and abrupt output voltage changes that are likely to vary. Compensation of shifting trajectory references is a major challenge to controllers. Because of its strong responses and model-free architecture, the RL-ABSC approach can follow the reference signal with rapid dynamism, as shown in Figure 7 for both operational converters.

The gains of the BS controller, on the other hand, are inappropriate for these demanding circumstances; they experience saturation and find it difficult to accurately follow these voltage references. Additionally, because of its appropriate gain tuning procedure, the RL-PID exhibits appropriate behavior and quick tracking in these changes. The negative effects that variations in the tracking performance of traditional techniques might have on the internal components thus highlight the vital need for an online tuning mechanism in practical applications.

5.2. Case 2: Supply Voltage Variation

Another difficult condition is when the supply voltage fluctuates. The supply voltage is sometimes equated to a battery charged with renewable energy. We try to manage the output voltage while taking these uncertainties into account in order to meet this requirement. Therefore, Figure 8 tests the controllers’ performance under this scenario, and the converter’s three-step alterations are investigated. Both BS-based controllers are insensitive to these changes, according to the results for the Boost converter; however, the RL-PID controller experiences more digression and slower dynamics in overcoming these variations. BSC has, however, demonstrated saturation in tracking abrupt changes in the supply voltage when operating in Buck mode. The RL-ABSC technique accounts for the effects of these uncertainties with appropriate reactions.

5.3. Case 3: Load Uncertainty

The next scenario concerns the introduction of uncertainties into the structure of the system, which is a crucial issue for converter controllers. Unfavorable operating conditions may be the source of the system’s structural uncertainties. As shown in Figure 9, we abruptly changed the load on the converters to mimic the impact of parametric fluctuations on controller performance.

In Figure 9, sudden load variations are examined with a variation of 50 Ω to 10 Ω at 0.6 s and then from 10 Ω to 100 Ω at 1.2 s for both converters. In the Buck mode, BSC could not handle the first variation and faced saturation in its dynamics, while the RL-PID controller showed batter adaptation with slower dynamics than the RL-ABSC method. However, looking at Figure 9, it is obvious that all of the controllers are showing proper regulation in these cases, but BS-based controllers are showing higher adaptability in these variations.

5.4. Case 3: Noise Impact

It is an indisputable fact that converters are working in real-time application under various disturbances. In addition, noise is one of the most unpredictable circumstance in these scenarios, with high impact on the converter’s components, which can lead to malfunction or error. To validate the performance of the controller under these challenging cases, we have injected different noise in the ranges of 0.5–2 variance. Figure 10 is dedicated to this case showing different variances of noise applied to the converters. Despite the high impact of this influential disturbance, the proposed robust adaptive controller is able to compensate for this concern and keep the output signal at the desired level. However, two other controllers are not working suitably in this case, making them less practical for real-time applications.

6. Experimental Implementation

The Typhoon HIL platform was employed for experimental validation owing to its FPGA-based real-time solver, which ensures high fidelity in capturing the fast switching dynamics of power converters, as well as its seamless integration with DSP-based digital controllers. In contrast to general-purpose real-time simulators, Typhoon HIL offers specialized libraries for power electronic components, making it particularly suitable for validating adaptive control strategies in converter applications. Furthermore, its safe and versatile environment enables comprehensive testing across diverse operating conditions without posing any risk of hardware damage.

The experimental system was developed using a Typhoon HIL 404 real-time simulator (Somerville, MA, USA) (Figure 11) and a Texas Instruments LaunchXL-F28379D microcontroller (Dallas, TX, USA). The system models both DC–DC Buck and Boost converters with real-time control and monitoring capabilities. The experimental setup includes all primary components of the converter, such as inductors, capacitors, switches, and control circuitry, configured to represent real-world converter dynamics. The experiments were performed under different conditions to assess the controller’s robustness. More details for this technology are listed in [12].

This configuration speeds up development, improves safety, and offers a real-time evaluation of system responsiveness and stability. The SCADA setup and HIL simulation used in this investigation are shown in Figure 12. The online platform for the converters with the suggested voltage controller is displayed in Figure 12a, and the real-time SCADA platform for the online results is displayed in Figure 12b. This real-time testing configuration is used to test various scenarios in order to assess the effectiveness of the suggested controller for the voltage regulation of the converter.

6.1. Case: Tracking Performance

To evaluate the performance of this structure in real-time situations, the RL-ABS controller tracks references of 70–100 V in Boost mode and 60–80 V in Buck mode without any disruptions (Figure 13).

The convergent behavior of the RL-ABSC controller with different reference values of both working converters is shown in Figure 13 and leads to optimal operation and minimum error. It needs to be mentioned that real-time adaptation of the controller to both types of converters exhibits high tracking and quick convergence. Another possibility is to modify the reference voltage to accommodate different loads. Using numerous step adjustments, the impact of this sudden shift is assessed in Figure 14, showcasing the controller’s outstanding performance for both architectures. With a significant response and low error, it is clear from looking at Figure 14 that the designed controller successfully reduces rapid changes in the reference signal.

6.2. Case: Supply Voltage Variation

We have then shown that this approach is robust when it comes to changing the supply voltage level. The supply voltage of the converter is comparable to a battery that is charged by solar energy or other renewable energy sources. It is feasible that the supply voltage level may fluctuate during the converter’s operation. Therefore, the tracking signal must be maintained at the same level as the reference signal in order to maintain the robustness of the controller that was developed in response to this disturbance. In order to investigate the impact of these phenomena on the systems, supply voltage fluctuations are evaluated in Figure 15. In Figure 15, the supply voltage has experienced a range of sudden variations within the range of 40–66 V for Boost, while the tracking signal is set at 70 V and a range of 70–120 V for the Buck converter with set value of 60 V. The performance of the controller depicts robust performance against these sudden alterations with a low sensitivity and no digression from the desired value. These results ensure the robustness of the controller and provide promising outcomes for real-time applications.

6.3. Case: Load Variation

There is a considerable chance of parametric deviations or changes in output load in industrial settings where converters are used since ideal conditions are not available. A variety of load types will be given to the converter in order to verify the controller’s exceptional efficacy and robust performance in real-time applications. The continuous resistive load may not always be the same as the load applied to the converter. The converter’s reactions to abrupt changes in load are shown in Figure 16. In this instance, the load value fluctuates between 5 Ω and 1 kΩ, and the tracking value is set at 70 V in the Buck mode and 80 V in the Boost mode. Figure 16 illustrates how the RL-ABSC converges to adapt and robustly compensate for variations in the converters’ loads. It demonstrates that while lower resistances cause more dynamical changes in the systems, they have a greater effect on the controller’s performance.

6.4. Case: Noise Impact

The effect of noise presents another difficult situation. These power applications are negatively impacted by a variety of disturbances, so it is essential to provide a strong framework to counteract the detrimental effects of noisy settings. Additionally, the system will be evaluated with topologies that have been altered by 1.5 and 3. In both operating modes, Figure 17 shows an amazing noise correction with significant responses.

6.5. Case: Sudden Variation

Moreover, to test the performance of the controller against sudden alterations, an impulse reference signal is applied to the system in Figure 18 to both systems. In this scenario, an impulse pulse is set as the reference signal in the range of 90 V to 100 V for the Boost converter and in the range of 90 V to 70 V for the Buck converter. This result makes it abundantly evident that the suggested controller is following these abrupt changes with remarkable robustness.

Figure 13, Figure 14, Figure 15, Figure 16, Figure 17 and Figure 18 demonstrate the advantages of using RL-ABSC for both converters, since this method is robust and adaptive structure, and it performed well under different and challenging scenarios. To better highlight the superiority of this technique, the following section compares the effective elements and criteria for various controllers.

7. Discussion

The HIL experiments demonstrated that the proposed RL-ABSC controller achieves reliable tracking and robust performance under a variety of operating conditions, including supply voltage fluctuations, abrupt load variations, noise injection, and sudden reference changes. Beyond these performance outcomes, it is crucial to discuss the stability of the DDPG agent when deployed in real-time hardware, as reinforcement learning approaches are often criticized for instability and unsafe exploration. Several mechanisms were integrated into the framework to ensure stable operation in the HIL environment. First, the reward function was shaped by incorporating a Lyapunov-based penalty term, which guided the learning process away from unstable trajectories. This ensured that the adaptation of controller gains remained bounded and consistent with the theoretical stability condition. Second, the GWO algorithm was used to pre-train the controller gains, providing a safe initialization that significantly reduced the likelihood of divergence during the early exploration phase. Third, standard stabilization techniques from the DDPG algorithm—such as target networks, replay buffer, and soft updates—were employed to avoid instability caused by correlated updates or rapid parameter changes. From an implementation perspective, the controller was executed on the TI LaunchXL-F28379D microcontroller with a fixed sampling rate and known computation delay. Duty cycle limits, anti-windup mechanisms, and output voltage/current safety envelopes were enforced to protect the converter during online training. These constraints served as an additional safeguard against instability when the agent explored control actions in real time. Importantly, discrete-time Lyapunov monitoring was performed during HIL runs, confirming that the Lyapunov function remained non-increasing across all tested scenarios. No divergence, oscillatory instability, or unsafe behavior was observed in the hardware experiments. Overall, the combination of Lyapunov-guided reward design, safe initialization via GWO, practical implementation safeguards, and HIL-based validation provides strong evidence that the DDPG-enhanced ABSC framework can maintain stable operation in real-world converter control. This addresses one of the main limitations of reinforcement learning deployment in power electronics and highlights the feasibility of integrating intelligent adaptive controllers into practical hardware platforms.

To validate Lyapunov stability in hardware, the candidate function defined in Equations (25)–(27) was implemented on the DSP using real-time measurements of voltage error and adaptive terms. At each sampling step, the function value $V\left[k\right]$ was computed, and its discrete difference $\mathrm{\Delta }V\left[k\right]$ was monitored. Stability was confirmed, as $\mathrm{\Delta }V\left[k\right]$ remained non-positive and the function trajectory decayed towards a bounded value across all tested scenarios. This approach ensures that the theoretical Lyapunov guarantees are directly observable in the HIL environment.

Although Table 4 provides a quantitative comparison of GWO with PSO, GA, and ALO in simulation, reproducing the same comparison in HIL was not feasible. Running multiple optimization algorithms in hardware would require separate converter setups or repeated firmware reconfiguration, which introduces safety risks and significant resource constraints. For this reason, we limited the HIL validation to the GWO-initialized controller, which was already shown in simulation to offer superior performance. Future work may investigate multi-algorithm HIL benchmarking frameworks to further expand this comparison.

8. Conclusions

This paper has developed a reliable adaptive BSC method for Boost and Buck converters. Conventional approaches lack the necessary efficiency to safeguard power converters from harmful disruptions, necessitating the need for more sophisticated and robust controllers to ensure reliable functioning in a variety of real-world scenarios. Therefore, an adaptive Backstepping method has been designed based on Lyapunov stability theory to both ensure the robustness of the converters in different working conditions and enhance their performance under disturbances. However, there are a number of scenarios and parametric unknowns that could cause disruptions to this approach. To solve this problem, we used the DPG algorithm in conjunction with a hardware configuration and Reinforcement Learning (RL) scheme to regulate the Buck converter’s output voltage. In particular, DDPG combines the advantages of the deterministic policy gradient method, deep Q-networks, and actor–critic networks. Additionally, a new and well-behaved metaheuristic method known as the GWO algorithm was used to increase the controller’s initial gain. We also intended to compare the RL-ABS controller with traditional ABSC and RL-PID controllers to demonstrate the superiority of this work. In the final analysis, the RL-ABSC technique outperformed conventional techniques thanks to its robust dynamics in all working modes. To improve the dynamical performance of the RL technique, the GWO algorithm was adopted to set the most suitable initial parameters for the RL block. This controller demonstrated faster dynamics and significantly better convergence. In addition, the results provided in a real-time platform using an HIL device ensure the significant robustness of the proposed controller and its well-behaved dynamics in disturbance-rejection modes. This work can further be improved using identification processes to better adapt the ABSC method to the undesired working environments. Future works will also explore the extension of the proposed RL-ABSC framework to multiphase and multi-converter systems, where the effective system order must first be assigned according to the topology before controller design.