Bayesian Optimization for the Synthesis of Generalized State-Feedback Controllers in Underactuated Systems

Abstract

Underactuated systems, such as rotary and double inverted pendulums, challenge traditional control due to nonlinear dynamics and limited actuation. Classical methods like state-feedback and Linear Quadratic Regulators (LQRs) are commonly used but often require high gains, leading to excessive control effort, poor energy efficiency, and reduced robustness. This article proposes a generalized state-feedback controller with its own internal dynamics, offering greater design flexibility. To automate tuning and avoid manual calibration, we apply Bayesian Optimization (BO), a data-efficient strategy for optimizing closed-loop performance. The proposed method is evaluated on two benchmark underactuated systems, including one in simulation and one in a physical setup. Compared with standard LQR designs, the BO-tuned state-feedback controller achieves a reduction of approximately 20% in control signal amplitude while maintaining comparable settling times. These results highlight the advantages of combining model-based control with automatic hyperparameter optimization, achieving efficient regulation of underactuated systems without increasing design complexity.

1. Introduction

Underactuated systems pose a challenging problem, as their corresponding controllers must generate control signals that drive a given state to zero without necessarily being directly linked to an actuator, while also being capable of rejecting disturbances and handling model uncertainties. These systems are attractive due to their wide range of applications in areas such as aerospace and robotics [1,2,3].

State-feedback controllers are often preferred for their simple structure, typically consisting of a feedback gain. However, the dynamics of the plant can sometimes result in a static feedback controller requiring higher gains than desired, leading to increased control costs such as higher power consumption. In this work, we demonstrate the effectiveness of applying a generalized state-feedback structure to underactuated control systems. This approach can achieve similar performance to a traditional state-feedback controller while reducing control cost by introducing additional parameters into the model.

One of the most widely used benchmark experiments in the field of underactuated mechanical systems is the rotary inverted pendulum, which has a nonlinear model that can be approximated as linear around its unstable upright position. This plant consists of a controlled arm rotating around a central axis and a pendulum attached to one end of the arm. It enables the study of nonlinear dynamics in simplified models that can also be physically constructed, offering improved reproducibility.

The Linear Quadratic Regulator (LQR) is a well-known problem in linear control theory, where the system is assumed to be linear and the performance index is defined by a quadratic function [4]. The solution to the LQR problem is obtained by solving an Algebraic Riccati Equation. The Linear Quadratic Tracking (LQT) problem also assumes linear process dynamics and a quadratic cost function, but its objective is to design a controller that tracks an exogenous reference signal, making LQR a special case of LQT.

State-feedback controllers are favored not only for their simplicity but also for their low computational cost in generating control signals. The generalized state-feedback controller extends the classical structure by introducing an additional degree of freedom, though at the expense of extra parameters that must be tuned.

We address the controller design problem through automatic tuning using Bayesian Optimization, a global optimization method suitable for multi-modal, computationally expensive black-box functions. This contrasts with gradient-based techniques, which are more prone to getting stuck in local optima. Our work builds on the analysis in [5], where the generalized state-feedback controller was tuned using reinforcement learning and applied solely to the rotary inverted pendulum. In this paper, we extend that approach to include analysis of the double inverted pendulum as well.

The main contribution of this work is the integration of a generalized state-feedback controller structure with Bayesian Optimization to address the control of underactuated systems. Unlike classical LQR methods, the proposed controller includes internal dynamic states and additional degrees of freedom, allowing enhanced flexibility and energy efficiency. Bayesian Optimization is employed as a data-efficient strategy to automatically tune the controller parameters based on spectral radius minimization, avoiding manual calibration. This hybrid approach combines the theoretical rigor of model-based control with the adaptability of learning-based optimization. The method is validated on two benchmark systems—the physical rotary inverted pendulum and a simulated double inverted pendulum—demonstrating improved performance in terms of control effort and error reduction compared with traditional techniques.

The remainder of this article is structured as follows: Section 2 reviews related work on state-feedback controllers and their applications to underactuated systems. Section 3 presents the rotary inverted pendulum with its linear model obtained from a phenomenological analysis, the corresponding linearization, and the model for the double inverted pendulum. It concludes with the formulation of the Linear Quadratic Regulator problem and classical state-feedback control. Section 5 describes the Bayesian Optimization algorithm applied to tune the parameters of the generalized state-feedback controller. The results are reported in Section 6, followed by final remarks and conclusions in Section 7.

2. Literature Review

2.1. Underactuated Systems

The control community has extensively studied underactuated systems and introduced benchmark problems, such as the cart-pole system, where control requires not only stabilizing the pendulum in an unstable position but also managing the displacement of the cart, making the control problem more complex [6]. Other underactuated mechanical systems with simplified models also exist, allowing researchers to capture the essence of the problem without the complexity found in real-world applications—such as the rotary inverted pendulum (RIP), also known as the Furuta pendulum [7].

Various control techniques have been applied to underactuated systems. For example, Cui et al. [8] proposed a Fast Particle Swarm Optimization (Fast-PSO) algorithm, which improves the convergence speed and robustness of the original PSO through adaptive velocity control. Although not originally applied to the rotary inverted pendulum, this algorithm represents a class of optimization strategies that have inspired controller tuning methods for nonlinear and underactuated systems.

Similarly, Kong et al. [9] proposed an asymmetric bounded neural control method for uncertain underactuated systems, using both state and output feedback. Their approach addresses robustness and adaptability in the presence of model uncertainties. The controller’s performance is validated through simulations and compared with conventional methods, demonstrating improved stability and tracking performance under significant uncertainty.

Artificial neural networks can also enhance other learning approaches, such as reinforcement learning algorithms [10], which have been widely used to solve the Linear Quadratic Regulator (LQR) problem. In this case, the system is assumed to be linear, and the performance index is expressed as a quadratic function [4]. The solution is obtained by solving an Algebraic Riccati Equation (ARE). The Linear Quadratic Tracking (LQT) problem similarly assumes a linear process model and a quadratic cost function, but its objective is to design a controller such that the measured output follows an exogenous reference signal. The LQR can be considered a special case of the LQT problem.

However, the LQT problem has received limited attention in the literature, mainly because, for most reference signals, the infinite-horizon cost becomes unbounded [11]. Qin and Zhang tackled this issue in the continuous-time domain by solving an augmented ARE [12], while Kiumarsi et al. adopted a similar approach for the discrete-time case using a Q-learning algorithm [13,14]. Their results show that the proposed method achieves good stability and trajectory tracking performance even in the presence of unknown dynamics.

Although these approaches are innovative and do not require full model knowledge, they often involve high computational effort. The generalized state-feedback controllers proposed in this work include three additional design parameters, yet they yield lower-amplitude control signals and are expected to enhance convergence and system stationarity.

In contrast with model-free schemes, state-feedback controllers allow for systematic design using known dynamics and can provide closed-loop stability guarantees. However, their performance is highly dependent on the quality of gain selection. This work addresses that limitation by incorporating a generalized state-feedback structure with automated tuning to retain stability guarantees while improving adaptability.

Recent approaches have explored model-free and hybrid control schemes such as intelligent PID (iPID), sigmoid-based PID controllers, neuroendocrine PID, and brain-emotional-learning-based intelligent controllers (BELBICs) [15,16,17]. These methods aim to improve tracking and robustness in nonlinear systems without relying on full model knowledge. While effective in some contexts, they typically require extensive tuning or simulation-based calibration and may not offer formal guarantees on stability or optimality.

2.2. Bayesian Optimization

The Bayesian Optimization (BO) method [18] constructs a representation of the unknown objective function using a surrogate model and then optimizes it via iterative sampling, guided by an acquisition function defined based on the surrogate model [19,20,21]. BO has been extensively studied for the optimization of unknown functions across various domains. Its success relies heavily on the choice of acquisition function to identify the most promising regions of the search space. To prevent unnecessary evaluations, Nguyen et al. [22] proposed convergence criteria for acquisition functions.

BO has been successfully applied to a wide range of applications, including machine learning [23], analog circuit design [24], voltage fault diagnosis [25], aerospace engineering [26], pharmaceutical research [27], gas–liquid separation in laboratories [28], multi-objective optimization [29], free-electron lasers [30], and autonomous systems.

Alternative optimization methods—such as improved sparrow search algorithms, particle swarm variants, and neuro-fuzzy tuning—have been applied to controller design in complex systems [31,32]. While powerful, these heuristics often lack convergence guarantees and require extensive trial runs. Bayesian Optimization, by contrast, is particularly suited for tuning in low-sample, expensive-to-evaluate contexts such as control design, offering principled convergence and efficient exploration–exploitation trade-offs.

In this work, Bayesian Optimization (BO) is employed to tune the gain matrices of a state-feedback controller such that the resulting closed-loop system places its eigenvalues near a set of desired locations. The optimization problem is formulated as the minimization of a scalar cost function, which quantifies the deviation between the current and target eigenvalues, reflecting the dynamic performance of the system. The BO algorithm models this objective via a Gaussian Process (GP), which provides a probabilistic estimate of the function and its uncertainty across the parameter space. At each iteration, candidate gain parameters are selected by optimizing an acquisition function—specifically, the Expected Improvement (EI)—which guides the search by trading off between exploration (uncertain areas) and exploitation (promising low-cost areas).

The Expected Improvement at a candidate point $\mathbf{x}$ is defined as

$$\mathrm{EI}(\mathbf{x})=\mathbb{E}\left[max(f_{min}-f(\mathbf{x}),0)\right],$$

(1)

where $f_{min}$ is the best (i.e., minimum) observed value of the objective function so far.

Given the GP posterior mean $\mu (\mathbf{x})$ and standard deviation $\sigma (\mathbf{x})$, the EI can be expressed in closed form as

$$\mathrm{EI}(\mathbf{x})=(f_{min}-\mu (\mathbf{x}))\mathrm{\Phi }\left({\displaystyle \frac{f_{min}-\mu (\mathbf{x})}{\sigma (\mathbf{x})}}\right)+\sigma (\mathbf{x})\varphi \left({\displaystyle \frac{f_{min}-\mu (\mathbf{x})}{\sigma (\mathbf{x})}}\right),$$

(2)

where $\mathrm{\Phi }(·)$ and $\varphi (·)$ denote the cumulative distribution function and probability density function of the standard normal distribution, respectively. This formulation encourages sampling in regions that either offer potentially better solutions (exploitation) or where the model is uncertain (exploration).

At each iteration, the selected parameters are used to construct the closed-loop system, compute its eigenvalues, and evaluate the cost function. These data are used to update the surrogate model, thereby refining the search landscape.

The procedure continues until a stopping criterion is satisfied. In this work, the algorithm terminates after a fixed number of iterations, although, in general, stopping criteria may also include convergence of the objective value, stabilization of the acquisition function, or practical constraints such as computational budget or runtime limits [33,34].

3. Theoretical Framework

3.1. Underactuated Systems

The control community has extensively studied underactuated systems and introduced benchmark problems, such as the cart-pole system, where control requires not only stabilizing the pendulum in its unstable position but also managing the displacement of the cart—making the control problem more complex [6]. Other underactuated mechanical systems with simplified models also exist, allowing researchers to capture the essence of the problem without the complexity of real-world applications—such as the rotary inverted pendulum (RIP), also known as the Furuta pendulum [7].

Consider a second-order, controllable dynamical system given by

$$\ddot{\mathbf{q}}=f(\mathbf{q},\dot{\mathbf{q}},\mathbf{u},t),$$

(3)

where $\mathbf{u}$ is the control input vector; $\mathbf{q}$ and $\dot{\mathbf{q}}$ denote the position and velocity vectors, respectively; and t represents the possible time dependence of the acceleration vector $\ddot{\mathbf{q}}$. In the case considered in this study—where the dynamics are affine in the commanded torque—the arbitrary function f in Equation (3) can be rewritten using the two functions $f_1$ and $f_2$ as

$$\ddot{\mathbf{q}}=f_1(\mathbf{q},\dot{\mathbf{q}},t)+f_2(\mathbf{q},\dot{\mathbf{q}},t)\mathbf{u}.$$

(4)

A system described in Equation (3) is formally defined as underactuated in the configuration $(\mathbf{q},\dot{\mathbf{q}},t)$ if it is not possible to generate instantaneous acceleration in an arbitrary direction, i.e.,

$$rank(f_2(\mathbf{q},\dot{\mathbf{q}},t))<dim(\mathbf{q}),$$

(5)

where $dim(M)$ denotes the dimension of matrix M, and $rank(M)$ refers to the maximum number of linearly independent columns in M.

3.2. Rotary Inverted Pendulum

The rotary inverted pendulum (RIP), also referred to as the Furuta pendulum, is a classic example of an underactuated system, as shown in Figure 1. The system consists of an arm in the horizontal plane that rotates around a central axis and a pendulum attached to one end of the arm, rotating in the vertical plane. This particular setup offers a platform for investigating the nonlinear dynamics of simplified models that are feasible to construct physically, resulting in mechatronic systems with improved reproducibility.

As shown in Figure 1, a DC motor controls the arm position, measured by ${\theta }_0$, through the armature voltage. Its main parameters are the armature resistance and inductance, denoted by $R_a$ and $L_a$, respectively.

3.3. Model Formulation for the Rotary Inverted Pendulum

We define the parameters $\{l_p,m_p,I_p,{\theta }_1\}$, where $l_p$ is the distance from the pendulum’s rotation axis to its center of mass, $m_p$ is the mass of the pendulum, $I_p$ is its moment of inertia, and ${\theta }_1$ is its angular position. The arm has radius r and moment of inertia $I_a$, and a counterweight of mass $m_c$ brings the center of mass of the rotating arm to a height h. The rotation angle of the arm is denoted by ${\theta }_0$. A detailed description of all parameters and their values for the physical prototype is shown in

Table 1. RIP parameters.

Symbol	Description	Value
$m_p$	Pendulum mass	$0.1$ kg
$m_c$	Counterweight mass	$0.01$ kg
$I_p$	Pendulum inertial moment	$5.1\times {10}^{-4}$ kg ${\mathrm{m}}^2$
$I_a$	Arm inertial moment	$3.1\times {10}^{-3}$ kg ${\mathrm{m}}^2$
r	Arm radius	$0.13$ m
$l_p$	Pendulum mass center	$0.125$ m
h	Arm center of mass height	$0.055$ m
$C_0$	Arm friction coefficient	${10}^{-4}$
$C_1$	Pendulum friction coefficient	${10}^{-4}$
$R_a$	Armature resistor	8 $\mathrm{\Omega }$
$L_a$	Motor inductance	10 mH
$I_m$	Motor inertia	$1.9\times {10}^{-6}$ kg ${\mathrm{m}}^2$
$M_f$	Motor mutual inductance	$0.0214$ N m/A
$K_g$	Gear reduction coefficient	59,927
$K_{eg}$	External gear reduction coefficient	16
g	Gravitational acceleration	$9.806$ m/${\mathrm{s}}^2$

, derived from a phenomenological analysis consistent with the actual system [35].

The Lagrangian $L(\theta ,\dot{\theta })$ is given by

$$L(\mathbf{q},\dot{\mathbf{q}})=E_k(\mathbf{q},\dot{\mathbf{q}})-E_p(\mathbf{q},\dot{\mathbf{q}}),$$

(6)

where $E_k$ and $E_p$ denote the kinetic and potential energies, respectively, and $\mathit{\theta }$ is the generalized coordinates vector, $\mathit{\theta }={\left[\begin{array}{cc}{\theta }_0& {\theta }_1\end{array}\right]}^{⊺}$.

The kinetic energy of the pendulum includes both translational and rotational components, while the arm’s kinetic energy includes rotational and tangential terms, as follows:

$$E_k={\displaystyle \frac{1}{2}}{\widehat{J}}_0{\dot{\theta }}_0^2+{\displaystyle \frac{1}{2}}{\widehat{J}}_1{\dot{\theta }}_1^2+{\displaystyle \frac{1}{2}}{m}_p{l}_p^2{\dot{\theta }}_0^2{sin}^2({\theta }_1)-m_{p}r{l}_p{\dot{\theta }}_0{\dot{\theta }}_{1}cos({\theta }_1),$$

(7)

where ${\dot{\theta }}_0$ and ${\dot{\theta }}_1$ are the angular velocities of the arm and pendulum, respectively.

The equivalent angular masses of the arm and pendulum, ${\widehat{J}}_0$ and ${\widehat{J}}_1$, are given by

$${\widehat{J}}_0=I_a+r^2(m_p+m_c),$$

(8)

$${\widehat{J}}_1=I_p+m_p{l}_p^2,$$

(9)

while the potential energy is defined by the pendulum and counterweight masses, as follows:

$$E_p=m_{p}g{l}_{p}cos({\theta }_1)+m_{c}gh.$$

(10)

Then, according to the Euler–Lagrange equation,

$${\displaystyle \frac{\partial }{\partial t}}\left({\displaystyle \frac{\partial L}{\partial {\dot{q}}_i}}\right)-{\displaystyle \frac{\partial L}{\partial q_i}}={\tau }_i,i={\theta }_0,{\theta }_1$$

(11)

where ${\tau }_i$ represents the torque applied to each coordinate, leading to

$$M_1({\theta }_1)\ddot{q}+M_2({\theta }_1)\dot{q}+M_3({\theta }_1)=T,$$

(12)

where $M_1$, $M_2$, and $M_3$ correspond to the inertia, centripetal, and Coriolis torque matrices, respectively, and T is the torque vector, as follows:

$$\begin{array}{cc}\hfill M_1({\theta }_1)& =\left[\begin{array}{cc}{\widehat{J}}_0+m_p{l}_p^2{sin}^2({\theta }_1)& -m_{p}r{l}_{p}cos({\theta }_1)\\ -m_{p}r{l}_{p}cos({\theta }_1)& {\widehat{J}}_1\end{array}\right],\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill M_2({\theta }_1)& =\left[\begin{array}{cc}{m}_p{l}_p^2{\dot{\theta }}_{1}sin(2{\theta }_1)+C_0& m_{p}r{l}_p{\dot{\theta }}_{1}sin({\theta }_1)\\ -{\displaystyle \frac{1}{2}}{m}_p{l}_p^2{\dot{\theta }}_{0}sin(2{\theta }_1)& C_1\end{array}\right],\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill M_3({\theta }_1)& =\left[\begin{array}{c}0\\ -m_{p}g{l}_{p}sin({\theta }_1)\end{array}\right],\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill T& =\left[\begin{array}{c}{\tau }_l\\ 0\end{array}\right],\hfill \end{array}$$

(16)

where $C_0$ and $C_1$ are the friction coefficients of the arm and pendulum, respectively, with values given in Table 1. From Equation (13), we observe that the motor acts only on the arm, and the matrices model the motion transfer to the pendulum. The load torque ${\tau }_l$ is related to the motor dynamics, which are themselves associated with the electrical torque ${\tau }_e$ and the electrical angular velocity ${\omega }_e$, given by

$$\begin{array}{cc}\hfill V(t)& =R_{a}i(t)+L_a{\displaystyle \frac{d}{dt}}i(t)+M_f{\omega }_e,\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill {\tau }_e& =M_f{K}_{g}i(t),\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill {\tau }_e-{\tau }_l& =I_m{\displaystyle \frac{d}{dt}}{\omega }_e,\hfill \end{array}$$

(19)

where all parameters are described in Table 1, and $V(t)$ and $i(t)$ are the voltage and current applied to the motor at time t, respectively.

Finally, the applied torque and angular velocity are related to their corresponding electrical variables as follows:

$$\omega =K_g^{-1}{\omega }_e,$$

(20)

$$\tau =K_g{\tau }_e.$$

(21)

Then, the RIP model including motor dynamics is expressed as

$$\ddot{q}={\overline{M}}_1{({\theta }_1)}^{-1}\left(\overline{T}-{\overline{M}}_2({\theta }_1)\dot{q}-M_3({\theta }_1)\right),$$

(22)

where the updated matrices are defined as

$${\overline{M}}_1({\theta }_1)=\left[\begin{array}{cc}{\widehat{J}}_0+m_p{l}_p^2{sin}^2({\theta }_1)+K_g^2{I}_m& -m_{p}r{l}_{p}cos({\theta }_1)\\ -m_{p}r{l}_{p}cos({\theta }_1)& {\widehat{J}}_1\end{array}\right],$$

(23)

$${\overline{M}}_2({\theta }_1)=\left[\begin{array}{cc}{m}_p{l}_p{\dot{\theta }}_{1}sin(2{\theta }_1)+C_0& m_{p}r{l}_p{\dot{\theta }}_{1}sin({\theta }_1)\\ -{\displaystyle \frac{1}{2}}{m}_p{l}_p{\dot{\theta }}_{0}sin(2{\theta }_1)& C_1\end{array}\right],$$

(24)

$$\begin{array}{cc}\hfill \overline{T}& =\left[\begin{array}{c}{M}_f{K}_{g}i\\ 0\end{array}\right].\hfill \end{array}$$

(25)

Define the state vector $\mathbf{x}={\left[\begin{array}{ccccc}{\theta }_0& {\dot{\theta }}_0& {\theta }_1& {\dot{\theta }}_1& i\end{array}\right]}^{⊺}$. To obtain a linearized model around the equilibrium point $x=\left[\begin{array}{ccccc}0& 0& 0& 0& 0\end{array}\right]$, recall Equation (17) and solve for the derivative of the current i, as follows:

$${\displaystyle \frac{d}{dt}}i(t)={\displaystyle \frac{1}{L_a}}V(t)-{\displaystyle \frac{R_a}{L_a}}i(t)-{\displaystyle \frac{M_f}{L_a}}{\omega }_e.$$

(26)

From Equation (20), and noting that the prototype includes an external gear reduction system that relates the arm’s angular velocity to the corresponding motor variable, we write

$${\dot{\theta }}_0=K_{eg}^{-1}\omega ,$$

(27)

which allows rewriting Equation (26) as:

$${\displaystyle \frac{d}{dt}}i(t)={\displaystyle \frac{1}{L_a}}V(t)-{\displaystyle \frac{R_a}{L_a}}i(t)-{\displaystyle \frac{M_f}{L_a{K}_g}}{\dot{\theta }}_0.$$

(28)

To derive the following linear model, we linearize the nonlinear equations around the upright equilibrium point ${\theta }_1=0$, ${\dot{\theta }}_1=0$, ${\theta }_0=0$. Small-angle approximations such as $sin({\theta }_1)\approx {\theta }_1$ and $cos({\theta }_1)\approx 1$ are applied. Friction and motor dynamics are retained to preserve physical accuracy in the linearized model. Then, the following linear model is obtained:

$$\dot{x}(t)=Ax(t)+Bu(t)+v(t),$$

(29)

$$y(t)=Cx(t)+Du(t)+w(t),$$

(30)

where x is the previously defined state vector, and u corresponds to the control input (applied voltage). In this formulation, $v(t)\sim \mathcal{N}(0,P_v)$ and $w(t)\sim \mathcal{N}(0,P_w)$ are assumed to be uncorrelated, zero-mean Gaussian white noise processes with constant covariance matrices $P_v$ and $P_w$, respectively.

Matrices A and B are given in Equation (32), with $\alpha$ defined as

$$\alpha ={\displaystyle \frac{1}{{\widehat{J}}_0{\widehat{J}}_1-{(m_{p}r{l}_p)}^2}}.$$

(31)

$$\begin{array}{cc}\hfill A& =\alpha ·\left[\begin{array}{ccccc}0& {\displaystyle \frac{1}{\alpha }}& 0& 0& 0\\ \\ 0& -{\widehat{J}}_1{C}_0& m_p^2{l}_p^{2}gr& m_{p}r{l}_p{C}_1& M_f{K}_g{\widehat{J}}_0{\widehat{J}}_1\\ \\ 0& 0& 0& {\displaystyle \frac{1}{\alpha }}& 0\\ \\ 0& m_{p}r{l}_p{C}_0& {\widehat{J}}_0{m}_{p}g{l}_p& -{\widehat{J}}_0{C}_1& K_g{M}_f{m}_{p}r{l}_p\\ \\ 0& -{\displaystyle \frac{M_f}{L_a{K}_g\alpha }}& 0& 0& -{\displaystyle \frac{R_a}{L_a\alpha }}\end{array}\right],\hfill \end{array}$$

(32)

$$\begin{array}{cc}\hfill B& ={\left[\begin{array}{ccccc}0& 0& 0& 0& {\displaystyle \frac{1}{L_a}}\end{array}\right]}^{⊺},\hfill \end{array}$$

(33)

Finally, noting that the pendulum angles ${\theta }_0$ and ${\theta }_1$, as well as the motor current i, are the measurable outputs, matrices C and D in Equation (30) become

$$\begin{array}{cc}\hfill C& =\left[\begin{array}{ccccc}1& 0& 0& 0& 0\\ 0& 0& 1& 0& 0\\ 0& 0& 0& 0& 1\end{array}\right],\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill D& =0.\hfill \end{array}$$

(35)

Matrices $(A,B,C,D)$ represent the state-space variables [36], with dimensions $A\in {\mathbb{R}}^{n_x\times n_x}$, $B\in {\mathbb{R}}^{n_x\times n_u}$, and $C\in {\mathbb{R}}^{n_y\times n_x}$, where $n_x$, $n_u$, and $n_y$ denote the number of states, control inputs, and outputs, respectively.

3.4. Model Formulation for Double Inverted Pendulum

As another example of an underactuated system, consider the double inverted pendulum shown in Figure 2, which is formed by attaching one pendulum directly to another without an actuator between the two links. Each pendulum consists of a mass connected to a massless rigid rod that is constrained to move only within a vertical plane, with the pivot of the first pendulum fixed at point $\mathbf{O}$.

The fixed point $\mathbf{O}$ is taken as the origin of the corresponding Cartesian coordinate system, with the x-axis pointing horizontally and the y-axis pointing vertically upward. Let ${\theta }_1$ and ${\theta }_2$ denote the angles formed by the first and second rods, respectively, with the vertical axis. Then, the positions of the masses are given by

$$\begin{array}{cc}\hfill x_1& =l_{1}sin({\theta }_1),\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill x_2& =l_{1}sin({\theta }_1)+l_{2}sin({\theta }_2),\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill y_1& =-l_{1}cos({\theta }_1),\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill y_2& =-l_{1}cos({\theta }_1)-l_{2}cos({\theta }_2).\hfill \end{array}$$

(39)

Differentiating the expressions above yields the horizontal and vertical components of the velocities of the corresponding masses, i.e., ${\dot{x}}_1$ and ${\dot{y}}_1$ for the first-link velocity $v_1$ and ${\dot{x}}_2$ and ${\dot{y}}_2$ for the second-link velocity $v_2$, as follows:

$$\begin{array}{cc}\hfill {\dot{x}}_1& =l_1{\dot{\theta }}_{1}cos({\theta }_1),\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill {\dot{x}}_2& =l_1{\dot{\theta }}_{1}cos({\theta }_1)+l_2{\dot{\theta }}_{2}cos({\theta }_2),\hfill \end{array}$$

(41)

$$\begin{array}{cc}\hfill {\dot{y}}_1& =l_1{\dot{\theta }}_{1}sin({\theta }_1),\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill {\dot{y}}_2& =l_1{\dot{\theta }}_{1}sin({\theta }_1)+l_2{\dot{\theta }}_{2}sin({\theta }_2).\hfill \end{array}$$

(43)

As with the rotary inverted pendulum, the Lagrangian for the double inverted pendulum is defined by Equation (6), with the difference lying in the expressions for kinetic and potential energy of the first and second links, corresponding to masses $m_1$ and $m_2$, respectively, as follows:

$$\begin{array}{cc}\hfill E_k& ={\displaystyle \frac{1}{2}}{m}_1{v}_1^2+{\displaystyle \frac{1}{2}}{m}_2{v}_2^2\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}{m}_1\left({\dot{x}}_1^2+{\dot{y}}_1^2\right)+{\displaystyle \frac{1}{2}}{m}_2\left({\dot{x}}_2^2+{\dot{y}}_2^2\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{2}}{m}_1{l}_1^2{\dot{\theta }}_1^2+{\displaystyle \frac{1}{2}}{m}_2\left(l_1^2{\dot{\theta }}_1^2+l_2^2{\dot{\theta }}_2^2+2l_1{l}_2{\dot{\theta }}_1{\dot{\theta }}_{2}cos({\theta }_1-{\theta }_2)\right)\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill E_p& =m_{1}g{y}_1+m_{2}g{y}_2\hfill \\ \hfill & =-m_{1}g{l}_{1}cos({\theta }_1)-m_{2}g\left(l_{1}cos({\theta }_1)+l_{2}cos({\theta }_2)\right)\hfill \\ \hfill & =-\left(m_1+m_2\right)g{l}_{1}cos({\theta }_1)-m_{2}g{l}_{2}cos({\theta }_2)\hfill \end{array}$$

(45)

Then, the Lagrangian of the system is given by

$$\begin{array}{cc}\hfill L=& {\displaystyle \frac{1}{2}}\left(m_1+m_2\right)l_1^2{\dot{\theta }}_1^2+{\displaystyle \frac{1}{2}}{m}_2{l}_2^2{\dot{\theta }}_2^2+\hfill \\ \hfill & m_2{l}_1{l}_2{\dot{\theta }}_1{\dot{\theta }}_{2}cos\left({\theta }_1-{\theta }_2\right)+\hfill \\ \hfill & \left(m_1+m_2\right)g{l}_{1}cos({\theta }_1)+m_{2}g{l}_{2}cos({\theta }_2)\hfill \end{array}$$

(46)

Then, according to the Euler–Lagrange equation,

$${\displaystyle \frac{\partial }{\partial t}}\left({\displaystyle \frac{\partial L}{\partial {\dot{q}}_i}}\right)-{\displaystyle \frac{\partial L}{\partial q_i}}={\tau }_i,i={\theta }_1,{\theta }_2$$

(47)

we obtain

$$\begin{array}{c}\left(m_1+m_2\right)l_1{\ddot{\theta }}_1+m_2{l}_2{\ddot{\theta }}_{2}cos\left({\theta }_1-{\theta }_2\right)+\\ m_2{l}_2{\dot{\theta }}_{2}sin\left({\theta }_1-{\theta }_2\right)+\left(m_1+m_2\right)gsin({\theta }_1)={\tau }_1,\end{array}$$

(48)

$$\begin{array}{cc}\hfill l_2{\ddot{\theta }}_2+l_1{\ddot{\theta }}_{1}cos\left({\theta }_1-{\theta }_2\right)-l_1{\dot{\theta }}_1^{2}sin\left({\theta }_1-{\theta }_2\right)+gsin({\theta }_2)& ={\tau }_2,\hfill \end{array}$$

(49)

Evaluating above expressions on equilibrium point where ${\theta }_1={\theta }_2\sim 0$, ${\dot{\theta }}_1={\dot{\theta }}_2\sim 0$, and $sin({\theta }_i)\sim {\theta }_i$ for $i=1,2$,

$$\begin{array}{cc}\hfill \left(m_1+m_2\right)l_1{\ddot{\theta }}_1+m_2{l}_2{\ddot{\theta }}_2+\left(m_1+m_2\right)g{\theta }_1& ={\tau }_1,\hfill \end{array}$$

(50)

$$\begin{array}{cc}\hfill l_2{\ddot{\theta }}_2+l_1{\ddot{\theta }}_1+g{\theta }_2& ={\tau }_2,\hfill \end{array}$$

(51)

where ${\tau }_1$ and ${\tau }_2$ stand for the torque commanded on the first and second link, respectively.

To obtain the following linear state-space model, the system is linearized around the upright equilibrium point ${\theta }_1=0$, ${\theta }_2=0$, with zero velocities, since it is desired to stabilize the pendulum in the upright position and to remain fixed on that position. Standard small-angle approximations are applied: $sin({\theta }_i)\approx {\theta }_i$ and $cos({\theta }_i)\approx 1$ for $i=1,2$. These assumptions are valid for small perturbations near the vertical position and are commonly used in control-oriented modeling.

Then, rewriting the above expressions in matrix form and replacing plant parameters such as in [37], with $m_1$ = 0.4 kgs, $m_2$ = 0.5 kgs, $l_1$ = $l_2$ = 5 mts, and g = 9.8 mts/${\sec }^2$, we obtain the following state-space representation, with state vector defined as $x(t)={\left[\begin{array}{cccc}{\theta }_1(t)& {\theta }_2(t)& {\dot{\theta }}_1(t)& {\dot{\theta }}_2(t)\end{array}\right]}^{⊺}$, as follows:

$$\begin{array}{cc}\hfill \dot{x}(t)& =Ax(t)+Bu(t),\hfill \end{array}$$

(52)

$$\begin{array}{cc}\hfill y(t)& =Cx(t)+Du(t),\hfill \end{array}$$

(53)

with

$$\begin{array}{cc}\hfill A& =\left[\begin{array}{cccc}0& 0& 1& 0\\ 0& 0& 0& 1\\ -39.69& -24.5& 0& 0\\ -44.1& -49& 0& 0\end{array}\right],\hfill \end{array}$$

(54)

$$\begin{array}{cc}\hfill B& =\left[\begin{array}{cc}0& 0\\ 0& 0\\ 4.5& 2.5\\ 5& 5\end{array}\right],\hfill \end{array}$$

(55)

$$\begin{array}{cc}\hfill C& =\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right],\hfill \end{array}$$

(56)

$$\begin{array}{cc}\hfill D& =0.\hfill \end{array}$$

(57)

The derivation of the double inverted pendulum model in this work is based on standard formulations, following the reference in [37]. This model is used solely for simulation-based validation and is not experimentally implemented in a physical setup.

4. Controller Synthesis

4.1. Linear Quadratic Regulator

Consider a system described in state-space form with the linear model given by Equations (29) and (30). The Linear Quadratic Regulator (LQR) [4] consists of a state-feedback controller, as illustrated in Figure 3, which defines a cost function to be minimized, as follows:

$$J={\int }_0^{\infty }\left(x^{⊺}(t)Qx(t)+u^{⊺}(t)Ru(t)\right)dt,$$

(58)

where Q and R are positive semi-definite and positive definite weighting matrices, respectively. The state-feedback control law that minimizes the cost function in Equation (58) is given by

$$u(t)=-Kx(t),$$

(59)

where K is defined as

$$K=R^{-1}{B}^{⊺}P,$$

(60)

and P is the unique positive semi-definite solution to the Algebraic Riccati Equation (ARE), as follows:

$$A^{⊺}P+PA-PB{R}^{-1}{B}^{⊺}P+Q=0.$$

(61)

4.2. State-Feedback Controller

The plant G is assumed to follow the form described in Equations (32) and (34). A classical state-feedback control scheme, shown in Figure 3, corresponding to the Linear Quadratic Regulator defined in Equation (59), is considered. The gain matrix K multiplies the plant state $x(t)$ to generate the control signal $u(t)$ for tracking the reference $r(t)$ at time t. In a state-feedback control scheme such as the one in Figure 3, $r(t)$ is typically a pre-filtered reference derived from a desired reference $\overline{r}(t)\in {\mathbb{R}}^{n_y}$, which represents the target output value $y(t)$ at time t, and is mapped into a vector of appropriate dimensions.

A stabilizing controller (with $K\in {\mathbb{R}}^{n_u\times n_x}$) must be designed such that the eigenvalues of $A-BK$ lie in the left half-plane in the continuous-time case, or within the unit circle in the discrete-time case [4]. Moreover, to achieve reference tracking—i.e., to ensure that $y(t)$ follows $\overline{r}(t)$—a pre-filter F (a static gain) is introduced as

$$F=-{\left(C{\left(A-BK-I\right)}^{-1}B\right)}^{-1},$$

(62)

so that the transfer function from $\overline{r}(t)$ to $y(t)$ is unity.

We now consider a state-feedback control scheme in which the controller has its own internal dynamics, as shown in Figure 4.

Here, the reference vector $r(t)$ is assumed to be a pre-filtered version of $\overline{r}(t)$, resulting in an $n_u$-dimensional vector representing the desired output components within $\overline{r}(t)$. The controller, labeled $Co$ in Figure 4, is assumed to be linear and is described by the following state-space model:

$$\begin{array}{cc}\hfill {\dot{x}}_c(t)& =A_c{x}_c(t)+B_{c}x(t),\hfill \end{array}$$

(63)

$$\begin{array}{cc}\hfill u(t)& =r(t)-\left(C_c{x}_c(t)+D_{c}x(t)\right),\hfill \end{array}$$

(64)

where the subscript c distinguishes the controller matrices $(A_c,B_c,C_c,D_c)$ from those of the plant $(A,B,C,D)$, and $x_c(t)$ represents the controller’s internal state.

Once the controller has been designed, the pre-filter F used to transform $\overline{r}(t)$ into $r(t)$ must be chosen such that the transfer function from $\overline{r}(t)$ to $y(t)$ is unitary to ensure asymptotic tracking. In this case, the static gain F is given by

$$F=-{\left(C{\left(A-I-B{D}_c+B{C}_c{(A_c-I)}^{-1}{B}_c\right)}^{-1}B\right)}^{-1}.$$

(65)

From Equation (63), we observe that when $C_c=0$, the state-feedback law reduces to the simpler static controller shown in Figure 3, with $D_c$ equivalent to K. Even in this case, the controller may still have an internal state, though it may not influence the output directly.

Since we are dealing with a dynamic system that includes process and measurement noise, the concept of stability must be extended. Specifically, we focus on Mean Square Stability (MSS) [38]. A system defined by Equations (32) and (34) is said to be mean square stable if and only if there exist ${\mu }_x\in {\mathbb{R}}^{n_x}$ and $M_x\in {\mathbb{R}}^{n_x\times n_x}$, with $M_x\ge 0$, such that

$$\begin{array}{cc}\hfill \underset{t\to \infty }{lim}\mathbb{E}\left\{x(t)\right\}& ={\mu }_x,\hfill \end{array}$$

(66)

$$\begin{array}{cc}\hfill \underset{t\to \infty }{lim}\mathbb{E}\left\{x(t)x^{⊺}(t)\right\}& =M_x.\hfill \end{array}$$

(67)

In practice, an internally unstable controller may still stabilize the control loop, but this is undesirable in real-world applications—especially in robotics—where instability could lead to physical damage.

The controller matrices have dimensions $A_c\in {\mathbb{R}}^{n_{x_c}\times n_{x_c}}$, $B_c\in {\mathbb{R}}^{n_{x_c}\times n_x}$, $C_c\in {\mathbb{R}}^{n_u\times n_{x_c}}$, and $D_c\in {\mathbb{R}}^{n_u\times n_x}$, where $n_{x_c}$ is the dimension of the controller’s internal state $x_c(t)$. Note that $n_{x_c}$ does not need to be equal to $n_x$.

If we set $r(t)=0$ for all t and $C_c=0$, the problem reduces to a regulation task, where the goal is for $x(t)\to 0$ as $t\to \infty$.

Consider a strictly causal plant as described in Equations (32) and (34) with $D=0$, and a controller defined by Equation (63). Then, the controller stabilizes the system in the mean square sense (as per Equation (66)) if and only if the eigenvalues of the block matrix $\overline{A}$ lie in the left half-plane (continuous-time) or within the unit circle (discrete-time), where

$$\overline{A}=\left[\begin{array}{cc}A-B{D}_c& -B{C}_c\\ B_c& A_c\end{array}\right].$$

(68)

If $\overline{A}$ is neither symmetric nor triangular, necessary and sufficient stability conditions cannot be easily obtained from the eigenvalues of its submatrices. However, the eigenvalues of the block matrix $\overline{A}$ are given by

$$\mathrm{eig}(\overline{A})=\mathrm{eig}(A-B{D}_c)\cup \mathrm{eig}(A_c+B_c{(A-B{D}_c)}^{-1}B{C}_c),$$

(69)

where, with slight abuse of notation, $\mathrm{eig}(X)$ denotes the set of dominant eigenvalues of matrix X.

Define the augmented state vector $\overline{x}(t)={\left[\begin{array}{cc}{x}^{⊺}(t)& x_c^{⊺}(t)\end{array}\right]}^{⊺}$. Then, the augmented system is

$$\begin{array}{c}\hfill \overline{\dot{x}}(t)=\underset{\overline{A}}{\underbrace{\left[\begin{array}{cc}A-B{D}_c& -B{C}_c\\ B_c& A_c\end{array}\right]}}\overline{x}(t)+\underset{\overline{B}}{\underbrace{\left[\begin{array}{c}B\\ 0_{n_{x_c}\times n_u}\end{array}\right]}}r(t)+\underset{\overline{v}(t)}{\underbrace{\left[\begin{array}{c}v(t)\\ 0_{n_{x_c}\times 1}\end{array}\right]}},\end{array}$$

(70)

where $0_{n_{x_c}\times 1}$ is a zero column vector of size $n_{x_c}$.

Discretizing the system using a zero-order hold and unit sampling time and unrolling the system in terms of its initial state yields

$$\overline{x}(t+1)={\overline{A}}^{t+1}\overline{x}(0)+\sum _{i=0}^t{\overline{A}}^i\left(\overline{B}r(t-i)+\overline{v}(t-i)\right).$$

(71)

Given that $v(t)$ is zero-mean white noise, the expectation is

$$\mathbb{E}\left\{\overline{x}(t+1)\right\}={\overline{A}}^{t+1}\overline{x}(0)+\sum _{i=0}^t{\overline{A}}^i\overline{B}\mathbb{E}\left\{r(t-i)\right\}.$$

(72)

This is a matrix power series, so to ensure a finite expectation $\mathbb{E}\left\{\overline{x}(t+1)\right\}$ as $t\to \infty$, it is required that

$$\parallel \overline{A}{\parallel }_2<1,$$

(73)

which, in terms of the dominant eigenvalue, becomes

$$|\mathrm{eig}(\overline{A})|<1.$$

(74)

A similar analysis for the second-order moment of $\overline{x}(t)$ leads to the same condition on $\overline{A}$, since the only term contributing to the variance is $v(t)$, which is assumed to have finite variance.

Note that the magnitude of the eigenvalues directly affects the system’s convergence rate: the closer the eigenvalues are to the origin (while remaining within the unit circle), the faster the transient response decays. Conversely, if eigenvalues lie outside the unit circle, the system will exhibit divergent or oscillatory behavior.

Finally, assume for a moment that $B_c=0$ or $C_c=0$. In either case, the controller cannot be internally unstable, and $D_c$ must be designed such that $A-B{D}_c$ has all eigenvalues within the unit circle. Otherwise, neither the loop nor the controller can be guaranteed to be stable.

4.3. Lyapunov–Based Stability Analysis

In the classical LQR framework, the solution P to the Algebraic Riccati Equation (ARE) in Equation (61) defines a quadratic Lyapunov function $V(x)=x^{\top }Px$, which certifies asymptotic stability of the closed-loop system when $u(t)=-Kx(t)$ and $A-BK$ have eigenvalues in the left half-plane.

For the generalized state-feedback controller in Equation (63), the stability condition $\parallel \overline{A}{\parallel }_2<1$ in Equation (73) ensures Mean Square Stability (MSS) in the presence of zero-mean Gaussian noise. While this is equivalent to a stochastic Lyapunov condition in expectation, direct construction of such a Lyapunov function for the full augmented system remains a complex task and is considered future work.

5. Bayesian Optimization

Bayesian Optimization (BO) [39] is a derivative-free approach for the global optimization of expensive black-box functions f. It belongs to a class of sequential model-based optimization algorithms that use past evaluations of the function to determine the next sampling point.

To understand its necessity, consider a generic maximization problem, as follows:

$${\mathbf{x}}^{*}=arg\underset{\mathbf{x}\in \mathcal{X}}{max}f(\mathbf{x}),$$

(75)

where ${\mathbf{x}}^{*}$ is the global optimizer, and $\mathcal{X}\subset {\mathbb{R}}^m$ is a bounded domain from which $\mathbf{x}$ is selected.

Different variants of Bayesian Optimization employ various acquisition functions to determine the next evaluation point based on the current posterior distribution of the function. The method typically uses Gaussian Processes (GPs) [40] as surrogate models, characterized by a mean function $\mu (\mathbf{x})$ and a covariance function $k(\mathbf{x},{\mathbf{x}}^{\prime })$. For n data points, the function values $f_{1:n}=f({\mathbf{x}}_1),\dots ,f({\mathbf{x}}_n)$ can be modeled using a multivariate Gaussian distribution, as follows:

$$f_{1:n}\sim \mathcal{N}(\mu ({\mathbf{x}}_{1:n}),\mathbf{K}),$$

(76)

where $\mathbf{K}$ is the $n\times n$ kernel matrix defined as

$$\mathbf{K}=\left(\begin{array}{ccc}k({\mathbf{x}}_1,{\mathbf{x}}_1)& \dots & k({\mathbf{x}}_1,{\mathbf{x}}_n)\\ ⋮& \ddots & ⋮\\ k({\mathbf{x}}_n,{\mathbf{x}}_1)& \dots & k({\mathbf{x}}_n,{\mathbf{x}}_n)\end{array}\right),$$

(77)

for a positive-definite kernel such as the Gaussian or Matérn kernel.

An acquisition function determines where to sample next by balancing the trade-off between exploitation and exploration. In this paper, we focus on Expected Improvement (EI) [41].

Expected Improvement evaluates the function at the point where the Expected Improvement over the best observed value $f({\mathbf{x}}^{+})$ is maximized, given the current GP model, as follows:

$$\begin{array}{cc}\hfill EI(\mathbf{x})=& \mathbb{E}[max\{0,f(\mathbf{x})-f({\mathbf{x}}^{+})-\xi \}]\hfill \\ \hfill =& (\mu (\mathbf{x})-f({\mathbf{x}}^{+})-\xi )\mathrm{\Phi }\left({\displaystyle \frac{\mu (\mathbf{x})-f({\mathbf{x}}^{+})-\xi }{\sigma (\mathbf{x})}}\right)\hfill \\ \hfill & +\sigma (\mathbf{x})\varphi \left({\displaystyle \frac{\mu (\mathbf{x})-f({\mathbf{x}}^{+})-\xi }{\sigma (\mathbf{x})}}\right),\hfill \end{array}$$

(78)

where

-

$f({\mathbf{x}}^{+})$ is the best function value observed so far;

-

$\mu (\mathbf{x})$ is the posterior mean at $\mathbf{x}$;

-

$\sigma (\mathbf{x})$ is the posterior standard deviation;

-

$\xi$ is a small positive parameter encouraging exploration;

-

$\mathrm{\Phi }(·)$ and $\varphi (·)$ denote the cumulative distribution function (CDF) and the probability density function (PDF) of the standard normal distribution, respectively.

In our implementation, Bayesian Optimization is used to tune the generalized controller parameters $(A_c,B_c,C_c,D_c)$ with the goal of minimizing a spectral norm-based cost function $\parallel \mathrm{Re}\left\{\mathrm{eig}(\overline{A})\right\}{\parallel }_{\infty }$, which directly controls the dominant eigenvalue and thus the convergence rate. The optimization is carried out using Gaussian Processes and the Expected Improvement acquisition function. Each function evaluation corresponds to a closed-loop simulation or eigenvalue computation.

6. Simulation Results

In Section 6.1, we first introduce the generalized state-feedback controller in its simplest form, focusing solely on its structure without addressing parameter tuning. This presentation is based on Chapter 3 of [5]. While the aforementioned reference focuses on the rotary inverted pendulum, Section 6.2 and Section 6.3 employ Bayesian Optimization to tune the controller parameters instead of using reinforcement learning as in [5].

6.1. Toy Example

Consider a discrete-time SISO plant (single-input single-output, with $n_u=1$ and $n_y=1$) given by

$$\begin{array}{cc}\hfill x[k+1]& =Ax\left[k\right]+Bu\left[k\right]+v\left[k\right],\hfill \end{array}$$

(79)

$$\begin{array}{cc}\hfill y\left[k\right]& =Cx\left[k\right]+w\left[k\right],\hfill \end{array}$$

(80)

where $v\left[k\right]$ and $w\left[k\right]$ denote process and measurement noise, respectively—both assumed to be zero-mean white noise with unit variance.

Let A, B, C, and D be defined as

$$\begin{array}{cc}\hfill A& =\left[\begin{array}{cc}0.5& 0\\ 0.7& 1.2\end{array}\right],\hfill \end{array}$$

(81)

$$\begin{array}{cc}\hfill B& =\left[\begin{array}{c}0\\ 0.1\end{array}\right],\hfill \end{array}$$

(82)

$$\begin{array}{cc}\hfill C& =\left[\begin{array}{cc}1& 1\end{array}\right],\hfill \end{array}$$

(83)

$$\begin{array}{cc}\hfill D& =0.\hfill \end{array}$$

(84)

The plant is internally unstable, as one of its eigenvalues lies outside the unit circle. Therefore, controller parameters $A_c$, $B_c$, $C_c$, and $D_c$ must be designed to generate the scalar control signal $u\left[k\right]$, as follows:

$$\begin{array}{cc}\hfill x_c[k+1]& =A_c{x}_c\left[k\right]+B_{c}x\left[k\right],\hfill \end{array}$$

(85)

$$\begin{array}{cc}\hfill u\left[k\right]& =r\left[k\right]-\left(C_c{x}_c\left[k\right]+D_{c}x\left[k\right]\right),\hfill \end{array}$$

(86)

with the following matrix values:

$$\begin{array}{cc}\hfill A_c=0.4& B_c=\left[\begin{array}{cc}1& -1.52\end{array}\right],\hfill \\ \hfill C_c=-0.5& D_c=\left[\begin{array}{cc}0.3& 2.1\end{array}\right].\hfill \end{array}$$

These values are chosen such that the augmented system matrix $\overline{A}$ has prescribed eigenvalues: $\mathrm{eig}(\overline{A})\in \{0.5,0.59,0.8\}$.

Note from Equation (85) that, in this example, full-state feedback is assumed. The reference is defined as

$$r\left[k\right]=\left\{\begin{array}{cc}0\hfill & \mathrm{if}k<60,\hfill \\ 10\hfill & \mathrm{if}k\ge 60.\hfill \end{array}\right.$$

(87)

We compare this control scheme to the classical state-feedback architecture shown in Figure 3, where the output is $y\left[k\right]$ and the control signal is

$$u\left[k\right]=r\left[k\right]-Kx\left[k\right],$$

(88)

with gain matrix, as follows:

$$K=\left[\begin{array}{cc}0.3& 4\end{array}\right],$$

chosen so that $\mathrm{eig}(A-BK)\in \{0.5,0.8\}$.

As expected from the eigenvalue analysis of $\overline{A}$ and $A-BK$, the closed-loop system reaches stationarity at the same rate (defined by the dominant eigenvalue, which governs the slowest decaying mode). However, $D_c$ and K differ, because the generalized controller in Figure 4 introduces additional degrees of freedom compared with the classical scheme in Figure 3, at the cost of adding an extra natural mode to the closed-loop response.

Simulation results in Figure 5 show that the additional degrees of freedom allow tuning the controller such that stationarity is achieved at the same rate while producing a smaller control signal than in the classical case.

Figure 6 presents the same comparison using a sinusoidal reference. Results confirm the previous observation: both controllers yield the same convergence speed, but the dynamic controller results in lower control signal peaks due to its additional degrees of freedom. Note, however, that tracking is only achievable if the closed-loop dynamics are sufficiently fast to follow changes in the reference.

6.2. Experimental Results: Rotary Inverted Pendulum

When considering the parameters from Table 1 for the model described in Equations (32) and (34), the following matrices are obtained:

$$\begin{array}{cc}\hfill A& =\left[\begin{array}{ccccc}0& 1& 0& 0& 0\\ 0& -0.31& 2.99& -0.02& 81.47\\ 0& 0& 0& 1& 0\\ 0& -0.02& 61.49& -0.5& 63.88\\ 0& -261.94& 0& 0& -800\end{array}\right],\hfill \end{array}$$

(89)

$$\begin{array}{cc}\hfill B& ={\left[\begin{array}{ccccc}0& 0& 0& 0& 100\end{array}\right]}^{⊺},\hfill \end{array}$$

(90)

$$\begin{array}{cc}\hfill C& =\left[\begin{array}{ccccc}1& 0& 0& 0& 0\\ 0& 0& 1& 0& 0\\ 0& 0& 0& 0& 1\end{array}\right].\hfill \end{array}$$

(91)

The performance of the proposed controller in Equation (63) for the rotary inverted pendulum is compared against the classical state-feedback controller defined in Equation (59). To this end, matrices Q and R from Equation (58) are designed to assign more weight to states that are more relevant to the specific experiment. In this case, priority is given to the pendulum position ${\theta }_1$, while arm and pendulum velocities are also considered important. The chosen matrices are

$$\begin{array}{cc}\hfill Q& =\left[\begin{array}{ccccc}1& 0& 0& 0& 0\\ 0& 10& 0& 0& 0\\ 0& 0& 100& 0& 0\\ 0& 0& 0& 10& 0\\ 0& 0& 0& 0& 1\end{array}\right],\hfill \end{array}$$

(92)

$$\begin{array}{cc}\hfill R& =10,\hfill \end{array}$$

(93)

where the penalty matrix R is selected to avoid sudden changes in the control signal that could destabilize the pendulum due to the DC motor’s behavior.

Solving Equation (61) with the model in Equation (89) yields

$$K=\left[\begin{array}{ccccc}-0.31& -5.26& 70.74& 8.92& 0.17\end{array}\right].$$

(94)

Alternatively, for controllers of the form given in Equation (63), parameters are obtained by applying Bayesian Optimization with a maximum of 150 iterations, minimizing

$${∥\mathrm{Re}\left\{\mathrm{eig}(\overline{A})\right\}∥}_{\infty },$$

(95)

where $\overline{A}$ is the closed-loop matrix defined in Equation (68), and $\mathrm{Re}\{\mathrm{eig}(M)\}$ denotes the real parts of the eigenvalues of matrix M. The minimization of the infinity norm constrains the maximum eigenvalue, thus limiting the dominant system response speed. This yields

$$\begin{array}{cc}\hfill A_c& =-100,\hfill \end{array}$$

(96)

$$\begin{array}{cc}\hfill B_c& =\left[\begin{array}{ccccc}0& 0& 0& 0& 0\end{array}\right],\hfill \end{array}$$

(97)

$$\begin{array}{cc}\hfill C_c& =-0.5,\hfill \end{array}$$

(98)

$$\begin{array}{cc}\hfill D_c& =\left[\begin{array}{ccccc}-20.96& -39.76& 72.74& 92.61& -0.58\end{array}\right].\hfill \end{array}$$

(99)

Figure 7 shows the arm and pendulum positions in the upper and middle subplots, respectively. It can be observed that both controllers successfully stabilize the pendulum while controlling the arm position. The lower subplot confirms the same behavior seen in Figure 6, where the controller in Equation (63) achieves similar performance to the LQR controller in Equation (59), but with reduced control effort. Figure 8 further illustrates this when the arm reference is changed while maintaining pendulum stabilization.

Although not shown with additional figures, the simulations included variations in reference signals (through step functions) and initial conditions. The controller consistently achieved stable regulation and reference tracking, demonstrating robustness and effective performance across different dynamic scenarios.

It is important to note that Bayesian Optimization is not used to compute the conventional K gain in Figure 3, which can be derived via ARE. Instead, Bayesian Optimization tunes the parameters of the generalized dynamic controller in Figure 4, enabling improved performance (e.g., lower control energy) not achievable by LQR alone.

The code for obtention of results for this section is available at https://github.com/miguel-a-solis/MDPI_Mathematics_SI-2025 (accessed on 1 August 2025).

6.3. Double Inverted Pendulum

Based on the model from Equation (54), the matrices Q and R in Equation (58) are defined to prioritize the pendulum position over the first-link position, which is less relevant in this case. The matrices are

$$\begin{array}{cc}\hfill Q& =\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 10& 0& 0\\ 0& 0& 100& 0\\ 0& 0& 0& 10\end{array}\right].\hfill \end{array}$$

(100)

$$\begin{array}{cc}\hfill R& =10,\hfill \end{array}$$

(101)

Solving Equation (61) yields the LQR gain, as follows:

$$K=\left[\begin{array}{cccc}6.95& -0.06& -0.03& 2.21\end{array}\right].$$

(102)

By minimizing the infinity norm of the real parts of the closed-loop eigenvalues as described in Equation (95), the following controller parameters are obtained:

$$\begin{array}{cc}\hfill A_c& =-1.29,\hfill \end{array}$$

(103)

$$\begin{array}{cc}\hfill B_c& =\left[\begin{array}{cccc}0& 0& 0& 0\end{array}\right],\hfill \end{array}$$

(104)

$$\begin{array}{cc}\hfill C_c& =-0.5,\hfill \end{array}$$

(105)

$$\begin{array}{cc}\hfill D_c& =\left[\begin{array}{cccc}60.01& 22.84& 0.53& 94.06\end{array}\right].\hfill \end{array}$$

(106)

Figure 9 shows the pendulum position (top subplot) and control signal (bottom subplot). Both controllers stabilize the pendulum effectively; however, the generalized state-feedback controller achieves this with a lower-amplitude control signal, which corresponds to a reduced voltage demand on the motor.

The proposed controller was tested under different operating points and state configurations. Despite the structural differences between the rotary inverted pendulum and double inverted pendulum systems, the generalized controller maintained comparable performance with the LQR approach, while reducing control effort. This highlights its adaptability and potential applicability to a broader class of underactuated systems.

6.4. Numerical Performance

The following

Table 2. Numerical performance indices on different scenarios.

System	Settling Time (s)	Max Overshoot (°)	ISE
Double Inverted Pendulum (BO)	∼18	13.5	43.1
Furuta (Step Reference, BO)	∼6	10.0	170.2
Furuta (Constant Reference, BO)	∼9	40.0	129.7

summarizes key performance indices for each control scenario, including the settling time, maximum overshoot, and Integral Square Error (ISE), which reflects the cumulative deviation from the desired output, as follows:

The generalized controller shows reasonably fast convergence, limited overshoot, and bounded control effort, even under large reference changes. The inclusion of ISE complements this analysis by quantifying the overall tracking accuracy and disturbance rejection.

Beyond the settling time and overshoot metrics, the Integral of Squared Error (ISE) provides an additional perspective on the control performance. In the double inverted pendulum scenario, the ISE remains within acceptable bounds given the system’s high instability and extended transient phase. For the Furuta pendulum with step reference, the controller achieves fast convergence and low ISE, indicating precise tracking with minimal sustained error. In contrast, the constant reference case exhibits a higher ISE, mainly due to the sharp change in the reference signal that induces a brief but significant transient error. Nevertheless, the system stabilizes quickly, and no long-term oscillations are observed. These results confirm that the controller tuned via Bayesian Optimization offers reliable performance across diverse conditions, balancing transient accuracy with stability, even under nonlinear and underactuated dynamics.

Although the proposed approach demonstrates promising results, one limitation lies in the practical implementation of full-state feedback controllers. In real-world scenarios, not all state variables are directly measurable or observable, especially in underactuated systems where sensing is limited due to cost, hardware constraints, or noise susceptibility. As discussed in [42], full-state feedback often assumes ideal access to all state variables, which is rarely the case. Instead, output-feedback or observer-based designs are commonly used in practice [43], which introduces additional challenges such as robustness to estimation errors and increased computational complexity.

Therefore, while the generalized controller proposed in this article provides a flexible structure for performance improvement, future work should address its integration with state estimators or reduced-order observers to enhance its applicability in practical control systems.

7. Conclusions

This work has presented a Bayesian Optimization approach for tuning a generalized state-feedback controller and compared its performance against the classical controller obtained by solving the Linear Quadratic Regulator (LQR) problem. Simulation results demonstrated that the proposed control scheme achieves the same tracking performance as the classical one, but with reduced control effort.

In addition, a well-known underactuated system—the rotary inverted pendulum (also known as the Furuta pendulum)—was studied, and a corresponding linear model was derived. Simulations were conducted using this plant model, and once again, the generalized controller exhibited tracking performance comparable to that of the classical state-feedback controller, while requiring lower control signal amplitudes. In contrast with the rotary inverted pendulum, on the double inverted pendulum, all results were obtained in a simulated environment, where implementation on a physical prototype is left as future work.

It is important to note that our methodology assumes full state availability, which may not be realistic in many practical systems. For instance, the second state on the observation matrix for the rotary inverted pendulum is typically not directly measurable. There is literature on observer-based control, such as proportional-integral observers for wind energy systems and attack-tolerant observers in cyber-physical systems, that provides promising alternatives to reconstruct unmeasured states. Future work will explore integrating such observer architectures into the Bayesian tuning process.