Optimal Control Based on Reinforcement Learning for Flexible High-Rise Buildings with Time-Varying Actuator Failures and Asymmetric State Constraints

Abstract

This study centers on the vibration suppression of high-rise building systems under extreme conditions, exploring a reinforcement learning (RL)-based vibration control strategy for flexible building systems with time-varying faults and asymmetric state constraints. A mathematical model precisely depicting the dynamic characteristics of flexible high-rise buildings, considering the time-varying nature of actuator faults, is initially established. Subsequently, a reinforcement learning-based controller is devised to counteract the negative impacts of faults on system performance. By introducing a time-varying asymmetric Lyapunov function, system state constraints are ensured, safeguarding system stability and security. The stability of the closed-loop system is rigorously proven using the Lyapunov stability theory, guaranteeing stable vibration suppression performance even in the presence of faults. The simulation results indicate that the proposed reinforcement learning vibration control method can effectively reduce the vibration response of flexible high-rise buildings when facing time-varying actuator faults. This demonstrates its remarkable robustness and adaptability, presenting a novel and effective solution for vibration control in real-world flexible high-rise buildings.

1. Introduction

In the field of modern civil engineering, high-rise buildings are being developed to be higher and more flexible, which is in line with the concept of smart cities pursuing efficient, intelligent and sustainable development [1]; however, this development trend also leads to a continuous decline in the overall structural stiffness and damping of the building system. This poses severe challenges to the control of structural responses under dynamic loads such as wind and earthquakes, including the nonlinear and time-varying nature of the load, the coupling of high-frequency and low-frequency vibrations, the uncertainty in the load patterns, the limitations of actuators under extreme conditions, and the need for real-time adaptive control to ensure stability and safety. Due to their inherent structural characteristics, flexible high-rise buildings have complex vibration modes and low damping, making them prone to significant vibrations. This not only affects the comfort of occupants within the building but also potentially endangers the structural safety and durability. To enhance comfort and safety, various countries have established certain standards for building systems [2]. To suppress the vibrations of building systems, scholars have proposed numerous methods. Traditional control measures, such as passive control [3], can reduce structural vibrations to a certain extent, but they often have fixed parameter settings and are difficult to flexibly adjust the control effect according to the changing external excitation and real-time structural status. In contrast, active control systems have real-time adjustment capabilities [4,5], although they highly rely on precise structural models and sensor information, and have strict requirements for the performance stability of actuators. However, active control can better adapt to the changes in natural frequencies caused by the nonlinear characteristics of the building system and changes in the external environment, and is more suitable for the current situation of high-rise buildings. Therefore, active control methods are receiving increasing attention.

In practical applications, sudden faults may occur in high-rise building systems, which can affect the overall structure and safety of the system, especially sensor faults [6] and actuator faults [7,8]. The actuator output characteristics can change significantly, which can seriously damage the expected performance of the control system and lead to an unstable structural response. A more practical but challenging situation is when the fault parameters change over time, resulting in time-varying actuator faults [9]. Ji et al. [10] designed an adaptive control law with a Nussbaum function and an auxiliary signal for a three-dimensional variable-length flexible string system modeled by PDEs, considering time-varying actuator faults and unknown control directions. They verified the control effect through simulations, providing a basis for practical engineering and planning future research directions. Li et al. [11] studied a vibration single-link flexible manipulator system modeled by PDEs, considering nonlinear and time-varying actuator faults as well as unknown control directions. Their research provides important references and inspiration for this study in dealing with time-varying actuator faults, highlighting the necessity of innovative control strategies in complex fault scenarios.

Meanwhile, considering the safety of the system operation, it is necessary to restrict the input, output and state of the actual high-rise building system to ensure its safe and reliable operation. Zhang et al. [12] proposed boundary control for high-rise building systems with active mass dampers under input and output constraints for large flexible building systems. In actual high-rise building systems, the stiffness, mass and damping characteristics of the structure in different directions are complicated due to the asymmetry of the architectural design, which greatly increases the complexity of the control strategy design. Traditional control methods based on the assumption of symmetrical structures make it difficult to effectively address these issues. Wang [13] introduced a time-varying asymmetric constraint Lyapunov function for the tracking control of nonlinear periodic systems and used the adaptive backstepping method and single-parameter adjustment to achieve output tracking error convergence and state constraints. Numerical examples have verified its effectiveness for asymmetric constraint systems. Song et al. [14] combined error transfer transformation with BLF to develop a tracking control method that can handle delays and asymmetric time-varying constraints. Wan et al. [15] addressed the control problem of non-strict feedback nonlinear systems with uncertainties and asymmetric time-varying output constraints by using an adaptive fuzzy state observer and an asymmetric time-varying barrier Lyapunov function, which is worth learning from. Therefore, the research on asymmetric output constraints for flexible high-rise building systems is crucial and requires further in-depth study.

In recent years, reinforcement learning, as a cutting-edge technology in the field of artificial intelligence, has attracted extensive attention from scholars [16,17,18]. Lan et al. [19] proposed a scheme based on improved distributed cooperative reinforcement learning for the control problem of multi-agent systems in unknown dynamic obstacle environments, enhancing performance through a three-layer cooperative mechanism and neural network optimization, which was verified to be effective through simulation. Zhao et al. [20] designed fault-tolerant control for a two-degree-of-freedom nonlinear helicopter system by combining generalized learning neural networks with reinforcement learning, considering actuator faults to construct parameters and using the Lyapunov method to analyze stability. Simulation and experiments confirmed its effectiveness and feasibility, providing new methods and ideas. Meanwhile, in the field of intelligent structural control, adaptive variable stiffness intelligent structural vibration control based on a fuzzy control strategy combined with LSTM also provides a new research direction for structural vibration resistance [21]. In the method proposed in [22], the reinforcement learning framework adopted an actor–critic structure. The critic neural network was used to approximate the loss function to evaluate control performance and serve as input information, while the actor neural network adjusted the control output to minimize the cost function. In the face of high uncertainty caused by time-varying actuator faults and asymmetric constraints, reinforcement learning demonstrates unique advantages, as it can learn and adapt to system dynamics online, continuously optimize control decisions, and minimize structural vibration responses, thereby providing an innovative and highly promising solution for the vibration control of flexible high-rise buildings.

This study focuses on developing an optimal reinforcement learning control strategy for flexible high-rise building systems with time-varying actuator faults and asymmetric constraints. The main contributions of this study are as follows: (1) Unlike previous studies, this work addresses the common issue of time-varying actuator faults in high-rise buildings. The proposed RL strategy interacts with the environment during operation, capturing actuator fault information and dynamically adjusting control actions, overcoming the limitations of traditional methods and enhancing system stability and safety under fault conditions, thus advancing building safety control towards intelligent self-adaptation. (2) Due to asymmetric constraints, flexible high-rise buildings exhibit complex nonlinear behaviors, which traditional linear control methods fail to address. This study integrates asymmetric constraints with RL algorithms to optimize vibration control, reduce safety risks, and ensure the long-term stability of buildings, offering new insights for similar structures and contributing to innovation in civil engineering. (3) RL algorithms autonomously learn optimal strategies through environment interaction and reward feedback, with improvements in learning speed and convergence. Simulation analysis verifies the effectiveness of the proposed method and provides a new method for the flexible control of high-rise buildings, supporting the application of the algorithm in building control technology. This work fills a gap in the control of complex engineering systems, advancing the development of intelligent building control technologies.

2. Dynamic Model

The plan presented in Figure 1 depicts a single-story-tall building system. This system is composed of two flexible walls, which are interconnected by a rigid floor. An active mass damper in the form of a trolley moves on this rigid floor. This model is studied with reference to Quanser’s flexible building platform. To simplify this complex scenario, the left and right walls are modeled as two Euler–Bernoulli beams. Here, $L_f$ represents the story height. $J_c$ is the rotor moment of inertia of the damper. ${\mu }_f$ denotes the mass of the rigid floor, and ${\mu }_c$ is the mass of the damper. For a given time t, ${\iota }_l\left(y,t\right)$ and ${\iota }_r\left(y,t\right)$ are the elastic deflections of the floor walls at t time and y position, respectively. ${\iota }_l\left(L_f,t\right)$ and ${\iota }_r\left(L_f,t\right)$ are the elastic deflections of the left and right walls, respectively, at the position y. As the tops of the walls are linked by a rigid plate, the vibrations at the top of the left-hand wall and the right-hand wall are the same. In other words, we have ${\iota }_l\left(L_f,t\right)={\iota }_r\left(L_f,t\right)=\iota \left(L_f,t\right)$. The flexural rigidity of the flexible wall is denoted by $EI$, and $\rho$ represents the uniform mass per unit length of the flexible wall. $s\left(t\right)$ serves as the input for active boundary vibration control, while $x_c\left(t\right)$ indicates the position of the active mass damper within the xoy coordinate system.

Remark 1. 

For the sake of convenience, we present the following representation:

$${\left(\cdot \right)}^{\prime }={\displaystyle \frac{\partial \left(\cdot \right)}{\partial y}}, {\left(\cdot \right)}^{″}={\displaystyle \frac{{\partial }^2\left(\cdot \right)}{\partial y^2}}, {\left(\cdot \right)}^{‴}={\displaystyle \frac{{\partial }^3\left(\cdot \right)}{\partial y^3}}, {\left(\cdot \right)}^{\left(4\right)}={\displaystyle \frac{{\partial }^4\left(\cdot \right)}{\partial y^4}}, \stackrel{\cdot }{\left(\cdot \right)}={\displaystyle \frac{\partial \left(\cdot \right)}{\partial t}}, \stackrel{\cdot \cdot }{\left(\cdot \right)}={\displaystyle \frac{{\partial }^2\left(\cdot \right)}{\partial t^2}}$$

The kinetic energy $T$ of the system can be expressed as

$$\begin{array}{l}T={\displaystyle \frac{1}{2}}{{\displaystyle {\int }_0^{L_f}\rho \left[{\dot{\iota }}_l\left(y,t\right)\right]}}^{2}dy+{\displaystyle \frac{1}{2}}{{\displaystyle {\int }_0^{L_f}\rho \left[{\dot{\iota }}_r\left(y,t\right)\right]}}^{2}dy\\ +{\displaystyle \frac{1}{2}}{\mu }_f{\left[\dot{\iota }\left(L_f,t\right)\right]}^2+{\displaystyle \frac{1}{2}}{J}_c{\left[{\dot{x}}_c\left(t\right)\right]}^2\\ +{\displaystyle \frac{1}{2}}{\mu }_c{\left[{\dot{x}}_c\left(t\right)+\dot{\iota }\left(L_f,t\right)\right]}^2\end{array}$$

(1)

The potential energy $V$ is as follows:

$$V={\displaystyle \frac{1}{2}}{{\displaystyle {\int }_0^{L_f}EI\left[{\iota }_l^{″}\left(y,t\right)\right]}}^{2}dy+{\displaystyle \frac{1}{2}}{{\displaystyle {\int }_0^{L_f}EI\left[{\iota }_r^{″}\left(y,t\right)\right]}}^{2}dy$$

(2)

The virtual work, denoted as $\delta W\left(t\right)$, attributable to the non-conservative force is given as follows:

$$\delta W=s\left(t\right)\delta \left[x_c\left(t\right)+\iota \left(L_f,t\right)\right]$$

(3)

Hamilton’s principle is

$${\displaystyle {\int }_0^{t_f}\delta \left(T-V+W\right)}dt=0$$

(4)

We can obtain the dynamic model of a one-story flexible building system with an active mass damper as follows [12]:

$$\rho {\ddot{\iota }}_l\left(y,t\right)+EI{\iota }_l^{\left(4\right)}\left(y,t\right)=0$$

(5)

$$\rho {\ddot{\iota }}_r\left(y,t\right)+EI{\iota }_r^{\left(4\right)}\left(y,t\right)=0$$

(6)

$${\mu }_f\ddot{\iota }\left(L_f,t\right)-J_c{\ddot{x}}_c\left(t\right)-EI\left[{\iota }_l^{‴}\left(L_f,t\right)+{\iota }_r^{‴}\left(L_f,t\right)\right]=0$$

(7)

$${\iota }_l\left(0,t\right)={\iota }_r\left(0,t\right)={\iota }_l^{\prime }\left(0,t\right)={\iota }_r^{\prime }\left(0,t\right)={\iota }_l^{″}\left(L_f,t\right)={\iota }_r^{″}\left(L_f,t\right)=0$$

(8)

$${\iota }_l\left(L_f,t\right)={\iota }_r\left(L_f,t\right)=\iota \left(L_f,t\right)$$

(9)

$${\mu }_c\ddot{\iota }\left(L_f,t\right)+\left({\mu }_c+J_c\right){\ddot{x}}_c\left(t\right)=s\left(t\right)$$

(10)

Remark 2. 

Vibration Equations (5) and (6) describe the dynamics of the left and right walls of the building, respectively. Boundary conditions (8) and (9) describe the dynamical properties of the floor boundaries. Boundary conditions (7) and (10) describe the connection between the top floor and the controller.

To address the vibration control challenge in uncertain systems, this section incorporates the Assume Mode Method (AMM). The AMM is utilized to approximate the system dynamics, which possess infinite-dimensional characteristics. By relying on the Assume Mode Method, the elastic deflection of the flexible wall can be depicted as

$$\iota \left(y,t\right)={\displaystyle \sum _{i=1}^{\infty }{\phi }_i\left(y\right)}{T}_i\left(t\right)$$

(11)

where ${\phi }_i\left(y\right)$ and $T_i\left(t\right)$ correspond to the natural mode function and modal coordinates of the arm rod of the flexible manipulator, respectively. Considering that the low-order modes usually occupy the dominant position in the vibration process of a high-rise building system, this paper takes the first two modes for an in-depth study. We can obtain the following dynamic model [23]:

$$B\ddot{Q}+KQ=U$$

(12)

where $Q={\left[x_c,T_1,T_2\right]}^T\in R^3$ is a generalized coordinate, $U={\left[s\left(t\right),0,0\right]}^T\in R^3$ is the joint torque. $B\in R^{3\times 3}$ is the inertial matrix. $K\in R^{3\times 3}$ is the stiffness matrix. This is expressed as follows:

$$B\left[1,1\right]={\mu }_c+J_c$$

$$B\left[1,2\right]=B\left[2,1\right]={\displaystyle \frac{1}{4}}{\mu }_c{\phi }_1\left(L_f\right)$$

$$B\left[1,3\right]=B\left[3,1\right]={\displaystyle \frac{1}{4}}{\mu }_c{\phi }_3\left(L_f\right)$$

$$B\left[2,2\right]=\rho -{\displaystyle \frac{{\mu }_f+{\mu }_c}{2}}{\phi }_1^2\left(L_f\right)$$

$$B\left[3,3\right]=\rho -{\displaystyle \frac{{\mu }_f+{\mu }_c}{2}}{\phi }_2^2\left(L_f\right)$$

$$B\left[2,3\right]=B\left[3,2\right]={\displaystyle \frac{{\mu }_f+{\mu }_c}{4}}{\phi }_1\left(L_f\right){\phi }_2\left(L_f\right)$$

$$K=diag\left[0,w_1^2\rho ,w_2^2\rho \right]$$

where $w_n$ is the separation constant.

The system state space model is as follows:

If $x=\left[\begin{array}{l}{x}_1\\ x_2\end{array}\right]=\left[\begin{array}{l}Q\\ \dot{Q}\end{array}\right]$, there is

$$\left\{\begin{array}{l}{\dot{x}}_1=x_2\\ {\dot{x}}_2=-B^{-1}K{x}_1+B^{-1}U\end{array}\right.$$

(13)

This paper takes into account that faults are highly likely to occur during the long-term operation of high-rise building systems, and these faults often possess complex time-varying characteristics. Therefore, a kind of time-varying actuator fault is considered as follows:

$$U=\nu \tau +\overline{\tau }$$

(14)

where $\tau ={\left[{\tau }_1,0,0\right]}^T\in R^3$ denotes the designed actual control input, $\nu =diag\left[\hslash ,1,1\right]\in R^{3\times 3}$ is the unknown time-varying actuation effectiveness, and $\overline{\tau }={\left[{\overline{\tau }}_1,0,0\right]}^T\in R^3$ denotes the additive fault.

Assumption 1. 

There exist positive constants ${\underset{\_}{\nu }}_i$ and ${\overline{\nu }}_i$ such that ${\nu }_i\in \left[{\underset{\_}{\nu }}_i,{\overline{\nu }}_i\right]$. In addition, ${\overline{\tau }}_i$ is bounded.

When the above time-varying faults are considered, Formula (13) can be reduced to

$$\left\{\begin{array}{l}{\dot{x}}_1=x_2\\ {\dot{x}}_2=B^{-1}\left(-K{x}_1+\nu \tau +\overline{\tau }\right)\end{array}\right.$$

(15)

3. Controller Design and Stability Analysis

In this part of the discussion, an optimal control strategy for a flexible high-rise building system is introduced to realize vibration suppression. The state error is defined as $e=Q-Q_d$, where $Q={\left[x_c,T_1,T_2\right]}^T$ and $Q_d={\left[x_{cd},T_{1d},T_{2d}\right]}^T$ represents the desired value of the building, so $\dot{e}=\dot{Q}$. Therefore, the control goal aims to design a control scheme that satisfies $e\to 0,\dot{e}\to 0$ when $t\to \infty$.

3.1. Reinforcement Learning Control Algorithm

The actor–critic algorithm is used to design a reinforcement learning strategy to achieve high-rise building vibration control. For the control structure block diagram of the flexible high-rise building system in Figure 2, the whole control process can be clearly presented as follows: Firstly, the flexible high-rise building system is used as the control object, and the control law is generated through the network, which is the starting point of the whole control process. Subsequently, the actor NN starts to work, generates the update law, and determines the control strategy based on this update law. At the same time, the critic NN calculates the cost function, which leads to the update value and provides an evaluation basis for the system control. The RBF neural network is also involved, interacting with the outputs of the actor NN and the critic NN to assist in the system control. In terms of state processing, the desired state is set, which is compared to the actual state variables of the system and the error is calculated. At this point, the system needs to analyze and respond to possible failure scenarios, taking into account the potential factor of actuator failure. At the same time, asymmetric constraints come into play to limit the system state and ensure that the system operates within a reasonable range of states. After that, the stability of the system is analyzed in depth using the Lyapunov function to determine whether the system can operate stably. Finally, the system uncertainty is approximated, and the control strategy is further optimized based on the calculations and analyses in the previous sections, so that the flexible high-rise building system can reach the desired state as much as possible and achieve stable and reliable operation.

The long-term cost function is presented in the following way:

$$C_i\left(t\right)={\displaystyle {\int }_0^{\infty }{e}^{-\frac{m-t}{{\psi }_i}}{Y}_i\left(m\right)}dm$$

(16)

where ${\psi }_i$ is the constant, and used to approximate future cost. $Y_i\left(m\right)$ represents the immediate price, which is expressed as following:

$$Y_i\left(m\right)={\left(Q_i-Q_{di}\right)}^T{D}_i\left(Q_i-Q_{di}\right)+{{\tau }_i}^T{R}_i{\tau }_i$$

(17)

where matrices $D_i$ and $R_i$ are positive definite.

Due to the outstanding performance of the RBF neural network in nonlinear estimation, it is widely used in many fields in practical scenarios and plays an important role. For the continuous function $f\left(Z_i\right):{ℝ}^q\to ℝ$, define as follows:

$$f_{in}\left(Z_i\right)={W_i}^T{S}_i\left(Z_i\right)$$

(18)

where $Z_i$ represents the input of neural network, and $Z_i={\left[Z_{i1},Z_{i2},\cdots ,Z_{iq}\right]}^T\subset {ℝ}^q$. $W_i={\left[W_{i1},W_{i2},\cdots ,W_{iq}\right]}^T\subset {ℝ}^l$ with neural network node number $l>1$ denotes the weight of neural network. $S_i\left(Z_i\right)={\left[S_{i1}\left(Z_i\right),S_{i2}\left(Z_i\right),\cdots ,S_{il}\left(Z_i\right)\right]}^T$ is the radial basis function, where $S_{ik}\left(Z_i\right)$ adopts the Gaussian function form as follows:

$$S_{ik}\left(Z_i\right)=\exp \left[-{\displaystyle \frac{{\left(Z_i-{\upsilon }_k\right)}^T\left(Z_i-{\upsilon }_k\right)}{{\chi }^2}}\right]$$

(19)

where ${\upsilon }_k={\left[{\upsilon }_{k1},{\upsilon }_{k2},\cdots ,{\upsilon }_{kq}\right]}^T\left(k=1,2,\cdots ,l\right)$ is the center of the receptive field, and $\chi$ is the width of the Gaussian function. In this paper, an RBF neural network is applied to estimate unknown continuous functions on compact sets ${\mathsf{\Omega }}_{Z_i}\subset {ℝ}^q$.

$$f\left(Z_i\right)=W_i^{\ast T}{S}_i\left(Z_i\right)+{∍}_i,\forall Z_i\in {\mathsf{\Omega }}_{Z_i}$$

(20)

where ${W_i}^{\ast }$ represents the ideal weight, while ${∍}_i$ represents the bounded approximation error.

Based on the analysis above, consider $C_i=W_{iC}^{\ast T}{S}_{iC}\left(Z_{iC}\right)+{∍}_{iC}$; then, ${\widehat{C}}_i={\widehat{W}}_{iC}^T{S}_{iC}\left(Z_{iC}\right)$, where $Z_{iC}=e_{1i}$ is the error variables. ${∍}_{iC}$ is to critique the approximate residual of the neural network. By calculating (16), the approximate error of the cost function is expressed as

$${\Upsilon }_i\left(t\right)=Y_i\left(t\right)-{\displaystyle \frac{1}{{\psi }_i}}{\widehat{C}}_i\left(t\right)+{\dot{\widehat{C}}}_i\left(t\right)$$

(21)

When ${\psi }_i\to \infty$, the approximate error of the cost function can be set forth as

$${\Upsilon }_i\left(t\right)=Y_i\left(t\right)+{\dot{\widehat{C}}}_i\left(t\right)$$

(22)

By utilizing the gradient descent method, the critic neural network is updated as follows:

$${\dot{\widehat{W}}}_{iC}=-{\sigma }_{iC}{\displaystyle \frac{\partial E_{iC}}{\partial {\widehat{W}}_{iC}}}$$

(23)

where $E_{iC}={\displaystyle \frac{1}{2}}{{\Upsilon }_i}^T{\Upsilon }_i$ is to adjust the neural network weight ${\dot{\widehat{W}}}_{iC}$, and taking (22) into (23) yields

$$\begin{array}{l}{\dot{\widehat{W}}}_{iC}=-{\sigma }_{iC}{\Upsilon }_i\left(t\right){\displaystyle \frac{\partial {\Upsilon }_i}{\partial {\widehat{W}}_{iC}}}\\ =-{\sigma }_{iC}{\Upsilon }_i\left(t\right){\displaystyle \frac{\partial \left[Y_i\left(t\right)-\left(1/{\psi }_i\right){\widehat{C}}_i\left(t\right)+{\dot{\widehat{C}}}_i\left(t\right)\right]}{\partial {\widehat{W}}_{iC}}}\\ =-{\sigma }_{iC}\left(Y_i\left(t\right)+{\widehat{W}}_{iC}^T{\eta }_i\right){\eta }_i\end{array}$$

(24)

where ${\sigma }_{iC}>0$ represents the learning rate of the neural network, and ${\eta }_i=-{\displaystyle \frac{S_{iC}}{{\psi }_i}}+\nabla S_{iC}{\dot{Z}}_{iC}$, $\nabla$ is the gradient to $Z_{iC}$.

3.2. Stability Analysis

Define the error variables as follows:

$$e_1=Q-Q_d$$

(25)

$$e_2=\dot{Q}-\alpha$$

(26)

where $e_1$ is the state error and $e_2$ denotes the state variables, $Q$ and $Q_d$ denote the state variables and desired state variables of the high-rise building system, respectively. $\alpha ={\left[{\alpha }_1,{\alpha }_2,{\alpha }_3\right]}^T$ is a virtual variable; the specific expressions of dummy variables are given later. The derivation of Formulas (25) and (26) gives

$${\dot{e}}_1=e_2+\alpha =e_2-K_1{e}_1$$

(27)

$${\dot{e}}_2=\ddot{Q}-\dot{\alpha }=B^{-1}\left[\nu \tau +\overline{\tau }-KQ\right]-\dot{\alpha }$$

(28)

Choose the following Lyapunov candidate function as follows:

$$V=V_1+V_2$$

(29)

where

$$V_1={\displaystyle \frac{1}{2}}{e}_2^{T}B{e}_2$$

(30)

$$V_2={\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^{3}q\left(e_{1i}\right)ln\frac{N_{1i}^2}{N_{1i}^2-e_{1i}^2}}+{\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^3\left(1-q\left(e_{1i}\right)\right)}ln{\displaystyle \frac{N_{2i}^2}{N_{2i}^2-e_{1i}^2}}$$

(31)

where $q\left(e_{1i}\right)=\left\{\begin{array}{l}1 e_{1i}>0\\ 0 e_{1i}\le 0\end{array}\right.$, $N_{1i}$ and $N_{2i}$ are constraints on the error $e_{1i}$.

When $e_{1i}>0$, there are $q\left(e_{1i}\right)=1$, and Formula (31) can be reduced to

$$V_2={\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^{3}q\left(e_{1i}\right)ln\frac{N_{1i}^2}{N_{1i}^2-e_{1i}^2}}$$

(32)

When $e_{1i}\le 0$, there are $q\left(e_{1i}\right)=0$, and Formula (31) can be reduced to

$$V_2={\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^3\left(1-q\left(e_{1i}\right)\right)}ln{\displaystyle \frac{N_{2i}^2}{N_{2i}^2-e_{1i}^2}}$$

(33)

In summary, Formula (31) can be written as

$$V_2={\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^{3}ln\frac{1}{1-{\varpi }_i^2}}$$

(34)

where ${\varpi }_i=\left(1-q\left(e_{1i}\right)\right){\varpi }_{1i}+q\left(e_{1i}\right){\varpi }_{2i},i=1,2,3$, ${\varpi }_{1i}=\left[{\displaystyle \frac{e_{11}}{N_{21}}},{\displaystyle \frac{e_{12}}{N_{22}}},{\displaystyle \frac{e_{13}}{N_{23}}}\right]$, ${\varpi }_{2i}=\left[{\displaystyle \frac{e_{11}}{N_{11}}},{\displaystyle \frac{e_{12}}{N_{12}}},{\displaystyle \frac{e_{13}}{N_{13}}}\right]$.

The derivative of (29) is

$$\dot{V}={\dot{V}}_1+{\dot{V}}_2$$

(35)

where

$${\dot{V}}_1=e_2^{T}B{\dot{e}}_2$$

(36)

$$\begin{array}{l}{\dot{V}}_2={\displaystyle \sum _{i=1}^{3}ln\frac{{\varpi }_i{\dot{\varpi }}_i}{1-{\varpi }_i^2}}\\ ={\displaystyle \sum _{i=1}^{3}ln\frac{q_i{\varpi }_{2i}}{N_{1i}\left(1-{\varpi }_{2i}^2\right)}}\left({\dot{e}}_{1i}-e_{1i}{\displaystyle \frac{{\dot{N}}_{1i}}{N_{1i}}}\right)+{\displaystyle \sum _{i=1}^{3}ln\frac{\left(1-q_i\right){\varpi }_{1i}}{N_{2i}\left(1-{\varpi }_{1i}^2\right)}}\left({\dot{e}}_{1i}-e_{1i}{\displaystyle \frac{{\dot{N}}_{2i}}{N_{2i}}}\right)\end{array}$$

(37)

By further simplification, Equation (37) can be derived to

$$\begin{array}{l}{\dot{V}}_2={\displaystyle \sum _{i=1}^{3}ln\frac{q_i{\varpi }_{2i}}{N_{1i}\left(1-{\varpi }_{2i}^2\right)}}\left(e_{2i}+{\alpha }_i-e_{1i}{\displaystyle \frac{{\dot{N}}_{1i}}{N_{1i}}}\right)\\ +{\displaystyle \sum _{i=1}^{3}ln\frac{\left(1-q_i\right){\varpi }_{1i}}{N_{2i}\left(1-{\varpi }_{1i}^2\right)}}\left(e_{2i}+{\alpha }_i-e_{1i}{\displaystyle \frac{{\dot{N}}_{2i}}{N_{2i}}}\right)\end{array}$$

(38)

where $\alpha =-K_1{e}_1-K_2{e}_1$ and $K_1=K_1^T>0$, where $K_1$ is a positive definite 3-dimensional matrix, $K_2=diag\left[k_{21}\left(t\right),k_{22}\left(t\right),k_{23}\left(t\right)\right]$, with $k_{2i}\left(t\right)=\sqrt{{\left({\dot{N}}_{1i}/N_{1i}\right)}^2+{\left({\dot{N}}_{2i}/N_{2i}\right)}^2+o_i}$. The positive constants of $o_i$ are to be designed, ensuring that $k_{2i}\left(t\right)$ is always bounded.

Lemma 1. 

In the scope $\left|\varpi \right|<1$, the following inequality holds for $\varpi \in ℝ$ [24].

$$ln{\displaystyle \frac{1}{1-{\varpi }^2}}\le {\displaystyle \frac{{\varpi }^2}{1-{\varpi }^2}}$$

(39)

Using the formula ${\varpi }_{1i}$, ${\varpi }_{2i}$, $\alpha$, and Lemma 1, Formula (38) can be reduced to

$${\dot{V}}_2\le {\displaystyle \sum _{i=1}^3{\varphi }_i{e}_{1i}{e}_{2i}}-{\displaystyle \sum _{i=1}^3\frac{{\varpi }_i^2}{\left(1-{\varpi }_i^2\right)}}\left(k_{1i}+k_{2i}+q_i{\displaystyle \frac{{\dot{N}}_{1i}}{N_{1i}}}+\left(1-q_i\right){\displaystyle \frac{{\dot{N}}_{2i}}{N_{2i}}}\right)$$

(40)

where ${\varphi }_i={\displaystyle \frac{q_i}{N_{1i}^2-e_{1i}^2}}+{\displaystyle \frac{1-q_i}{N_{2i}^2-e_{1i}^2}}$. Considering that $k_{2i}+q_i{\displaystyle \frac{{\dot{N}}_{1i}}{N_{1i}}}+\left(1-q_i\right){\displaystyle \frac{{\dot{N}}_{2i}}{N_{2i}}}>0$ in Formula (40), the following can be obtained:

$${\dot{V}}_2\le {\displaystyle \sum _{i=1}^3{\varphi }_i{e}_{1i}{e}_{2i}}-{\displaystyle \sum _{i=1}^3\frac{{\varpi }_i^2}{\left(1-{\varpi }_i^2\right)}}{k}_{1i}$$

(41)

Considering (36) and (41), we obtain

$$\dot{V}\le e_2^{T}B{\dot{e}}_2+{\displaystyle \sum _{i=1}^3{\varphi }_i{e}_{1i}{e}_{2i}}-{\displaystyle \sum _{i=1}^3\frac{{\varpi }_i^2}{\left(1-{\varpi }_i^2\right)}}{k}_{1i}$$

(42)

Substitute Formula (28) into Formula (42) to obtain

$$\dot{V}\le e_2^T\left(\nu \tau +\overline{\tau }-KQ-B\dot{\alpha }\right)+{\displaystyle \sum _{i=1}^3{\varphi }_i{e}_{1i}{e}_{2i}}-{\displaystyle \sum _{i=1}^3\frac{{\varpi }_i^2}{\left(1-{\varpi }_i^2\right)}}{k}_{1i}$$

(43)

Because of this, the design control system input is as follows:

$$\tau 1={\nu }^{-1}\left(-\varphi e_1-K_3{e}_2+KQ+B\dot{\alpha }-\overline{\tau }\right)$$

(44)

where $K_3=K_3^T>0$ is an augmented moment matrix. Substituting control input (44) into (43) yields

$$\dot{V}\le -{\displaystyle \sum _{i=1}^3\frac{{\varpi }_i^2}{\left(1-{\varpi }_i^2\right)}}{k}_{1i}-e_2^T{K}_3{e}_2$$

(45)

Because of the uncertainty of dynamic information, the control system based on the above model is not suitable for a control system in the real world. To overcome this challenge, based on Section 3.1, we design the following control law:

$$\tau 2={\nu }^{-1}\left(-\varphi e_1-K_3{e}_2+{\widehat{W}}_a^T{S}_a\left(Z_a\right)\right)$$

(46)

where $W_a$ is the weight of executing the divine channel network, and $S_a\left(Z_a\right)$ represents the basis function of the execution network. The definition of ${\widehat{W}}_a^T{S}_a\left(Z_a\right)$ approaching value $W_a^{\ast T}{S}_a\left(Z_a\right)$ is as follows:

$$W_a^{\ast T}{S}_a\left(Z_a\right)=KQ+B\dot{\alpha }-\overline{\tau }+\epsilon \left(Z_a\right)$$

(47)

where $Z_a$ represents the input data, and $\epsilon \left(Z_a\right)$ represents the approach error.

The definite approximation error is ${\zeta }_a={\tilde{W}}_a^T{S}_a\left(Z_a\right)$ and the error of the actor neural network is designed as follows:

$${\mathsf{\Psi }}_a={\zeta }_a+K_4\left(\widehat{C}\left(t\right)-C_d\left(t\right)\right)$$

(48)

where $K_4\in {ℝ}^3>0$. The value of the hope cost function in the definite period is 0.

Definition ${\mathsf{\Xi }}_a={\displaystyle \frac{1}{2}}{\mathsf{\Psi }}_a^T{\mathsf{\Psi }}_a$. The law of design actor neural network update rate is as follows:

$${\dot{\widehat{W}}}_a=-{\sigma }_a{\displaystyle \frac{\partial {\mathsf{\Xi }}_a}{\partial {\widehat{W}}_a}}$$

(49)

Hence, substituting (48) into (49) yields

$${\dot{\widehat{W}}}_a=-{\sigma }_a{\displaystyle \frac{\partial {\mathsf{\Xi }}_a}{\partial {\mathsf{\Psi }}_a}}{\displaystyle \frac{\partial {\mathsf{\Psi }}_a}{\partial {\zeta }_a}}{\displaystyle \frac{\partial {\zeta }_a}{\partial {\widehat{W}}_a}}=-{\sigma }_a\left({\zeta }_a+K_4\widehat{C}\right)S_a$$

(50)

where ${\sigma }_a>0$ is the learning rate of actor neural network. The new updating law is defined as

$${\dot{\widehat{W}}}_a=-{\sigma }_a\left({\widehat{W}}_a^T{S}_a\left(Z_a\right)+K_4\widehat{C}\right)S_a$$

(51)

A Lyapunov function is defined as

$$V_3={\displaystyle \frac{1}{2}}{\displaystyle \sum _{i=1}^3{\tilde{W}}_{iC}^T{\tilde{W}}_{iC}}$$

(52)

where ${\tilde{W}}_{iC}=W_{iC}^{\ast }-{\widehat{W}}_{iC}$ is the parameter estimation error.

Derivation of the above equation:

$${\dot{V}}_3={\displaystyle \sum _{i=1}^3{\tilde{W}}_{iC}^T{\dot{\widehat{W}}}_{iC}}$$

(53)

Substitute Formula (24) into Formula (53) to obtain

$${\dot{V}}_3={\displaystyle \sum _{i=1}^3{\tilde{W}}_{iC}^T\left(-{\sigma }_{iC}\left(Y_i\left(t\right)+{\widehat{W}}_{iC}^T{\eta }_i\right){\eta }_i\right)}$$

(54)

When $t\to \infty$, there is ${\Upsilon }_i\left(t\right)\le \tau$ and $\tau$ is a small positive constant. For this, the following can be obtained:

$$\begin{array}{l}{Y}_i\left(t\right)\le {\displaystyle \frac{1}{{\psi }_i}}{\widehat{C}}_i\left(t\right)+{\dot{\widehat{C}}}_i\left(t\right)+\tau \\ \le W_{iC}^{\ast T}{\displaystyle \frac{S_{iC}\left(Z_{iC}\right)}{{\psi }_i}}+{\displaystyle \frac{{∍}_{iC}}{{\psi }_i}}-\nabla \left(W_{iC}^{\ast T}{S}_{iC}\left(Z_{iC}\right)+{∍}_{iC}\right){\dot{Z}}_{iC}+\tau \\ \le -W_{iC}^{\ast T}{\eta }_i+{\beta }_{iC}\end{array}$$

(55)

where ${\beta }_{iC}={\displaystyle \frac{{∍}_{iC}}{{\psi }_i}}-\nabla {∍}_{iC}{\dot{Z}}_{iC}+\tau$, $‖{\beta }_{iC}‖\le {\beta }_{iC,\mathit{max}}$ is bounded.

Formula (54) is obtained by substituting (55):

$${\dot{V}}_3\le {\displaystyle \sum _{i=1}^3-{\sigma }_{iC}{\tilde{W}}_{iC}^T\left({\tilde{W}}_{iC}^T{\eta }_i+{\beta }_{iC}\right){\eta }_i}$$

(56)

Further simplification of the above equation can be obtained:

$${\dot{V}}_3\le {\displaystyle \sum _{i=1}^3-\frac{1}{2}{\sigma }_{iC}{\tilde{W}}_{iC}^T{\tilde{W}}_{iC}^T{\eta }_i^T{\eta }_i}+{\displaystyle \sum _{i=1}^3\frac{1}{2}{\sigma }_{iC}{\beta }_{iC}^T{\beta }_{iC}}$$

(57)

For the purpose of the stability analysis, the Lyapunov function is formulated as below:

$$V=V_1+V_2+{\displaystyle \frac{1}{2}}{\tilde{W}}_a^T{\tilde{W}}_a+{\displaystyle \frac{1}{2}}{\tilde{W}}_C^T{\tilde{W}}_C$$

(58)

The derivation of the above Equation (58) yields:

$$\begin{array}{l}\dot{V}\le -e_2^T{K}_3{\dot{e}}_2-{\displaystyle \sum _{i=1}^3\frac{{\varpi }_i^2}{\left(1-{\varpi }_i^2\right)}}{k}_{1i}+{\displaystyle \sum _{i=1}^3{e}_{2i}^T\left({\tilde{W}}_{ia}^T{S}_{ia}-{\xi }_{ia}\right)}\\ -{\displaystyle \sum _{i=1}^3{\sigma }_{ia}{\tilde{W}}_{ia}^T{S}_{ia}\left({\tilde{W}}_{ia}^T{S}_{ia}+K_{4i}{\widehat{C}}_i\right)}-{\displaystyle \sum _{i=1}^3{\sigma }_{iC}{\tilde{W}}_{iC}^T\left(Y_i\left(t\right)+{\widehat{W}}_{iC}^T{\eta }_i\right){\eta }_i}\end{array}$$

(59)

In which ${\widehat{C}}_i=W_{iC}^{\ast T}{S}_{iC}+{\tilde{W}}_{iC}^T{S}_{iC}$, have ${\widehat{C}}_i^T{\widehat{C}}_i\le 2{\left(W_{iC}^{\ast T}{S}_{iC}\right)}^T{W}_{iC}^{\ast T}{S}_{iC}+2{\left({\tilde{W}}_{iC}^T{S}_{iC}\right)}^T{W}_{iC}^{\ast T}{S}_{iC}$, the above formula can be reduced to

$$\begin{array}{l}\dot{V}\le -e_2^T{K}_3{\dot{e}}_2-{\displaystyle \sum _{i=1}^3\frac{{\varpi }_i^2}{\left(1-{\varpi }_i^2\right)}}{k}_{1i}-{\displaystyle \sum _{i=1}^3\frac{1}{2}\left({\sigma }_{ia}-1\right){‖{\tilde{W}}_{ia}‖}^2{‖S_{ia}‖}^2}\\ +{\displaystyle \sum _{i=1}^{3}2{\sigma }_{ia}{K}_{4i}^2}{‖{\tilde{W}}_{iC}^{\ast }‖}^2{‖S_{iC}‖}^2-{\displaystyle \sum _{i=1}^3\frac{1}{2}\left({\sigma }_{iC}{\eta }_i^T{\eta }_i-4{\sigma }_{ia}{K}_{4i}^2{‖S_{iC}‖}^2\right){‖{\tilde{W}}_{iC}^T‖}^2}\\ +{\displaystyle \sum _{i=1}^3\frac{1}{2}{\sigma }_{iC}{\beta }_{iC,\mathit{max}}^T{\beta }_{iC,\mathit{max}}}+{\displaystyle \sum _{i=1}^3\frac{1}{2}{\sigma }_{ia}{‖W_{ia}^{\ast }‖}^2{‖S_{ia}‖}^2}\end{array}$$

(60)

Lemma 2. 

Let the initial condition $V\left(0\right)$ be bounded and $V\left(t\right)\ge 0$ be continuous and bounded with [25]:

$$\dot{V}\left(t\right)\le -\mathsf{\Gamma }V+\mathsf{\Lambda }$$

(61)

where $\mathsf{\Gamma },\mathsf{\Lambda }>0$.

Considering Lemma 1 and 2, (60) can be rewritten as

$$\begin{array}{l}\dot{V}\le -e_2^T{K}_3{\dot{e}}_2-{\displaystyle \sum _{i=1}^3{k}_{1i}ln\frac{1}{1-{\varpi }_i^2}}-{\displaystyle \sum _{i=1}^3\frac{1}{2}\left({\sigma }_{ia}-1\right){‖{\tilde{W}}_{ia}‖}^2{‖S_{ia}‖}^2}\\ +{\displaystyle \sum _{i=1}^{3}2{\sigma }_{ia}{K}_{4i}^2}{‖W_{iC}^{\ast }‖}^2{‖S_{iC}‖}^2-{\displaystyle \sum _{i=1}^3\frac{1}{2}\left({\sigma }_{iC}{\eta }_i^2-4{\sigma }_{ia}{K}_{4i}^2{‖S_{iC}‖}^2\right){‖{\tilde{W}}_{iC}^T‖}^2}\\ +{\displaystyle \sum _{i=1}^3\frac{1}{2}{\sigma }_{iC}{\beta }_{iC,\mathit{max}}^T{\beta }_{iC,\mathit{max}}}+{\displaystyle \sum _{i=1}^3\frac{1}{2}{\sigma }_{ia}{‖W_{ia}^{\ast }‖}^2{‖S_{ia}‖}^2}\\ \le -\mathsf{\Gamma }V+\mathsf{\Lambda }\end{array}$$

(62)

where

$$\mathsf{\Gamma }=\mathit{min}\left\{\begin{array}{l}{\lambda }_{\mathit{min}}\left(2K_3\right),{\lambda }_{\mathit{min}}\left(2K_1\right),{\lambda }_{\mathit{min}}\left({\sigma }_{ia}-1\right)b_{is},\\ {\lambda }_{\mathit{min}}\left({\sigma }_{iC}{g}_i^2-4{\sigma }_{ia}{K}_{4i}^2{‖S_{iC}‖}^2\right)\end{array}\right\}$$

$$\mathsf{\Lambda }=+{\displaystyle \sum _{i=1}^3\frac{1}{2}{\sigma }_{iC}{\beta }_{iC,\mathit{max}}^T{\beta }_{iC,\mathit{max}}}+{\displaystyle \sum _{i=1}^3\frac{1}{2}{\sigma }_{ia}{‖W_{ia}^{\ast }‖}^2{‖S_{ia}‖}^2}+{\displaystyle \sum _{i=1}^{3}2{\sigma }_{ia}{K}_{4i}^2}{‖W_{iC}^{\ast }‖}^2{‖S_{iC}‖}^2$$

where $g_i\le ‖{\eta }_i‖$, $b_{is}\le ‖S_{ia}‖$, and in order to ensure $\mathsf{\Gamma }>0$, the following conditions are given by ${\lambda }_{\mathit{min}}\left(K_1\right)>0$, ${\lambda }_{\mathit{min}}\left({\sigma }_{ia}-1\right)b_{is}>0$, ${\lambda }_{\mathit{min}}\left({\sigma }_{iC}{g}_i^2-4{\sigma }_{ia}{K}_{4i}^2{‖S_{iC}‖}^2\right)>0$, ${\lambda }_{\mathit{min}}\left(K_3\right)>0$.

Theorem 1. 

Taking into account that all state information of the flexible high-rise building system is available and the initial condition is within bounded limits, the presented reinforcement learning control algorithm makes sure that the closed-loop system remains semi-globally and uniformly bounded.

These system signals $e_1$, $e_2$, ${\tilde{W}}_a$, and ${\tilde{W}}_C$ will eventually converge to ${\mathsf{\Omega }}_{e_1}$, ${\mathsf{\Omega }}_{e_2}$, ${\mathsf{\Omega }}_{{\tilde{W}}_a}$, and ${\mathsf{\Omega }}_{{\tilde{W}}_C}$, which are defined as follows:

$${\mathsf{\Omega }}_{e_1}=\left\{e_1\in {ℝ}^3\left|‖e_1‖\le \sqrt{2\left(V\left(0\right)+{\displaystyle \frac{\mathsf{\Lambda }}{\mathsf{\Gamma }}}\right)}\right.\right\}$$

$${\mathsf{\Omega }}_{e_2}=\left\{e_2\in {ℝ}^3\left|‖e_2‖\le \sqrt{{\displaystyle \frac{2\left(V\left(0\right)+\frac{\mathsf{\Lambda }}{\mathsf{\Gamma }}\right)}{{\lambda }_{\mathit{min}}\left(H\right)}}}\right.\right\}$$

$${\mathsf{\Omega }}_{{\tilde{W}}_a}=\left\{{\tilde{W}}_a\in {ℝ}^{l\times 3}\left|‖{\tilde{W}}_a‖\le \sqrt{2\left(V\left(0\right)+{\displaystyle \frac{\mathsf{\Lambda }}{\mathsf{\Gamma }}}\right)}\right.\right\}$$

$${\mathsf{\Omega }}_{{\tilde{W}}_C}=\left\{{\tilde{W}}_C\in {ℝ}^l\left|‖{\tilde{W}}_C‖\le \sqrt{2\left(V\left(0\right)+{\displaystyle \frac{\mathsf{\Lambda }}{\mathsf{\Gamma }}}\right)}\right.\right\}$$

4. Numerical Simulations

Through MATLAB R2019b numerical simulations, this section demonstrates the effectiveness of reinforcement learning for flexible high-rise buildings experiencing time-varying actuator failures and presenting asymmetric states.

Table 1. Physical parameter table of flexible high-rise building system.

Parameter	Value	Unit
$L_f$	0.54	m
$\rho$	0.453	Kg/m
EI	20	Nm2
$J_c$	37	Kgm2
${\mu }_c$	0.65	Kg
${\mu }_f$	0.68	Kg

shows the physical parameters.

The control goal aims to design a control scheme that satisfies $e\to 0,\dot{e}\to 0$ when $t\to \infty$. The system initial values are $x_c\left(t\right)=0$, $\iota \left(L_f,t\right)=0.08$, and the system expected values are $x_{cd}\left(t\right)=0.08$, ${\iota }_d\left(L_f,t\right)=0$.

When $t=3 \mathrm{s}$, with sudden time-varying actuator failure, fault selection is as follows:

$$s(t)=20\cos (t){\tau }_1+0.1\sin 0.5$$

(63)

Case 1. No control input. ${\tau }_1=0$.

Case 2. Under PD control. The controller is presented as follows:

$${\tau }_1=-k_{p}e-k_d\dot{e}$$

(64)

where the parameters are given as $k_{p1}=463$ and $k_{v1}=96$.

Case 3. With proposed control (46). Control parameters are selected as follows: $K_{11}=1.2$, $K_{31}=23$, $K_{41}=57$. Asymmetric time-varying state constraints, ${\underset{\_}{k}}_{c_1}<x_c\left(t\right)<{\overline{k}}_{c_1}$, ${\underset{\_}{k}}_{c_2}<T_1\left(t\right)<{\overline{k}}_{c_2}$, ${\underset{\_}{k}}_{c_1}=0{.07\mathrm{e}}^{-0.5t}+0.01$, ${\overline{k}}_{c_1}=-0{.15\mathrm{e}}^{-2t}-0.03$, ${\underset{\_}{k}}_{c_2}=0{.06\mathrm{e}}^{-0.5t}+0.01$, ${\overline{k}}_{c_2}=-0{.12\mathrm{e}}^{-2t}-0.03$. The learning rate of reinforcement learning is ${\sigma }_{1a}=0.01$, ${\sigma }_{1c}=4.8$.

Based on the above three schemes, the control performance is compared by simulation analysis. The simulation results are as follows:

Figure 3 shows the three-dimensional vibration deformation curve of the flexible wall under uncontrolled action. In the above figure, the x axis represents the height of the flexible wall, the y axis represents the simulation time, and the z axis represents the change in the vibration of the flexible wall. It can be seen from the figure that the vibration deformation of the single-story building system cannot converge to 0 without control. Figure 4 shows the three-dimensional vibration deformation curve of the flexible wall under the action of the proposed controller (46). Similarly to the above described without control, the x axis, y axis and z axis represent the height of the flexible wall, simulation time and vibration change in the flexible wall, respectively. It can be seen from Figure 4 that sudden failure occurs at 3 s, the vibration of the flexible wall tends to be stable at 5 s, and the vibration is effectively suppressed.

From the comparison curves in Figure 5 and Figure 6, it can be seen that under the same initial conditions, both the floor slab vibration and the tracking error of the active mass damper are effectively suppressed and tracked within 5 s when the proposed controller (Case 3) is employed. Specifically, the proposed controller is able to rapidly reduce the vibration amplitude of the floor slab and maintain a small tracking error, which indicates its strong performance in dynamic control. In contrast, although the PD control scheme (Case 2) is also able to suppress the vibration, the effect is obviously inferior to that of the proposed controller, especially in terms of the accuracy and response speed. When a sudden time-varying fault occurs in the system at the moment t = 3, the proposed control scheme is able to cope with the fault more efficiently, keeping the vibration and error low, whereas the PD control scheme fails to deal with such a sudden change well, resulting in a poorer performance of the system. This phenomenon indicates that the proposed control scheme is more robust and adaptive in coping with time-varying faults.

Figure 7 demonstrates the comparison of the vibration acceleration of the floor slab, which further validates the advantage of the proposed control scheme in suppressing the vibration acceleration. By comparing with the PD control scheme, it is found that the proposed controller can not only effectively suppress the vibration amplitude of the floor slab, but can also significantly reduce the vibration acceleration to avoid structural damage caused by excessive acceleration. Finally, Figure 8 shows the torque comparison curves under the control of Case 2 and 3. From the figure, it can be seen that after a sudden time-varying fault occurs at the moment t = 3, the proposed controller is able to effectively compensate for the impact of the fault through fault-tolerant control to ensure the stable operation of the system, while the PD control scheme fails to provide a similar fault-tolerant function, resulting in unstable torque changes. Therefore, the proposed control scheme shows better performance in vibration suppression, acceleration control and fault tolerance, and is more adaptable to maintaining system stability and safety under complex and sudden fault conditions.

5. Conclusions

Focusing on the vibration suppression of high-rise building systems under extreme conditions, this paper investigates a reinforcement learning vibration control strategy for a class of flexible building systems with time-varying faults and asymmetric state constraints. Firstly, a mathematical model that accurately describes the dynamic characteristics of flexible high-rise buildings is established, taking into account the time-varying nature of actuator faults. Secondly, a reinforcement learning-based controller is designed to compensate for the impact of faults on system performance. By introducing the time-varying asymmetric Lyapunov function, the system state constraint is guaranteed to ensure the stability and security of the system. The stability of the closed-loop system is strictly proven through Lyapunov stability theory, ensuring that the system can maintain stable vibration suppression performance even in the presence of faults. Finally, the simulation results show that the proposed reinforcement learning vibration control method can effectively suppress the vibration response of flexible high-rise buildings in the face of time-varying actuator faults, demonstrating strong robustness and adaptability, and providing a new and effective approach for solving vibration control problems in actual flexible high-rise buildings.

However, this study has its limitations. The model developed may simplify certain complex practical applications. For example, it may not be able to fully consider the intricate interactions between building structures and foundation soils under extreme conditions, which may affect the practical application of control strategies. In addition, the validation is based on a simulation only. Factors such as unforeseen environmental perturbations, sensor errors, and the complex behavior of building materials under long-term stresses are not considered in practical applications, which poses a challenge to the direct implementation of the proposed method.

For future research, the existing research will be expanded. In terms of theoretical research, we will combine big data analysis and artificial intelligence algorithms to further optimize the reinforcement learning model, mine more potential system characteristics and control laws, and achieve more accurate prediction and response to complex and changing faults and environmental factors. In terms of experimental validation, we will build a simulation platform more similar to the actual situation, simulate different vibration environments and different building types of high-rise building conditions, and accumulate a large amount of experimental data to provide solid support for the theoretical research.

6. Patents

This paper mainly focuses on the vibration suppression problem of flexible high-rise building systems, considers the time-varying fault and asymmetric constraint problem, and also adopts the control strategy based on reinforcement learning to control the system vibration.