ADP-Based Fault-Tolerant Control with Stability Guarantee for Nonlinear Systems

Abstract

This paper develops the stability-guaranteed adaptive dynamic programming (ADP)-based fault tolerant control (FTC) for nonlinear systems with an actuator fault. Firstly, a fault observer is designed to identify the unknown actuator fault. Then, a critic neural network (NN) is built to approximate the optimal control of the nominal system. Meanwhile, a stability-aware weight update mechanism is proposed based on the Lyapunov stability theorem to relax the restriction of the initial admissible control on the system stability. By integrating the nominal optimal control and the fault estimation, the stability-guaranteed ADP-based FTC is developed to eliminate the influence of the actuator fault. Furthermore, the observer errors, critic NN weight estimation errors, and the closed-loop system are all shown to exhibit uniform ultimate boundedness by using Lyapunov’s direct method. Finally, simulation examples are given to demonstrate the validity of the proposed method.

1. Introduction

The increasing scale and complexity of modern industrial systems pose significant challenges to optimal controller design. Adaptive dynamic programming (ADP) has emerged as an effective tool of optimal control, particularly for high-dimensional nonlinear systems where exact mathematical models are unavailable. ADP derives the optimal control by integrating the technique of reinforcement learning (RL), dynamic programming, and neural networks (NNs). With years of developments, ADP has found widespread application in complex nonlinear systems, including robust control [1], fault tolerant control (FTC) [2], zero-sum games [3], actuator saturation [4], and so on.

Value iteration (VI) and policy iteration (PI) are two basic algorithms of ADP. Based on VI, the value function converges to the optima monotonically without requiring an initial admissible policy (IAP). The convergence analysis of continuous VI was presented in [5]. It was proved that the value function initialized with an arbitrary positive semidefinite function can converge to a neighborhood of the optimum. Ha et al. [6] improved the VI performance by developing a novel adaptive critic design with a discount factor, which accelerates the learning formulation and relaxes conditions on the system dynamics. However, VI cannot guarantee system stability during the learning process. In contrast, PI starts with the IAP and alternates between policy evaluation and improvement while ensuring stability at each iteration. Thus, PI is generally considered more suitable for practical applications. Nonetheless, one notable limitation of PI is its reliance on an IAP, which is difficult to obtain. To address this problem, Duan et al. [7] proposed a PI-based actor–critic approach with stability guarantees for nonlinear systems, where a warm-up phase was introduced to guide the initial policy toward admissibility. Combining the advantages of VI and PI, Luo et al. [8] developed a novel ADP method by adding a parameter to balance VI and PI, which not only accelerates VI but also eliminates the need of an IAP. In [9], Guo et al. studied the event-triggered tracking control utilizing ADP with a single critic NN, where the dependence on the IAP is removed by refining the weight update mechanism.

Due to the increasing complexity of industrial systems, faults in control components are a frequent occurrence, with actuator faults being one of the most common types. Such faults can lead to a significant degradation of system reliability, safety, and stability. In response to the severe challenges posed by actuator faults, FTC has emerged as a critical technology for ensuring system safety and enhancing reliability. A variety of classical FTC strategies have been established, such as observer-based methods [10,11,12], sliding mode control (ISMC) [13,14,15], and fuzzy logic control (FLC) [16,17,18]. Recently, increasing attention has been given to ADP-based FTC methods. A number of studies have achieved success by integrating ADP with observer-based techniques. Liu [19] employs an NN-based fault observer for online fault estimation and compensation. A neuro-dynamic programming-based FTC scheme is proposed in [20], which combines a state observer with a fault observer to simultaneously estimate the state and multiple faults. This observer-based paradigm has also been extended to handle more demanding dynamics, such as state time delays [21,22] and complex multi-agent interactions [23,24]. However, their reliance on model knowledge for observer design limits their application to systems with unknown dynamics. To address this limitation, model-free approaches have become a major research focus. Lin et al. [25] proposed a data-based FTC framework which uses the particle swarm optimization algorithm to optimize the NN identifier and the critic NN for unknown nonlinear systems. More recently, fully RL methods have been developed to learn the control policy directly from system data. For instance, Zhang et al. [26] utilizes a model-free, off-policy RL algorithm to learn and deploy a differential-game-based $H_{\infty }$ fault-tolerant controller. This controller enables effective compensation for actuator faults while mitigating external disturbances. Ref. [27] integrates the reference trajectory and system state. The authors’ method learns the optimal control policy using both offline pre-training and an online actor–critic framework, which enables optimal fault-tolerant tracking control for nonlinear systems. A notable challenge in these online learning schemes is the stringent persistent excitation (PE) condition. Wu et al. [28] developed a distributed tracking control scheme based on event-triggered ADP. This scheme introduces an event-triggered mechanism to save computation and communication resources while relaxing the PE condition. These model-free strategies have also been adapted to manage complex scenarios involving multi-fault handling and input constraints [29].

It is worth mentioning that many existing ADP-based FTC methods are developed based on the PI algorithm, which completely relies on an IAP policy. However, as is mentioned above, it is extremely difficult to obtain an IAP theoretically, which limits the application of the ADP-based FTC algorithms. To address this challenge, this paper proposes a stability-guaranteed ADP-based FTC strategy that eliminates the need for the admissible policy. The key contributions of this work are described as follows:

(1)

A stability-aware weight update mechanism with an auxiliary stabilizing term is proposed to evaluate real-time system stability and correct the learning process of the critic NN, which ensures the system stability in the absence of the initial stabilizing control.

(2)

By designing a fault observer, the unknown actuator fault is estimated accurately and compensated based on the nominal optimal control, thereby eliminating the influence of the actuator fault.

2. Problem Statement and Preliminaries

2.1. Problem Formulation

Consider the following continuous-time nonlinear system with the actuator fault:

$$\dot{x}\left(t\right)=f\left(x\left(t\right)\right)+g\left(x\left(t\right)\right)\left(u\left(x\left(t\right)\right)-f_a\left(t\right)\right),$$

(1)

where $x\in {\mathbb{R}}^n$ is the system state vector, $u\in {\mathbb{R}}^m$ is the control input vector, and $f_a\left(t\right)\in {\mathbb{R}}^m$ denotes the unknown actuator fault. The function $f\left(x\right)\in {\mathbb{R}}^n$ is a nonlinear function satisfying $f\left(0\right)=0$. $g\left(x\right)\in {\mathbb{R}}^{n\times m}$ is the input gain function. The initial state is given by $x\left(0\right)=x_0$.

Assumption 1.

The actuator fault $f_a\left(t\right)$ is unknown but bounded by a positive constant ${\delta }_1$, i.e., $\parallel f_a\left(t\right)\parallel \le {\delta }_1$ [30].

The objective of this paper is to design the FTC policy $u\left(x\right(t\left)\right)$ for the system (1) subject to the unknown but bounded actuator fault $f_a\left(t\right)$ satisfying Assumption 1. While actuator faults share the input channel $g\left(x\right)$ with matched disturbances, they are fundamentally distinct from typical external disturbances. First, actuator faults represent an internal malfunction that directly compromises the intended execution and effectiveness of the control signal, rather than being merely an additive exogenous input. This distinction necessitates the explicit fault identification and compensatory control strategies to maintain stability and recover performance. Second, actuator faults originate from random internal component failures within the system. This makes their occurrence difficult to predict and poses a far greater threat to system stability and safety than conventional disturbances.

2.2. Nominal Optimal Control

We first consider the nominal optimal control problem for the system in the absence of the fault, i.e., where $f_a\left(t\right)=0$. The nominal system dynamics become

$$\dot{x}\left(t\right)=f\left(x\left(t\right)\right)+g\left(x\left(t\right)\right)u\left(x\left(t\right)\right).$$

(2)

The performance index for the nominal system (2) is defined as

$$J\left(x_0\right)={\int }_0^{\infty }U(x\left(\tau \right),u\left(\tau \right))d\tau ,$$

(3)

where the utility function is defined by $U(x,u)=x^{\top }Qx+u^{\top }Ru$, with $Q\in {\mathbb{R}}^{n\times n}$ and $R\in {\mathbb{R}}^{m\times m}$ being constant symmetric positive-definite matrices.

Definition 1.

For the nominal system (2) (i.e., system (1) with $f_a\left(t\right)=0$), a control policy $u\left(x\right)$ is continuous on a compact set $\Omega \subset {\mathbb{R}}^n$, satisfies $u\left(0\right)=0$, stabilizes the system (2), and results in a finite performance index $J\left(x_0\right)$ as defined in (3) for all $x_0\in \Omega$, then the control policy $u\left(x\right)$ is said to be admissible.

Let $\Psi (\Omega )$ denote the set of admissible policies on $\Omega$. If $J\left(x\right)$ is continuously differentiable, the nonlinear Lyapunov equation is

$$0=U(x,u\left(x\right))+\nabla V{\left(x\right)}^{\top }\left(f\left(x\right)+g\left(x\right)u\left(x\right)\right),$$

(4)

where $V\left(0\right)=0$, and $\nabla V\left(x\right)={\displaystyle \frac{\partial V\left(x\right)}{\partial x}}$ denotes the gradient of $V\left(x\right)$ with respect to x.

The Hamiltonian function is given by

$$H(x,u,\nabla V\left(x\right))=U(x,u)+\nabla V{\left(x\right)}^{\top }\left(f\left(x\right)+g\left(x\right)u\right).$$

(5)

Using this definition, the Lyapunov Equation (4) can be simply expressed as

$$H(x,u(x),\nabla V(x\left)\right)=0.$$

(6)

The optimal control problem aims to find the control policy $u^{*}\left(x\right)\in \Psi (\Omega )$ that minimizes the performance index (3). The optimal performance index is defined as

$$J^{*}\left(x\right)=\underset{u\in \Psi (\Omega )}{min}{\int }_0^{\infty }U(x\left(\tau \right),u\left(x\left(\tau \right)\right))d\tau ,$$

(7)

and the corresponding HJB equation is

$$0=\underset{u\in \Psi (\Omega )}{min}\left\{U(x,u\left(x\right))+\nabla J^{*}{\left(x\right)}^{\top }\left(f\left(x\right)+g\left(x\right)u\left(x\right)\right)\right\},$$

(8)

where $\nabla J^{*}\left(x\right)=\partial J^{*}\left(x\right)/\partial x$. The HJB equation can also be written using the Hamiltonian as ${min}_{u\in \Psi (\Omega )}H(x,u\left(x\right),\nabla J^{*}\left(x\right))=0$, where $\nabla J^{*}\left(x\right)$ now takes the role of $\nabla V\left(x\right)$ in the Hamiltonian definition (5). By solving the HJB equation, an optimal control policy is obtained:

$$u^{*}\left(x\right)=-{\displaystyle \frac{1}{2}}{R}^{-1}{g}^{\top }\left(x\right)\nabla J^{*}\left(x\right).$$

(9)

The optimal control $u^{*}\left(x\right)$ given by (9) and its corresponding optimal value function $J^{*}\left(x\right)$ are typically found using iterative methods. PI is a widely used ADP approach that effectively solves the HJB equation. PI conventionally alternates between two main steps: policy evaluation, in which the value function for the current control policy is determined by solving the Bellman Equation (4), and policy improvement, where the control policy is updated using the gradient of this value function to better approximate $u^{*}\left(x\right)$. However, a significant challenge in conventional PI is its requirement for an initial admissible control policy to begin the iteration. This limitation motivates the development of techniques that can operate without a predefined initial admissible control.

From the optimal control law expression (9), we can derive a useful relationship:

$${\left(\nabla J^{*}\left(x\right)\right)}^{\top }g\left(x\right)=-2{\left(u^{*}\left(x\right)\right)}^{\top }R.$$

(10)

3. Stability-Guaranteed ADP-Based FTC Without IAP

3.1. Fault Observer Design

For the faulty system described in (1), we design the fault observer as

$$\dot{\widehat{x}}=\widehat{f}\left(x\right)+\widehat{g}\left(x\right)\left(u-{\widehat{f}}_a\right)+L_1(x-\widehat{x}),$$

(11)

where $\widehat{x}\in {\mathbb{R}}^n$ denotes the estimated system state, $L_1\in {\mathbb{R}}^{n\times n}$ is a positive-definite observer gain matrix, and ${\widehat{f}}_a\in {\mathbb{R}}^m$ is the estimated actuator fault. The fault estimate ${\widehat{f}}_a$ is updated using the following adaptive law:

$${\dot{\widehat{f}}}_a=-L_2{\widehat{g}}^{\top }\left(x\right)e_o,$$

(12)

where $L_2\in {\mathbb{R}}^{m\times m}$ is a positive-definite gain matrix, and $e_o=x-\widehat{x}$ denotes the observation error. According to the system dynamics (2) and the observer (11), the error dynamics are derived as

$$\begin{array}{cc}\hfill {\dot{e}}_o& =f\left(x\right)+g\left(x\right)\left(u-f_a\right)-\widehat{f}\left(x\right)-\widehat{g}\left(x\right)\left(u-{\widehat{f}}_a\right)-L_1{e}_o\hfill \\ \hfill & =\tilde{f}+\tilde{g}u-g\left(x\right)f_a+\widehat{g}\left(x\right){\widehat{f}}_a-L_1{e}_o,\hfill \end{array}$$

(13)

where $\tilde{f}=f\left(x\right)-\widehat{f}\left(x\right)$ and $\tilde{g}=g\left(x\right)-\widehat{g}\left(x\right)$ represent the observation errors of the nonlinear components $f\left(x\right)$ and $g\left(x\right)$, respectively.

Define $\varsigma =\tilde{f}+\tilde{g}(u-f_a)$; the error dynamics in (13) is expressed as

$${\dot{e}}_o=\varsigma -\widehat{g}\left(x\right)(f_a-{\widehat{f}}_a)-L_1{e}_o.$$

Assumption 2.

ς is norm-bounded, i.e., $\parallel \varsigma \parallel \le {\varsigma }_M$, where ${\varsigma }_M$ is a positive constant [30].

Theorem 1.

For the faulty system (1) with Assumptions 1 and 2, the fault observation error is uniformly ultimately bounded (UUB) using the fault observer (11) and the adaptive update law (12).

Proof. 

Select the Lyapunov function candidate as

$${\Sigma }_1={\displaystyle \frac{1}{2}}{e}_o^{\top }{e}_o+{\displaystyle \frac{1}{2}}{\tilde{f}}_a^{\top }{L}_2^{-1}{\tilde{f}}_a,$$

(14)

where ${\tilde{f}}_a=f_a-{\widehat{f}}_a$ is the estimation error of the actuator fault. Taking the time derivative of (14) and substituting (13) yields

$$\begin{array}{cc}\hfill {\dot{\Sigma }}_1& =e_o^{\top }{\dot{e}}_o-{\dot{\widehat{f}}}_a^{\top }{L}_2^{-1}{\tilde{f}}_a\hfill \\ \hfill & =e_o^{\top }\left(\tilde{f}+\tilde{g}(u-f_a)-\widehat{g}\left(x\right)(f_a-{\widehat{f}}_a)-L_1{e}_o\right)-{\dot{\widehat{f}}}_a^{\top }{L}_2^{-1}{\tilde{f}}_a\hfill \\ \hfill & \le {\varsigma }_M\parallel e_o\parallel -e_o^{\top }\widehat{g}\left(x\right){\tilde{f}}_a-{\lambda }_{min}\left(L_1\right){\parallel e_o\parallel }^2-{\dot{\widehat{f}}}_a^{\top }{L}_2^{-1}{\tilde{f}}_a\hfill \\ \hfill & =-\left({\lambda }_{min}\left(L_1\right)\parallel e_o\parallel -{\varsigma }_M\right)\parallel e_o\parallel -\left(e_o^{\top }\widehat{g}\left(x\right)+{\dot{\widehat{f}}}_a^{\top }{L}_2^{-1}\right){\tilde{f}}_a\hfill \end{array}$$

Substituting the adaptive law (12) into the above yields

$${\dot{\Sigma }}_1=-\left({\lambda }_{min}\left(L_1\right)\parallel e_o\parallel -{\varsigma }_M\right)\parallel e_o\parallel .$$

(15)

This implies that we have ${\dot{\Sigma }}_1<0$ as long as $e_o$ lies outside the compact set ${\displaystyle \parallel }{e}_o\parallel \le {\displaystyle \frac{{\varsigma }_M}{{\lambda }_{min}\left(L_1\right)}}$. According to the Lyapunov stability theorem, the fault observation error is UUB. The proof is thus complete. □

Remark 1.

The observer gains $L_1$ and $L_2$, as well as the disturbance bound ${\varsigma }_M$, directly affect the convergence rate and ultimate bound of the estimation error. A larger $L_1$ improves convergence speed, while $L_2$ regulates the responsiveness of fault estimation. Appropriate tuning is essential to balance estimation accuracy and robustness.

3.2. Nominal Optimal Control via Critic Neural Network

NNs are known to be universal approximators for nonlinear functions. In this work, the critic NN is utilized to approximate the optimal value function $J^{*}\left(x\right)$ (7), which is the solution to the HJB Equation (8) and is typically unknown, nonlinear, and non-analytic. The optimal value function $J^{*}\left(x\right)$ can be represented by an NN as

$$J^{*}\left(x\right)=W_c^{\top }\sigma \left(x\right)+{\epsilon }_c\left(x\right),$$

(16)

where $W_c\in {\mathbb{R}}^l$ is the ideal weight vector, $\sigma \left(x\right)\in {\mathbb{R}}^l$ is the vector of activation functions, l is the number of neurons in the hidden layer, and ${\epsilon }_c\left(x\right)$ is the NN approximation error. The gradient of the optimal value function is then

$$\nabla J^{*}\left(x\right)={(\nabla \sigma \left(x\right))}^{\top }{W}_c+\nabla {\epsilon }_c\left(x\right),$$

(17)

where $\nabla \sigma \left(x\right)=\partial \sigma \left(x\right)/\partial x\in {\mathbb{R}}^{l\times n}$. When substituting the NN approximation (17) into (8), the resulting expression can be separated into terms involving ${(\nabla \sigma \left(x\right))}^{\top }{W}_c$ and terms involving the approximation error gradient $\nabla {\epsilon }_c\left(x\right)$. We define $e_{cH}$ as the sum of all terms that include ${\epsilon }_c\left(x\right)$ or its gradient $\nabla {\epsilon }_c\left(x\right)$, such that the HJB equation is satisfied by setting the sum of the principal part (formed with ${(\nabla \sigma \left(x\right))}^{\top }{W}_c$) and $e_{cH}$ to zero. Specifically, $e_{cH}$ is derived as

$$\begin{array}{cc}\hfill e_{cH}& ={(\nabla {\epsilon }_c\left(x\right))}^{\top }f\left(x\right)-{\displaystyle \frac{1}{2}}{(\nabla {\epsilon }_c\left(x\right))}^{\top }G\left(x\right){(\nabla \sigma \left(x\right))}^{\top }{W}_c\hfill \\ \hfill & -{\displaystyle \frac{1}{4}}{(\nabla {\epsilon }_c\left(x\right))}^{\top }G\left(x\right)\nabla {\epsilon }_c\left(x\right),\hfill \end{array}$$

(18)

where $G\left(x\right)=g\left(x\right)R^{-1}{g}^{\top }\left(x\right)$.

Since the ideal weight vector $W_c$ is unknown, (16) is approximated as

$${\widehat{J}}^{*}\left(x\right)={\widehat{W}}_c^{\top }\sigma \left(x\right),$$

(19)

with gradient

$$\nabla {\widehat{J}}^{*}\left(x\right)={(\nabla \sigma \left(x\right))}^{\top }{\widehat{W}}_c,$$

(20)

where ${\widehat{W}}_c$ is the estimate of $W_c$.

By substituting $\nabla {\widehat{J}}^{*}\left(x\right)$, the approximate Hamiltonian error is defined as

$$\begin{array}{cc}\hfill e_c& =x^{\top }Qx+{{\displaystyle (}{(\nabla \sigma \left(x\right))}^{\top }{\widehat{W}}_c{\displaystyle )}}^{\top }f\left(x\right)\hfill \\ \hfill & -{\displaystyle \frac{1}{4}}{{\displaystyle (}{(\nabla \sigma \left(x\right))}^{\top }{\widehat{W}}_c{\displaystyle )}}^{\top }G\left(x\right){\displaystyle (}{(\nabla \sigma \left(x\right))}^{\top }{\widehat{W}}_c{\displaystyle )}.\hfill \end{array}$$

(21)

Let ${\tilde{W}}_c=W_c-{\widehat{W}}_c$ be the weight estimation error. The Hamiltonian error $e_c$ can be expressed in terms of ${\tilde{W}}_c$, $W_c$, and $e_{cH}$ as

$$\begin{array}{cc}\hfill e_c& =-{\tilde{W}}_c^{\top }\left(\nabla \sigma \left(x\right)f\left(x\right)-{\displaystyle \frac{1}{2}}\nabla \sigma \left(x\right)G\left(x\right){(\nabla \sigma \left(x\right))}^{\top }{W}_c\right)\hfill \\ \hfill & -{\displaystyle \frac{1}{4}}{\tilde{W}}_c^{\top }\nabla \sigma \left(x\right)G\left(x\right){(\nabla \sigma \left(x\right))}^{\top }{\tilde{W}}_c-e_{cH}.\hfill \end{array}$$

(22)

To update the critic weight vector ${\widehat{W}}_c$, the gradient descent algorithm is applied to minimize the objective function:

$$E_c={\displaystyle \frac{1}{2}}{e}_c^{\top }{e}_c.$$

(23)

The gradient of $E_c$ with respect to ${\widehat{W}}_c$ is given by

$${\displaystyle \frac{\partial E_c}{\partial {\widehat{W}}_c}}=e_c\left(\nabla \sigma \left(x\right)f\left(x\right)-{\displaystyle \frac{1}{2}}\nabla \sigma \left(x\right)G\left(x\right){\displaystyle (}{{\displaystyle (}\nabla \sigma \left(x\right))}^{\top }{\widehat{W}}_c{\displaystyle )}\right).$$

(24)

Based on (9) and (17), the optimal control policy is obtained by

$$u^{*}\left(x\right)=-{\displaystyle \frac{1}{2}}{R}^{-1}{g}^{\top }\left(x\right)\left({(\nabla \sigma \left(x\right))}^{\top }{W}_c+\nabla {\epsilon }_c\left(x\right)\right).$$

(25)

Using the estimated critic weights ${\widehat{W}}_c$, the approximated optimal control is given by

$$\widehat{u}\left(x\right)=-{\displaystyle \frac{1}{2}}{R}^{-1}{g}^{\top }\left(x\right){(\nabla \sigma \left(x\right))}^{\top }{\widehat{W}}_c.$$

(26)

Applying the approximate control policy $\widehat{u}\left(x\right)$ to the nominal system (2) yields the closed-loop dynamics

$$\dot{x}=f\left(x\right)-{\displaystyle \frac{1}{2}}g\left(x\right)R^{-1}{g}^{\top }\left(x\right){(\nabla \sigma \left(x\right))}^{\top }{\widehat{W}}_c.$$

(27)

3.3. Stability-Aware Weight Update Mechanism

The conventional critic weight update rule is typically derived under the assumption of an admissible initial control policy. Without a stabilizing initial policy, the learning process may fail to ensure system stability. To address this challenge, we introduce an auxiliary term related to system stability into the update law of the critic network. The enhanced critic weight update law is given by

$${\dot{\widehat{W}}}_c=-{\alpha }_c{\displaystyle \frac{\partial E_c}{\partial {\widehat{W}}_c}}+{\displaystyle \frac{1}{2}}{\alpha }_s\Pi (x,\widehat{u}\left(x\right))\nabla \sigma \left(x\right)G\left(x\right)\nabla J_s\left(x\right).$$

(28)

where ${\alpha }_c$ and ${\alpha }_s$ are learning rates. The binary function $\Pi (x,\widehat{u}\left(x\right))\in \{0,1\}$ is defined as

$$\Pi (x,\widehat{u}\left(x\right))=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}\nabla J_s{\left(x\right)}^{\top }(f\left(x\right)+g\left(x\right)\widehat{u}\left(x\right))<0\hfill \\ 1,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(29)

Assumption 3.

A continuously differentiable Lyapunov function $J_s\left(x\right)$ is selected such that

$${\dot{J}}_s\left(x\right)={(\nabla J_s\left(x\right))}^{\top }\dot{x}={(\nabla J_s\left(x\right))}^{\top }(f\left(x\right)+g\left(x\right)u^{*})<0.$$

(30)

Moreover, there exists a positive definite matrix $\Lambda \left(x\right)$ such that

$${(\nabla J_s\left(x\right))}^{\top }(f\left(x\right)+g\left(x\right)u^{*})=-{(\nabla J_s\left(x\right))}^{\top }\Lambda \left(x\right)\nabla J_s\left(x\right).$$

(31)

Furthermore, there exists a positive constant ${\lambda }_s>0$ satisfying

$$0<{\lambda }_s\parallel \nabla J_s\left(x\right)\parallel \le -{(\nabla J_s\left(x\right))}^{\top }\dot{x}.$$

(32)

Remark 2.

Assumption 3 [31] is commonly used for establishing the stability of the closed-loop system under the optimal control $u^{*}$. It relies on the premise that the closed-loop dynamics, $f\left(x\right)+g\left(x\right)u^{*}$, are suitably bounded. Specifically, for some positive constant $\eta >0$, it is often assumed that

$$\parallel f\left(x\right)+g\left(x\right)u^{*}\parallel \le \eta \parallel \nabla J_s\left(x\right)\parallel .$$

(33)

From this, using the Cauchy–Schwarz inequality, we have

$$\parallel {(\nabla J_s\left(x\right))}^{\top }(f\left(x\right)+g\left(x\right)u^{*})\parallel \le \parallel \nabla J_s\left(x\right)\parallel \parallel f\left(x\right)+g\left(x\right)u^{*}\parallel \le \eta \parallel \nabla J_s{\left(x\right)\parallel }^2.$$

The specific structure (31) provides a concrete way to ensure ${\dot{J}}_s\left(x\right)<0$ (for $\nabla J_s\left(x\right)\ne 0$) and is consistent with this derived upper bound. Such an assumption is generally considered reasonable, and $J_s\left(x\right)$ is often chosen as a quadratic function in practice.

The auxiliary stabilizing term provides real-time feedback on the stabilizing capability of the current policy based on the Lyapunov stability theorem and adjusts the gradient direction accordingly. It is worth noting that the binary function $\Pi (x,\widehat{u}\left(x\right))$ is defined based on Lyapunov stability conditions. When the nonlinear system is stable—i.e., when the time derivative of the Lyapunov function satisfies $\nabla J_s{\left(x\right)}^{\top }\left(f\left(x\right)+g\left(x\right)\widehat{u}\right)<0$—the auxiliary term is inactive, and the critic update is driven purely by minimizing the approximation error. In contrast, when the system tends to diverge, the auxiliary term is activated to redirect the learning process toward a stabilizing control policy. This modification provides a structural enhancement to conventional critic learning, enabling the critic to converge eventually without requiring any admissible initial controller. It significantly reduces sensitivity to initial conditions and improves the algorithm’s practicality in real-time control scenarios. The theoretical validity of this mechanism will be formally established in the next section using Lyapunov-based analysis.

Remark 3.

In ADP, PI typically relies on an IAP to maintain closed-loop stability, whereas VI is more flexible but often lacks a verifiable stability guarantee during purely online learning. To address this issue, we introduce in (28) a Lyapunov-criterion-based stability switch $\Pi (x,\widehat{u}\left(x\right))$. When the update direction may cause ${\dot{J}}_s\left(x\right)\ge 0$, the switch automatically activates an auxiliary update, which integrates HJB optimization and the stability requirement during learning within a single, unified framework.

Within the assumptions adopted in this paper and over the working domain $\Omega$, the design enables online optimization without an IAP and provides a proven safety guarantee; Section 3.4 establishes UUB. Compared with multi-stage or hybrid schemes [7,8], the proposed method embeds learning-phase stability directly into the update law, thereby delivering a verifiably safeguarded online learning process. This capability remains relatively uncommon among existing IAP-free ADP frameworks and is well aligned with the real-time control demands of safety-critical systems.

3.4. Fault Compensation Design

Based on the analysis in Section 3.1, we further propose a fault compensation control structure to mitigate the influence of the actuator fault. To this end, we propose the following fault compensation control law:

$$u_a\left(x\right)=\widehat{u}\left(x\right)+{\widehat{f}}_a.$$

(34)

Remark 4.

The proposed control scheme belongs to the category of active FTC. By incorporating a dynamically estimated fault compensation term, the controller can adapt to real-time fault variations and maintain fault-tolerant performance.

The structural diagram of the observer-based ADP scheme with the auxiliary stabilizing term for FTC is shown in Figure 1.

3.5. Stability Analysis

Assumption 4.

There exist positive constants ${\lambda }_{\varphi },{\lambda }_A,{\lambda }_1,{\lambda }_e,{\lambda }_G,{\lambda }_4,$ and ${\lambda }_6$ such that for all $x\in \Omega$, $\parallel \varphi \left(x\right)\parallel \le {\lambda }_{\varphi }$, $\parallel A\left(x\right)\parallel \le {\lambda }_A$, $\parallel A{W}_c\parallel \le {\lambda }_1$, $\parallel e_{cH}\parallel \le {\lambda }_e$, $\parallel G\left(x\right)\parallel \le {\lambda }_G$, $\parallel \nabla \sigma \left(x\right)\parallel \le {\lambda }_4$, and $\parallel \nabla {\epsilon }_c\left(x\right)\parallel \le {\lambda }_6$ [31].

Theorem 2.

Consider the system described by Equation (2). If the feedback control law is implemented as (26), and the critic network weights are updated according to the learning rule (28), then the closed-loop system state x and the weight estimation error ${\tilde{W}}_c$ are UUB.

Proof. 

To establish the stability analysis, we define a Lyapunov candidate function as

$$L={\displaystyle \frac{1}{2{\alpha }_c}}{\tilde{W}}_c^{\top }{\tilde{W}}_c+{\displaystyle \frac{{\alpha }_s}{2{\alpha }_c}}{J}_s\left(x\right).$$

(35)

Taking the time derivative of L, we obtain

$$\dot{L}={\displaystyle \frac{1}{{\alpha }_c}}{\tilde{W}}_c^{\top }{\dot{\tilde{W}}}_c+{\displaystyle \frac{{\alpha }_s}{2{\alpha }_c}}{(\nabla J_s\left(x\right))}^{\top }\dot{x}.$$

(36)

Substituting the critic weight update law, the dynamics of the weight estimation error are given by

$${\dot{\tilde{W}}}_c={\alpha }_c{\displaystyle \frac{\partial E_c}{\partial {\widehat{W}}_c}}-{\displaystyle \frac{1}{2}}{\alpha }_s\Pi (x,\widehat{u}\left(x\right))\nabla \sigma \left(x\right)g\left(x\right)R^{-1}{g}^{\top }\left(x\right)\nabla J_s\left(x\right).$$

(37)

Substituting (37) and (27) into (36) yields

$$\begin{array}{cc}\hfill \dot{L}=& -\left[{\tilde{W}}_c^{\top }\varphi \left(x\right)+{\displaystyle \frac{1}{4}}{\tilde{W}}_c^{\top }A\left(x\right){\tilde{W}}_c-{\displaystyle \frac{1}{2}}{\tilde{W}}_c^{\top }A\left(x\right)W_c+e_{cH}\right]\times \hfill \\ \hfill & \left[{\tilde{W}}_c^{\top }\varphi \left(x\right)+{\displaystyle \frac{1}{2}}{\tilde{W}}_c^{\top }A\left(x\right){\tilde{W}}_c-{\displaystyle \frac{1}{2}}{\tilde{W}}_c^{\top }A\left(x\right)W_c\right]\hfill \\ \hfill & -{\displaystyle \frac{{\alpha }_s}{2{\alpha }_c}}\Pi (x,\widehat{u}\left(x\right)){\tilde{W}}_c^{\top }\nabla \sigma \left(x\right)g\left(x\right)R^{-1}{g}^{\top }\left(x\right)\nabla J_s\left(x\right)\hfill \\ \hfill & +{\displaystyle \frac{{\alpha }_s}{2{\alpha }_c}}{(\nabla J_s\left(x\right))}^{\top }\dot{x},\hfill \end{array}$$

(38)

where

$$\begin{array}{cc}\hfill A\left(x\right)& =\nabla \sigma \left(x\right)g\left(x\right)R^{-1}{g}^{\top }\left(x\right){(\nabla \sigma \left(x\right))}^{\top },\hfill \\ \hfill \varphi \left(x\right)& =\nabla \sigma \left(x\right)f\left(x\right).\hfill \end{array}$$

Based on Assumption 4, we have

$$\begin{array}{cc}\hfill \dot{L}\le & -\left({\displaystyle \frac{1}{8}}-{\displaystyle \frac{3}{8}}{\varphi }_1^2-{\displaystyle \frac{3}{16}}{\varphi }_2^2\right){\lambda }_A^2{\tilde{W}}_c^4\hfill \\ \hfill & +\left\{{\displaystyle \frac{1}{2}}{\lambda }_A{\lambda }_e+\left(1+{\displaystyle \frac{3}{8{\varphi }_1^2}}\right){\lambda }_{\varphi }^2+\left({\displaystyle \frac{3}{4}}+{\displaystyle \frac{3}{16{\varphi }_2^2}}\right){\lambda }_1^2\right\}{∥\begin{array}{c}{\tilde{W}}_c\end{array}∥}^2\hfill \\ \hfill & +{\displaystyle \frac{3}{4}}{\lambda }_e^2-{\displaystyle \frac{{\alpha }_s}{2{\alpha }_c}}\Pi \left(x,\widehat{u}\right){\tilde{W}}_c^{\top }\nabla \sigma \left(x\right)g\left(x\right)R^{-1}{g}^{\top }\left(x\right)\nabla J_s\left(x\right)\hfill \\ \hfill & +{\displaystyle \frac{{\alpha }_s}{2{\alpha }_c}}{\left(\nabla J_s\left(x\right)\right)}^{\top }\dot{x}.\hfill \end{array}$$

(39)

□

Case 1: When $\Pi (x,\widehat{u})=0$, ${\left(\nabla J_s\left(x\right)\right)}^{\top }\dot{x}<0$. According to Assumption 3, there exists a positive constant ${\lambda }_s$ such that the inequality $0<{\lambda }_s\parallel \nabla J_s\left(x\right)\parallel \le -{(\nabla J_s\left(x\right))}^{\top }\dot{x}$ holds. Thus, (39) can then be expressed as

$$\begin{array}{cc}\hfill \dot{L}\le & -{\lambda }_2{∥{\tilde{W}}_c∥}^4+{\lambda }_3{∥{\tilde{W}}_c∥}^2\hfill \\ \hfill & +{\displaystyle \frac{3}{4}}{\lambda }_e^2-{\displaystyle \frac{{\alpha }_s}{{2\alpha }_c}}{\lambda }_s\Vert \nabla J_s\left(x\right)\Vert ,\hfill \end{array}$$

(40)

where

$$\begin{array}{cc}\hfill {\lambda }_2& =\left({\displaystyle \frac{1}{8}}-{\displaystyle \frac{3}{8}}{\varphi }_1^2-{\displaystyle \frac{3}{16}}{\varphi }_2^2\right){\lambda }_A^2,\hfill \\ \hfill {\lambda }_3& ={\displaystyle \frac{1}{2}}{\lambda }_A{\lambda }_e+\left(1+{\displaystyle \frac{3}{8{\varphi }_1^2}}\right){\lambda }_{\varphi }^2+\left({\displaystyle \frac{3}{4}}+{\displaystyle \frac{3}{16{\varphi }_2^2}}\right){\lambda }_1^2.\hfill \end{array}$$

Therefore, it follows that $\dot{L}<0$ when either of the following conditions is satisfied:

$$∥{\tilde{W}}_c∥\ge \sqrt{{\displaystyle \frac{{\lambda }_3+\sqrt{3{\lambda }_e^2{\lambda }_2+{\lambda }_3^2}}{2{\lambda }_2}}}\triangleq {\mathcal{A}}_1$$

(41)

or

$$\parallel \nabla J_s\left(x\right)\parallel \ge {\displaystyle \frac{1}{{\lambda }_s}}\left({\displaystyle \frac{2{\alpha }_c}{{\alpha }_s}}\right)\left({\displaystyle \frac{{\lambda }_3^2}{4{\lambda }_2}}+{\displaystyle \frac{3}{4}}{\lambda }_e^2\right)\triangleq {\mathcal{B}}_1.$$

(42)

Case 2: When $\Pi (x,\widehat{u}\left(x\right))=1$, the auxiliary term is active. According to (26) and (25), the difference between the optimal control and its estimate is given by

$$u^{*}\left(x\right)-\widehat{u}\left(x\right)=-{\displaystyle \frac{1}{2}}{R}^{-1}{g}^{\top }\left(x\right)\left({\left(\nabla \sigma \left(x\right)\right)}^{\top }{\tilde{W}}_c+\nabla {\epsilon }_c\left(x\right)\right).$$

(43)

Following the previous derivation, we have $\dot{x}=f\left(x\right)+g\left(x\right)\widehat{u}\left(x\right)=f\left(x\right)+g\left(x\right)u^{*}\left(x\right)+g\left(x\right)(\widehat{u}\left(x\right)-u^{*}\left(x\right))$. Substituting the control error term (43) into the expression of $\dot{x}$ yields

$$\dot{x}=f\left(x\right)+g\left(x\right)\left(u^{*}+{\displaystyle \frac{1}{2}}{R}^{-1}{g}^{\top }\left(x\right)\left({\left(\nabla \sigma \left(x\right)\right)}^{\top }{\tilde{W}}_c+\nabla {\epsilon }_c\left(x\right)\right)\right).$$

(44)

Therefore, the Lyapunov function derivative now becomes

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \dot{L}⩽& -\left({\displaystyle \frac{1}{8}}-{\displaystyle \frac{3}{8}}{\varphi }_1^2-{\displaystyle \frac{3}{16}}{\varphi }_2^2\right){\lambda }_A^2{∥{\tilde{W}}_c∥}^4\hfill \\ & +\left({\displaystyle \frac{1}{2}}{\lambda }_A{\lambda }_e+\left(1+{\displaystyle \frac{3}{8{\varphi }_1^2}}\right){\lambda }_{\varphi }^2+\left({\displaystyle \frac{3}{4}}+{\displaystyle \frac{3}{16{\varphi }_2^2}}\right){\lambda }_1^2\right){∥{\tilde{W}}_c∥}^2\hfill \\ & +{\displaystyle \frac{3}{4}}{\lambda }_e^2+{\displaystyle \frac{{\alpha }_s}{2{\alpha }_c}}{\left(\nabla J_s\left(x\right)\right)}^{\top }\left(f\left(x\right)+g\left(x\right)u^{*}\right)\hfill \\ & -{\displaystyle \frac{{\alpha }_s}{4{\alpha }_c}}{\tilde{W}}_c^{\top }\nabla \sigma \left(x\right)g\left(x\right)R^{-1}{g}^{\top }\left(x\right)\nabla J_s\left(x\right)\hfill \\ & +{\displaystyle \frac{{\alpha }_s}{4{\alpha }_c}}{\left(\nabla J_s\left(x\right)\right)}^{\top }g\left(x\right)R^{-1}{g}^{\top }\left(x\right)\nabla {\epsilon }_c\left(x\right).\hfill \end{array}\end{array}$$

(45)

Substituting (31) from Assumption 3 into (45), and applying the Cauchy–Schwarz and Young’s inequalities to the penultimate term of the above equation yields

$$\begin{array}{cc}\hfill -{\displaystyle \frac{{\alpha }_s}{4{\alpha }_c}}{\tilde{W}}_c^{\top }\nabla \sigma \left(x\right)g\left(x\right)R^{-1}{g}^{\top }\left(x\right)\nabla J_s\left(x\right)& \le {\displaystyle \frac{{\alpha }_s}{8{\alpha }_c}}\parallel {\tilde{W}}_c{\parallel }^2+{\displaystyle \frac{{\alpha }_s{\lambda }_4^2{\lambda }_G^2}{8{\alpha }_c}}{\parallel \nabla J_s\left(x\right)\parallel }^2,\hfill \end{array}$$

Thus, we have

$$\begin{array}{cc}\hfill \dot{L}\le & -{\lambda }_2\parallel {\tilde{W}}_c{\parallel }^4+\left({\lambda }_3+{\displaystyle \frac{{\alpha }_s}{8{\alpha }_c}}\right){\parallel {\tilde{W}}_c\parallel }^2\hfill \\ \hfill & +{\displaystyle \frac{3}{4}}{\lambda }_e^2-{\lambda }_5{\parallel \nabla J_s\left(x\right)\parallel }^2\hfill \\ \hfill & +{\displaystyle \frac{{\alpha }_s}{4{\alpha }_c}}{\lambda }_G{\lambda }_6\parallel \nabla J_s\left(x\right)\parallel ,\hfill \end{array}$$

(46)

where ${\lambda }_5={\displaystyle \frac{{\alpha }_s}{2{\alpha }_c}}\left({\lambda }_{min}(\Lambda )-{\displaystyle \frac{1}{4}}{\lambda }_G^2{\lambda }_4^2\right)$.

Therefore, it follows that $\dot{L}<0$ if either

$$∥{\tilde{W}}_c∥⩾\sqrt{{\displaystyle \frac{8{\alpha }_c{\lambda }_3+{\alpha }_s}{16{\alpha }_c{\lambda }_2}}+\sqrt{{\displaystyle \frac{{\left(8{\alpha }_c{\lambda }_3+{\alpha }_s\right)}^2}{256{\alpha }_c^2{\lambda }_2^2}}+{\displaystyle \frac{3{\lambda }_e^2}{4{\lambda }_2}}+{\displaystyle \frac{{\alpha }_s^2{\lambda }_G^2{\lambda }_6^2}{64{\alpha }_c^2{\lambda }_2{\lambda }_5}}}}\triangleq {\mathcal{A}}_2$$

(47)

or

$$∥\begin{array}{c}\nabla J_s\left(x\right)\end{array}∥\ge {\displaystyle \frac{{\alpha }_s{\lambda }_G{\lambda }_6}{8{\alpha }_c{\lambda }_5}}+\sqrt{{\displaystyle \frac{{\left(8{\alpha }_c{\lambda }_3+{\alpha }_s\right)}^2}{256{\alpha }_c^2{\lambda }_2{\lambda }_5}}+{\displaystyle \frac{3{\lambda }_e^2}{4{\lambda }_5}}+{\displaystyle \frac{{\alpha }_s^2{\lambda }_G^2{\lambda }_6^2}{64{\alpha }_c^2{\lambda }_5^2}}}\triangleq {\mathcal{B}}_2.$$

(48)

By combining the two cases above, we have $\dot{L}<0$ if the following condition holds:

$$∥{\tilde{W}}_c∥>max\left({\mathcal{A}}_1,{\mathcal{A}}_2\right)\mathrm{or}∥\nabla J_s\left(x\right)∥>max\left({\mathcal{B}}_1,{\mathcal{B}}_2\right).$$

(49)

According to the standard Lyapunov stability theorem, it follows that both the system state x and the critic weight estimation error ${\tilde{W}}_c$ are UUB. This concludes the proof.

4. Simulation

This section demonstrates the effectiveness of the proposed stability-guaranteed ADP-based FTC through two simulation examples.

4.1. Example 1

Consider the following nonlinear affine system:

$$\dot{x}=\left[\begin{array}{c}{x}_2-x_1\\ \begin{array}{c}-0.5x_1-0.5x_2+\\ 0.5x_2{(cos\left(2x_1\right)+2)}^2\end{array}\end{array}\right]+\left[\begin{array}{c}0\\ cos\left(2x_1\right)+2\end{array}\right](u\left(t\right)-f_a\left(t\right)),$$

(50)

with the initial state $x\left(0\right)={[0.5,-0.5]}^{\mathsf{T}}$. $f_a\left(t\right)$ represents the unknown actuator fault, which is defined as

$$f_a\left(t\right)=\left\{\begin{array}{cc}0.2cos\left({\displaystyle \frac{t}{2\pi }}\right),\hfill & 30\mathrm{s}\le t\le 60\mathrm{s}\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(51)

The parameters of the performance index function are selected as $Q=2I_2$ and $R=1$, respectively, where $I_2$ is the $2\times 2$ identity matrix. The initial state of the observer is set to $\widehat{x}\left(0\right)={[0.5,-0.5]}^{\mathsf{T}}$, and the initial value of the fault estimation is set to ${\widehat{f}}_a\left(0\right)=0$. The observer gain is chosen as $L_1=43I_2$ and the learning rate of the fault observer is set to $L_2=100$. A critic NN is constructed to approximate the value function. The weight vector is ${\widehat{W}}_c={[{\widehat{W}}_{c1},{\widehat{W}}_{c2},{\widehat{W}}_{c3}]}^{\mathsf{T}}$ with initial weights ${\widehat{W}}_c\left(0\right)={[0.7,0.9,0.6]}^{\mathsf{T}}$, and the activation function of the critic NN is $\sigma \left(x\right)=[x_1^2,x_1{x}_2,x_2^2]$. The learning rate for the critic network is set to ${\alpha }_c=1$. The parameter of the auxiliary stabilizing term is set as ${\alpha }_s=1$. The total simulation duration was 100 s, and the actuator fault $f_a\left(t\right)$ affects the system during the interval $t\in [30\mathrm{s},60\mathrm{s}]$.

Simulation results are shown in Figure 2, Figure 3, Figure 4 and Figure 5. The state trajectory performance is shown in Figure 2a. It shows that the states converge within 5 s. Figure 2b shows the time response curves of the observer estimation errors ${\tilde{x}}_1$ and ${\tilde{x}}_2$. As illustrated in Figure 2b, the state observation errors converge to a small neighborhood around the origin. Figure 3a presents the curves of the actual actuator fault $f_a\left(t\right)$ and the fault estimation ${\widehat{f}}_a\left(t\right)$. The results indicate that the designed fault observer can accurately identify the unknown fault signal rapidly after fault occurrence. Figure 3b shows the control signals of the nominal optimal control (26) and the ADP-based FTC policy (34). It can be observed that after the fault occurs, the ADP-based FTC policy adjusts accordingly. Figure 4a shows the evolution process of the critic NN weights ${\widehat{W}}_{c1}\left(t\right)$, ${\widehat{W}}_{c2}\left(t\right)$, and ${\widehat{W}}_{c3}\left(t\right)$, which converge to [1.14, 0.36, 0.88]. Figure 4b further shows the working mechanism of the auxiliary stabilizing term, including the stability condition ${\dot{J}}_s\left(t\right)$ and the stability indicator $\Pi \left(t\right)$. It can be seen that the stability indicator $\Pi \left(t\right)$ is frequently activated in the early learning phase and eventually converges to 0.

To further verify the effectiveness of the proposed auxiliary stabilizing term, a comparative analysis is performed against the conventional ADP method presented in [30]. The update law in [30] relies on a standard gradient descent approach, which typically requires an initial admissible policy to ensure stability. By removing our auxiliary stabilizing term, we effectively simulate this conventional approach under conditions where the initial policy may not be stabilizing. Figure 5a,b show the divergent critic NN weights and system states without the auxiliary stabilizing term, which indicates that the initial control fails to stabilize the system. Fortunately, with the auxiliary stabilizing term, the training process of the weight vector is reinforced until the system exhibits stable behavior (see Figure 4a,b). Thus, the requirement of an initial stabilizing control is relaxed.

4.2. Example 2

In this case, we consider a nonlinear mass-spring-damper system, with dynamics described as follows:

$$\dot{x}=\left[\begin{array}{c}{x}_2\\ -0.02x_1-0.67x_1^3-0.1x_2^3\end{array}\right]+\left[\begin{array}{c}0\\ 1\end{array}\right](u\left(t\right)-f_a\left(t\right)),$$

(52)

with the initial state $x\left(0\right)={[0.5,-0.5]}^{\mathsf{T}}$. The control input $u\left(t\right)$ is defined as in Example 1. In this case, the actuator fault $f_a\left(t\right)$ is chosen as a different model, specifically as follows:

$$f_a\left(t\right)=\left\{\begin{array}{cc}0.1sin\left({\displaystyle \frac{1}{\pi }}t\right),\hfill & 40\mathrm{s}\le t\le 70\mathrm{s}\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(53)

The observer gain is chosen as $L_1=5I_2$ and the learning rate of the fault observer is set to $L_2=100$. The weight vector is ${\widehat{W}}_c={[{\widehat{W}}_{c1},{\widehat{W}}_{c2},{\widehat{W}}_{c3}]}^{\mathsf{T}}$ with initial weights selected as ${\widehat{W}}_c\left(0\right)={[0.03,0.2,0.04]}^{\mathsf{T}}$. The learning rate is ${\alpha }_c=3$. The total simulation duration was still 100 s, with the actuator fault $f_a\left(t\right)$ affecting the system during the interval $t\in [40\mathrm{s},70\mathrm{s}]$. Other parameters and initial conditions are chosen the same as Example 1.

Simulation results for Example 2 are shown in Figure 6, Figure 7, Figure 8 and Figure 9. Figure 6a shows the trajectories of the stable system states. Figure 6b shows the convergent observer errors. Figure 7a presents the curves of the actual actuator fault and its estimation. It can be observed that the fault observer can estimate the actuator fault, verifying the effectiveness of the fault observer. Figure 7b demonstrates the nominal control and ADP-based FTC signals. Figure 8a shows the trajectories of the critic weights converging to [0.44, 0.54, 1.04]. Figure 8b illustrates the working condition of the auxiliary stabilizing term. The stability indicator is mainly triggered in the early stages and eventually converges to 0. Figure 9a,b show that without the auxiliary stabilizing term, the critic NN weights and system trajectories diverge, indicating that the control policy fail to stabilize the system. Figure 6a and Figure 8a together with Figure 9a,b verify that the stability-aware Weight update mechanism can effectively overcome the dependency on an initial admissible control under different models.

5. Conclusions

This paper presents a stability-guaranteed ADP-based FTC scheme for nonlinear systems with the actuator fault, which eliminates the dependency on an initial admissible policy. This is primarily achieved by embedding a Lyapunov-based stability condition directly into the critic network’s learning process. In parallel, the actuator fault is estimated by an observer, and compensated based on the nominal optimal control, which enhances the system’s resilience. We have rigorously proven the UUB stability of all signals, including the closed-loop system states, the estimation error of critic NN weights and the observer error, and demonstrated the scheme’s effectiveness through simulations. Future research will focus on extending this framework to more complex scenarios. Key directions include designing observers for intermittent faults, developing data-driven strategies for systems with unknown dynamics, and applying the method to distributed networked architectures to address challenges such as asynchronous group consensus [32], containment control with deferred constraints [33], and resilience against cyber-attacks via event-triggered control [34].