Robust ADP-Based Sliding-Mode Fault-Tolerant Control for Nonlinear Systems with Application to Spacecraft

Abstract

This paper considers a novel fault-tolerant control (FTC) scheme for a category of cascade nonlinear systems with mismatched uncertainties and unknown actuator faults. The robust adaptive dynamic programming (RADP) is used to design a novel optimal sliding surface (SS) off-line, which renders the corresponding sliding-mode dynamics able to obtain robustness of stability to mismatched uncertainties. Subsequently, a simple sliding-mode control (SMC) with the adaptive fault compensation is developed to guarantee reachability of the sliding mode. Then, it is proven that the weight errors of neural networks (NN) of RADP and the closed-loop system are stable based on Lyapunov stability theory. In the simulation section, this proposed scheme is used to deal with the attitude FTC of a spacecraft, and simulation results verify the effectiveness of the proposed novel control scheme.

1. Introduction

In areas with high safety requirements such as spacecraft, the fault-tolerant control (FTC) has always attracted the attention [1,2] of many scholars and engineers. In [1], a novel discontinuous adaptive failure compensation controller with system chattering rejection based on a common approximate function was designed for a rigid satellite with actuator failure. In [2], an ADP-based tracking FTC with an adaptive fault compensation was given for a spacecraft attitude system with additive faults. In the FTC community, a common and very important method is the sliding-mode control (SMC) that was widely applied in industrial and academic fields due to the extremely simple design and strong anti-disturbance ability [3]. The equivalent control of SMC can cope with matched uncertainties inherently, but for mismatched uncertainties, including faults or failures, it is an open problem [4,5]. The SMC with a new sliding surface including the information of mismatched uncertainties through a disturbance observer was designed in [4]. The invariant manifold-based SS was applied to transform the multiple mismatched disturbances into matched ones in [5]. It has always been very interesting to find more and better ways to solve this open problem.

The ADP approaches have developed a relatively complete paradigm for the optimal control [6]. An online adaptive algorithm was proposed in [7], which involves the simultaneous updating of both critic and actor NN. A classical thinking [8] of transformation from robust control into optimal control was used in RADP methods [9] for adaptive control problems subject to matched or mismatched uncertainties. Specifically, in [10], ADP was introduced to integral sliding-mode control (ISMC) for systems with mismatched disturbance, but the SS design was not considered. Hence, combining SMC with ADP to cope with mismatched uncertainties induces our attentions.

The idea of using other control methods to design SS is an optional combination solution, which is an effective way to deal with nonlinear FTC problems. In [11], the classical control-based SS design combined two control algorithms. When the system states are in sliding mode, the classical control is conducted. The method [12] also adopted the above idea and used a pseudo control input in the SS design process. In [13], a novel fast terminal sliding mode control without the reaching phase was designed, which makes the derivative of an augmented state satisfy an optimal cost function. In [14], the optimal SS similar to [15] was given for a linear system. The robustness of SS to mismatched uncertainties were not considered in these two papers [13,14]. In references [16,17,18,19], optimal SSs were developed based on the first subsystem of the normal form [19]. In [16], an optimal sliding manifold was designed based on kinematics without uncertainties and the angular velocity was seen as the control input in this design process. In [17], optimal design of reaching phase and sliding phase of the SMC was given. In [18], the successive approximation approach was used to solve a novel optimal sliding mode that was designed by the same method. In [19], after an optimal equivalent control was first solved, a novel SS was derived from the above equivalent control. Recently, the ADP algorithm has also been introduced into the integral SMC design [10,20]. In [10], ADP was used to solve the optimal guaranteed cost control problem of sliding mode dynamics. In [20], ADP was employed to design the nominal control, which aimed at stabilizing sliding mode dynamics. However, in these two papers [10,20], ADP was adopted to stabilize sliding mode dynamics rather than to design SS directly.

In this paper, we intend to explore a unifying design of sliding-mode FTC with the RADP-based SS for mismatched uncertain nonlinear cascade systems with actuator faults. To this end, the RADP-based SS is developed to obtain robust sliding mode for mismatched uncertainties, and an adaptive sliding-mode FTC is designed to guarantee the reachability of above sliding mode in the presence of faults. The detailed contributions are summarized as follows.

• The RADP-based SS is designed, which is robust to the mismatched uncertainties. An adaptive fault compensation is used to deal with the actuator faults. Compared with the traditional sliding-mode FTC [21], the novel scheme in the present paper achieves a certain sense of optimized performance by introducing the RADP-based SS. To the best knowledge of the authors, the ADP-based SS does not appear in previous reports, and the more important factor of a novel SMC tackling mismatched uncertainties based on this new SS is not reported either.
• The idea of the ADP-based SS design method may be open a door that we can obtain many kinds of sliding modes with various properties inherited from ADP methods, such as finite-horizon optimal SS [22] or safety SS [23]. These provide more methods or possibilities for solving FTC problems.

The rest of this article is organized as follows. Section 2 gives the problem formulation. In Section 3, an adaptive sliding-mode FTC with RADP-based SS for the uncertain cascade systems is developed. The simulation examples are provided in Section 4 followed by the conclusions in Section 5.

2. Problem Formulation

In this section, the system model, the fault model, and the problem formulation are given.

2.1. System Model and Fault Model

Consider a class of the nonlinear cascade systems [24,25] encountering mismatched uncertainties and actuator faults [26,27], and it can be depicted by

$$\begin{array}{cc}\hfill {\dot{x}}_1\left(t\right)=& f_1\left(x_1\left(t\right)\right)+g_1\left(x_1\left(t\right)\right)x_2\left(t\right)+△f\left(x_1\left(t\right)\right),\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill {\dot{x}}_2\left(t\right)=& f_2\left(x\left(t\right)\right)+g_2\left(x\left(t\right)\right)(u\left(t\right)+f_a),\hfill \end{array}$$

(2)

where $x_1\left(t\right)\in {\mathbb{R}}^{n_1}$, $x_2\left(t\right)\in {\mathbb{R}}^{n_2}$ are system states, $u\left(t\right)\in {\mathbb{R}}^m$ is the control input, and $f_i(·)$ and $g_i(·)$ ($i=1,2$) are differentiable in their arguments with $f_i\left(0\right)=0$ ($i=1,2$), $△f\left(x_1\right)$ is the unknown perturbation with $△f\left(0\right)=0$, $f_a$ is the actuator fault vector.

For ease of the description and the following RADP design, nominal systems of above systems (1) and (2) are given as

$$\begin{array}{cc}\hfill & {\dot{x}}_1\left(t\right)=f_1\left(x_1\left(t\right)\right)+g_1\left(x_1\left(t\right)\right)x_2\left(t\right),\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & {\dot{x}}_2\left(t\right)=f_2\left(x\left(t\right)\right)+g_2\left(x\left(t\right)\right)u\left(t\right).\hfill \end{array}$$

(4)

In order to simplify the expression, some function arguments are omitted in the description below. Then, for convenience of later analysis, the following assumptions and definition are given.

Assumption 1.

It is assumed that the control input matrix $g_1\left(x_1\right)$ is bounded i.e., $\Vert g_1\left(x_1\right)\Vert \le {\lambda }_g$, where ${\lambda }_g$ is a positive constant. The $g_2(·)$ is a full row rank matrix and the function $g_2^{+}(·)$ is defined as the Moore–Penrose pseudo-inverse of $g_2(·)$, i.e., $g_2^{+}(·)=g_2^T(·){\left(g_2(·)g_2^T(·)\right)}^{-1}$.

Assumption 2.

It is assumed that the uncertainty is bounded by a function ${\lambda }_f\left(x_1\right)$, i.e., $\parallel △f\left(x_1\right)\parallel \le {\lambda }_f\left(x_1\right)$ with ${\lambda }_f\left(0\right)=0$, and $f_a$ is the unknown constant bias actuator fault vector.

Definition 1

([7]). The equilibrium point denoted $x=0$ of a dynamical system is said to be uniformly ultimately bounded (UUB) if there exists a compact set $\Omega \subset R^n$ so that for all state variable initial value $x_0\in \Omega$ there exists a bound B and a time $T(B,x_0)$ such that $\Vert x-x_0\Vert \le B$ for all $t\ge t_0+T$.

2.2. Control Objective

The purpose of the present paper is to design a novel SS and the corresponding FTC algorithm, which causes the following novel quadratic performance index to be minimized when the closed-loop system is reaching the SS:

$$J\left(x_1\left(t\right)\right)={\int }_t^{\infty }\mathcal{Q}\left(x_1\left(\tau \right)\right)+U(x_1\left(\tau \right),x_2\left(\tau \right))d\tau ,$$

(5)

where $\mathcal{Q}$ is positive semi-definite matrix, which is associated with the dynamical uncertainty, and $U(x_1,x_2)=x_1^{T}Q{x}_1+x_2^{T}R{x}_2$ is a utility function. The overall utility function $\mathcal{Q}\left(x_1\right)+U(x_1,x_2)$ reflects the uncertainty, regulation, and control policy simultaneously. In particular, the state $x_2$ in subsystem (3) is so-called virtual control [18], which is designed based on a nominal subsystem (3).

Remark 1.

The choice of the utility function is a critical step for optimal control problems. In this paper, a robust control problem is transformed into an optimal control problem by choosing a novel utility function as described above. For an optimal control with input constraints, a popular utility function [28] is chosen as

$$U(x,u)=x{\left(t\right)}^{T}Qx\left(t\right)+2{\int }_0^u{\left({\Phi }^{-1}\left(v\right)\right)}^{T}Rdv,$$

where $\Phi \left(v\right)={[\varphi \left(v_1\right)\dots \varphi \left(v_m\right)]}^T$, $v\in {\mathbb{R}}^m$, $\Phi \in {\mathbb{R}}^m$, and $\varphi (·)$ is a bounded one-to-one function that belongs to $C^p(p⩾1)$ and $L_2(\Omega )$. The hyperbolic tangent $\varphi (·)=tanh(·)$ is an example. For an optimal control problem with state constraints, the utility function can be selected as

$$U(x,u)=x{\left(t\right)}^{T}Qx\left(t\right)+V_x+u{\left(t\right)}^{T}Ru\left(t\right),$$

where $V_x$ is a barrier function with respect to state $x\left(t\right)$ [29]. It is not difficult to see from the above examples that the utility function is not only related to the requirements for state and control, but also directly related to the uncertainty and constraints to be dealt with.

Remark 2.

The value function (5) is designed based on the idea that the robust stabilization problem can be transformed into an optimal control of nominal systems [30].

3. Sliding Surface and Control Design

Concretely, for the system (1) and (2), a RADP-based SS is designed as

$$s=x_2+\frac{1}{2}{R}^{-1}{g}_1^T\left(x_1\right){(\nabla {\sigma }_c\left(x_1\right))}^T{\widehat{W}}_c,$$

(6)

where s is the sliding-mode variable for the mismatched uncertain cascade system, the matrix R is a penalty matrix in the utility function, the ${\sigma }_c$ is an activation function vector of critic neural network (CNN) in ADP, the $\nabla {\sigma }_c$ is gradient with respect to $x_1$ denoted as $\nabla {\sigma }_c=\frac{\partial {\sigma }_c}{\partial x_1}$, and ${\widehat{W}}_c$ is an estimation of ideal weight vector in ADP. The following assumption is a standard assumption in ADP-related literatures, such as [2,10,31].

Assumption 3.

The ideal NN weights $W_c$ is bounded, i.e., $\parallel W_c\parallel \le {\lambda }_W$. The NN activation functions and their gradient are bounded, i.e., $\parallel {\sigma }_c\left(x_1\right)\parallel \le {\lambda }_{\sigma }$, and $\parallel \nabla {\sigma }_c\left(x_1\right)\parallel \le {\lambda }_{\nabla \sigma }$. The NN approximation error and its gradient are bounded on the compact set Ω, i.e., $\parallel {\epsilon }_c\left(x_1\right)\parallel \le {\lambda }_{\epsilon }$, and $\parallel \nabla {\epsilon }_c\left(x_1\right)\parallel \le {\lambda }_{\nabla \epsilon }$.

3.1. Optimal Control-Based Sliding Surface Design

In what follows, the $x_2$ is regarded as the control input of subsystem (3) and assumed that it is admissible. For the continuously differentiable cost function $J\left(x_1\right)$, we can obtain the nonlinear Lyapunov equation is

$$\begin{array}{cc}\hfill 0=& \mathcal{Q}\left(x_1\right)+U(x_1,x_2)\hfill \\ & +{(\nabla J\left(x_1\right))}^T(f_1\left(x_1\right)+g_1\left(x_1\right)x_2),\hfill \end{array}$$

(7)

where $J\left(0\right)=0$. The Hamiltonian function of this optimal control problem is

$$\begin{array}{cc}\hfill H(x_1,x_2,\nabla J\left(x_1\right))=& \mathcal{Q}\left(x_1\right)+U(x_1,x_2)\hfill \\ \hfill & +{(\nabla J\left(x_1\right))}^T(f_1\left(x_1\right)\hfill \\ \hfill & +g_1\left(x_1\right)x_2).\hfill \end{array}$$

(8)

Define the optimal cost function (OCF) as

$$\begin{array}{c}\hfill J^{*}\left(x_1\left(t\right)\right)=\underset{x_2\in \mathcal{A}(\Omega )}{min}{\int }_t^{\infty }\mathcal{Q}\left(x_1\left(\tau \right)\right)+U(x_1\left(\tau \right),x_2\left(\tau \right))d\tau ,\end{array}$$

(9)

where $\mathcal{A}(\Omega )$ is the admissible control set on the $\Omega$, and the HJB equation of this system is given as

$$\begin{array}{c}\hfill 0=\underset{x_2\in \mathcal{A}(\Omega )}{min}H(x_1,x_2,\nabla J^{*}\left(x_1\right)).\end{array}$$

(10)

Therefore, by differentiating (10) with respect to $x_2^{*}$, the optimal control policy is

$$\begin{array}{c}\hfill x_2^{*}=-\frac{1}{2}{R}^{-1}{g}_1^T\left(x_1\right)\nabla J^{*}\left(x_1\right).\end{array}$$

(11)

The control policy (11) is unbounded, and if the problem of input constraints is to be studied, an alternative approach is to use a generalized non-quadratic utility function [28]. Substitute the optimal control (11) into the Equation (10) and then the HJB equation can be expressed as

$$\begin{array}{cc}\hfill 0=& \mathcal{Q}\left(x_1\right)+U(x_1,x_2^{*})+{(\nabla J^{*}\left(x_1\right))}^T(f_1\left(x_1\right)+g_1\left(x_1\right)x_2^{*})\hfill \\ \hfill =& \mathcal{Q}\left(x_1\right)+x_1^{T}Q{x}_1+{(\nabla J^{*}\left(x_1\right))}^T{f}_1\left(x_1\right)\hfill \\ \hfill & -\frac{1}{4}{(\nabla J^{*}\left(x_1\right))}^T{g}_1\left(x_1\right)R^{-1}{g}_1^T\left(x_1\right)\nabla J^{*}\left(x_1\right),\hfill \end{array}$$

(12)

which means that $H(x_1,x_2^{*},\nabla J^{*}\left(x_1\right))=0$.

The design of $\mathcal{Q}\left(x_1\right)$ is a key point in the robust stabilization scheme. In what follows, the term $\mathcal{Q}\left(x_1\right)$ is specified as

$$\begin{array}{c}\hfill \mathcal{Q}\left(x_1\right)=\frac{1}{8}{(\nabla J^{*}\left(x_1\right))}^T\nabla J^{*}\left(x_1\right)+2{\lambda }_f^2\left(x_1\right),\end{array}$$

(13)

and $\mathcal{Q}\left(x_1\right)\ge 0$ meets. Based on the help of this form, the following lemma showing stability of the sliding-mode dynamics of the systems (1) and (2) is derived as follows.

Lemma 1.

Consider the nominal subsystem (3) and the cost function (5) with the term (13). It is assumed the HJB Equation (12) is of a solution $J^{*}\left(x_1\right)$, $x_2^{*}$ is given by (11) and the SS designed as $s=x_2-x_2^{*}=0$ exists. According to these conditions, on this SS, the sliding-mode dynamics of this mismatched uncertain nonlinear system described by (1) and (2) is asymptotic stability.

Proof of Lemma 1. 

According to the Theorem 1 of [30], we know that the optimal control policy (11) can ensure asymptotic stability of the uncertain nonlinear subsystem (1). On the SS, there is $x_2=x_2^{*}\left(x_1\right)$ and explicitly the state $x_2$ is also asymptotically convergent to the equilibrium point. □

3.2. RADP-Based Sliding Surface Design

In what follows, the RADP with NN approximation is used to find this SS through obtaining a numerical solution for the above HJB equation, the analytical solution of which is rather difficult to calculate.

Based on the universal approximation property of NN, approximate $J^{*}\left(x_1\right)$ by a NN with only one hidden layer as

$$J^{*}\left(x_1\right)=W_c^T{\sigma }_c\left(x_1\right)+{\epsilon }_c\left(x_1\right),$$

(14)

where $W_c\in {\mathbb{R}}^p$ is the ideal weight, ${\sigma }_c\left(x_1\right)\in {\mathbb{R}}^p$ is the activation function, p is the number of neurons, and ${\epsilon }_c\left(x_1\right)$ is the unknown approximation error of this NN. Next, partial derivative of the OCF is written as

$$\begin{array}{cc}\hfill \nabla J^{*}\left(x_1\right)& =\frac{\partial J^{*}\left(x_1\right)}{\partial x_1}\hfill \\ & ={(\nabla {\sigma }_c\left(x_1\right))}^T{W}_c+\nabla {\epsilon }_c\left(x_1\right).\hfill \end{array}$$

(15)

However, the ideal NN weights are unknown, the estimation of NN weight vector ${\widehat{W}}_c$ approximates the OCF.

$${\widehat{J}}^{*}\left(x_1\right)={\widehat{W}}_c^T{\sigma }_c\left(x_1\right).$$

(16)

The partial derivative of the OCF by approximation is

$$\nabla {\widehat{J}}^{*}\left(x_1\right)=\frac{\partial {\widehat{J}}^{*}\left(x_1\right)}{\partial x_1}={(\nabla {\sigma }_c\left(x_1\right))}^T{\widehat{W}}_c.$$

(17)

From (15), the optimal control can be given as

$$\begin{array}{cc}\hfill x_2^{*}=& -\frac{1}{2}{R}^{-1}{g}_1^T\left(x_1\right)({(\nabla {\sigma }_c\left(x_1\right))}^T{W}_c+\nabla {\epsilon }_c\left(x_1\right)).\hfill \end{array}$$

(18)

From (17), the approximate optimal control policy can be expressed as

$$\begin{array}{c}\hfill {\widehat{x}}_2^{*}=-\frac{1}{2}{R}^{-1}{g}_1^T\left(x_1\right){(\nabla {\sigma }_c\left(x_1\right))}^T{\widehat{W}}_c.\end{array}$$

(19)

For the following derivation, two non-negative matrices are denoted as follows

$$\begin{array}{c}\hfill \mathcal{A}\left(x_1\right)=\nabla {\sigma }_c\left(x_1\right)g_1\left(x_1\right)R^{-1}{g}_1^T\left(x_1\right)({(\nabla {\sigma }_c\left(x_1\right))}^T,\end{array}$$

$$\begin{array}{c}\hfill \mathcal{B}\left(x_1\right)=\nabla {\sigma }_c\left(x_1\right)({(\nabla {\sigma }_c\left(x_1\right))}^T.\end{array}$$

By using the cost function and control policy denoted by the NN, the Hamiltonian function could be given by

$$\begin{array}{cc}\hfill H(x_1,W_c)=& x_1^{T}Q{x}_1+W_c^T\nabla {\sigma }_c\left(x_1\right)f_1\left(x_1\right)\hfill \\ \hfill & -\frac{1}{4}{W}_c^T\mathcal{A}\left(x_1\right)W_c+2{\lambda }_f^2\left(x_1\right)\hfill \\ \hfill & +\frac{1}{8}{W}_c^T\mathcal{B}\left(x_1\right)W_c+e_{cH}=0,\hfill \end{array}$$

(20)

where the term $e_{cH}$ is the residual error of the NN expression.

$$\begin{array}{cc}\hfill e_{cH}=& {(\nabla {\epsilon }_c(x_1))}^T{f}_1(x_1)\hfill \\ \hfill & -\frac{1}{2}{(\nabla {\epsilon }_c(x_1))}^T{g}_1(x_1)R^{-1}{g}_1^T(x_1)({(\nabla {\sigma }_c(x_1))}^T{W}_c\hfill \\ \hfill & -\frac{1}{4}{(\nabla {\epsilon }_c(x_1))}^T{g}_1(x_1)R^{-1}{g}_1^T(x_1)\nabla {\epsilon }_c(x_1)\hfill \\ \hfill & +\frac{1}{4}{(\nabla {\epsilon }_c(x_1))}^T{(\nabla {\sigma }_c(x_1))}^T{W}_c\hfill \\ \hfill & +\frac{1}{8}{(\nabla {\epsilon }_c(x_1))}^T\nabla {\epsilon }_c(x_1).\hfill \end{array}$$

(21)

The approximate Hamiltonian by the estimated weight is described as

$$\begin{array}{cc}\hfill \widehat{H}(x_1,{\widehat{W}}_c)=& x_1^{T}Q{x}_1+{\widehat{W}}_c^T\nabla {\sigma }_c\left(x_1\right)f_1\left(x_1\right)\hfill \\ \hfill & -\frac{1}{4}{\widehat{W}}_c^T\mathcal{A}\left(x_1\right){\widehat{W}}_c\hfill \\ \hfill & +2{\lambda }_f^2\left(x_1\right)+\frac{1}{8}{\widehat{W}}_c^T\mathcal{B}\left(x_1\right){\widehat{W}}_c,\hfill \end{array}$$

(22)

letting $e_c=\widehat{H}(x_1,{\widehat{W}}_c)-H(x_1,W_c)$, and ${\tilde{W}}_c=W_c-{\widehat{W}}_c$. Based on (20) and (22), the $e_c$ with ${\tilde{W}}_c$ is formulated by

$$\begin{array}{cc}\hfill e_c=& \widehat{H}(x_1,{\widehat{W}}_c)-H(x_1,W_c)\hfill \\ \hfill =& -{\tilde{W}}_c^T\nabla {\sigma }_c\left(x_1\right)f_1\left(x_1\right)-\frac{1}{4}{\tilde{W}}_c^T\mathcal{A}\left(x_1\right){\tilde{W}}_c\hfill \\ \hfill & +\frac{1}{2}{\tilde{W}}_c^T\mathcal{A}\left(x_1\right)W_c+\frac{1}{8}{\tilde{W}}_c^T\mathcal{B}\left(x_1\right){\tilde{W}}_c\hfill \\ \hfill & -\frac{1}{4}{\tilde{W}}_c^T\mathcal{B}\left(x_1\right)W_c-e_{cH}.\hfill \end{array}$$

(23)

Now, the next step is to train the CNN and obtain an update law of weights by minimizing the simple cost criterion $E_c=(1/2)e_c^2$. For avoiding the difficulty of finding an admissible initial control policy, the improved weight updating rule [30] is also selected. The following general assumption [2,32] are given before the training process and the stability analysis.

Assumption 4.

Considering the nominal subsystem (3) with the cost function (5) and substituting the optimal feedback control (11) into the subsystem (3). Let $J_s\left(x_1\right)$ be a continuously differentiable function and simultaneously a Lyapunov function candidate. It can be written as

$${\dot{J}}_s\left(x_1\right)={(\nabla J_s\left(x_1\right))}^T(f_1\left(x_1\right)+g_1\left(x_1\right)x_2^{*})<0.$$

(24)

The $\Gamma \in R^{n_1}$ is a positive definite matrix ensuring that

$$\begin{array}{c}\hfill (\nabla J_s\left(x_1\right){)}^T(f_1\left(x_1\right)+g_1\left(x_1\right)x_2^{*})\\ \hspace{1em}\hspace{1em}=-{(\nabla J_s\left(x_1\right))}^T\Gamma \nabla J_s\left(x_1\right)\hfill \\ \hspace{1em}\hspace{1em}\le -{\lambda }_{min}(\Gamma ){\Vert \nabla J_s\left(x_1\right)\Vert }^2,\hfill \end{array}$$

(25)

is true, where ${\lambda }_{min}(\Gamma )$ is the minimal eigenvalue of the matrix Γ.

For implementing this algorithm, $J_s\left(x_1\right)$ can be given by some polynomials of the system states, such as $J_s\left(x_1\right)=0.5x_1^T{x}_1$. Then, a weight update law of the CNN is constructed

$$\begin{array}{cc}\hfill {\dot{\widehat{W}}}_c=& -{\alpha }_c\left(\frac{\partial E_c}{\partial {\widehat{W}}_c}\right)\hfill \\ \hfill & -{\alpha }_s\left(\frac{\partial \left[{(\nabla J_s\left(x_1\right))}^T(f_1\left(x_1\right)+g_1\left(x_1\right){\widehat{x}}_2^{*})\right]}{\partial {\widehat{W}}_c}\right),\hfill \end{array}$$

(26)

where ${\alpha }_c>0$ is the basic update rate of the critic network, and ${\alpha }_s>0$ is the learning rate of the additional stabilization term. Substitute the optimal control (19) into (26) and then the update law is expressed as

$$\begin{array}{cc}\hfill {\dot{\widehat{W}}}_c=& -{\alpha }_c\left(\frac{\partial E_c}{\partial {\widehat{W}}_c}\right)\hfill \\ \hfill & +\frac{1}{2}{\alpha }_s\nabla {\sigma }_c\left(x_1\right)g_1\left(x_1\right)R^{-1}{g}_1^T\left(x_1\right)\nabla J_s\left(x_1\right).\hfill \end{array}$$

(27)

According to the above formula, ${\dot{\tilde{W}}}_c=-{\dot{\widehat{W}}}_c$ and dynamics of the weight estimation error (WEE) could be expressed as

$$\begin{array}{cc}\hfill {\dot{\tilde{W}}}_c=& {\alpha }_c\left(\frac{\partial E_c}{\partial {\widehat{W}}_c}\right)-\frac{1}{2}{\alpha }_s\nabla {\sigma }_c\left(x_1\right)g_1\left(x_1\right)\hfill \\ & \times R^{-1}{g}_1^T\left(x_1\right)\nabla J_s\left(x_1\right),\hfill \end{array}$$

(28)

where

$$\begin{array}{cc}\hfill \frac{\partial E_c}{\partial {\widehat{W}}_c}=& e_c\frac{\partial e_c}{\partial {\widehat{W}}_c}\hfill \\ \hfill =& e_c(\nabla {\sigma }_c\left(x_1\right)f_1\left(x_1\right)-\frac{1}{2}\mathcal{A}\left(x_1\right){\widehat{W}}_c+\frac{1}{4}\mathcal{B}\left(x_1\right){\widehat{W}}_c).\hfill \end{array}$$

Thus, dynamics of WEE could be expressed as

$$\begin{array}{cc}\hfill {\dot{\tilde{W}}}_c=& {\alpha }_c(-{\tilde{W}}_c^T\nabla {\sigma }_c\left(x_1\right)f_1\left(x_1\right)-\frac{1}{4}{\tilde{W}}_c^T\mathcal{A}\left(x_1\right){\tilde{W}}_c\hfill \\ \hfill & +\frac{1}{2}{\tilde{W}}_c^T\mathcal{A}\left(x_1\right)W_c+\frac{1}{8}{\tilde{W}}_c^T\mathcal{B}\left(x_1\right){\tilde{W}}_c\hfill \\ \hfill & -\frac{1}{4}{\tilde{W}}_c^T\mathcal{B}\left(x_1\right)W_c-e_{cH})\hfill \\ \hfill & \times (\nabla {\sigma }_c\left(x_1\right)f_1+\frac{1}{2}\mathcal{A}\left(x_1\right){\tilde{W}}_c\hfill \\ \hfill & -\frac{1}{2}\mathcal{A}\left(x_1\right)W_c-\frac{1}{4}\mathcal{B}\left(x_1\right){\tilde{W}}_c+\frac{1}{4}\mathcal{B}\left(x_1\right)W_c)\hfill \\ \hfill & -\frac{1}{2}{\alpha }_s\nabla {\sigma }_c\left(x_1\right)g_1\left(x_1\right)\hfill \\ & \times R^{-1}{g}_1^T\left(x_1\right)\nabla J_s\left(x_1\right).\hfill \end{array}$$

(29)

According to this design, the following Lemma 2 shows the property of the system on the novel robust SS.

Lemma 2.

Assuming the sliding mode exists, and considering the nominal system described by (3) and (4) under the proposed RADP-based SS (6), the states of system (1) and (2) on this SS and the weight error dynamics are both UUB.

Proof of Lemma 2. 

Based on Theorem 2 of [30], we can know that state $x_1$ on this SS and dynamics of WEE are both UUB. For brevity, the proof process will not be repeated. Additionally, according to the SS $s=0$, state $x_2$ of the system on this SS is also UUB. □

Remark 3.

The use of ADP faces some problems, such as the difficulty of choosing the initial admissible control, how to choose activation functions of the neural network, and the training taking too much time. In order to avoid the impact of these problems, the off-line training method and other empirical methods are used to ensure the real-time performance of the overall control system as much as possible.

3.3. Sliding-Mode Control Design

After this novel SS is obtained through the above off-line design process, an adaptive fault-tolerant sliding-mode control is developed.

Considering the system (1) and (2) and the approximating optimal control policy of (19), the sliding-mode variable can be expressed as

$$s=x_2-{\widehat{x}}_2^{*}\left(x_1\right).$$

(30)

Note that the control ${\widehat{x}}_2^{*}\left(x_1\right)$ is differential according to (19), and the factory is that ${\widehat{W}}_c$ is a constant vector after convergence. Therefore, the derivative of the sliding-mode variable is written as

$$\begin{array}{cc}\hfill \dot{s}& ={\dot{x}}_2-{\dot{\widehat{x}}}_2^{*}\left(x_1\right)\hfill \\ \hfill & =f_2\left(x\right)+g_2\left(x\right)(u+f_a)-{\dot{\widehat{x}}}_2^{*}\left(x_1\right).\hfill \end{array}$$

(31)

The novel SMC design with the RADP-based SS and the adaptive fault compensation law is designed as

$$\begin{array}{c}\hfill u=g_2^{+}\left(x\right)({\dot{\widehat{x}}}_2^{*}\left(x_1\right)-f_2\left(x\right)-K_s\left(sgn\left(s\right)\right))-{\widehat{f}}_a,\end{array}$$

(32)

and

$$\begin{array}{c}\hfill {\dot{\widehat{f}}}_a=K_f{g}_2^T\left(x\right)s,\end{array}$$

(33)

respectively, where $K_s,K_f>0$ are the control gain and learning rate of the adaptive law and ${\widehat{f}}_a$ is estimation of actuator faults. Define the fault estimation error as ${\tilde{f}}_a=f_a-{\widehat{f}}_a$.

Theorem 1.

Considering the mismatched uncertain and faulty system described by (1) and (2) under the proposed RADP-based sliding-mode variable (6), the control policy (32) and adaptive fault compensation law (33), the fault estimation error is bounded and the closed-loop system dynamics is UUB.

Proof of Theorem 1. 

Choose the following Lyapunov function candidate

$$\begin{array}{c}\hfill V\left(t\right)=\frac{1}{2}{s}^{T}s+\frac{1}{2K_f}{\tilde{f}}_a^T{\tilde{f}}_a.\end{array}$$

(34)

The derivative of the Lyapunov function candidate (34) with respect to time along the sliding mode variable dynamics is

$$\begin{array}{cc}\hfill \dot{V}\left(t\right)& =s^T\dot{s}-\frac{1}{K_f}{\tilde{f}}_a^T{\dot{\widehat{f}}}_a\hfill \\ \hfill & =s^T(f_2\left(x\right)+g_2\left(x\right)(u+f_a)-{\dot{\widehat{x}}}_2^{*}\left(x_1\right))\hfill \\ \hfill & -\frac{1}{K_f}{\tilde{f}}_a^T{\dot{\widehat{f}}}_a.\hfill \end{array}$$

(35)

Substituting the control policy (32) and the actuator fault compensation law (33) into above formula (35), we obtain

$$\begin{array}{cc}\hfill \dot{V}\left(t\right)& =-K_s\parallel s\parallel +s^T{g}_2\left(x\right){\tilde{f}}_a-\frac{1}{K_f}{\tilde{f}}_a^T{\dot{\widehat{f}}}_a\hfill \\ \hfill & =-K_s\parallel s\parallel \hfill \\ \hfill & \le 0.\hfill \end{array}$$

(36)

According to the above derivation process, we could find that all signals of closed-loop system are bounded. Based on the Lyapunov theorem and Lemma 2, the sliding mode exists and the states of the whole system are uniformly ultimately bounded. From the above derivation, it can be seen that the reaching time of the SMC can be estimated by $t_r\le \frac{2V^{1/2}\left(0\right)}{\alpha }$, where $t_r$ is the reaching time, $V\left(0\right)$ is the initial value of the Lyapunov function candidate, and $\alpha$ is a positive design parameter. Therefore, the corresponding time can be reduced by adjusting the parameter $\alpha$. The proof ends here. □

Novelty and comparison: How to make the designed sliding surface (SS) insensitive to the mismatched uncertainties is crucial for the SMC design and the SMC-based FTC.

In [4], a mismatched disturbance is estimated by a disturbance observer, and then the obtained estimation signal of this disturbance is used to design the SS. The disturbance observer makes the system more complex, which increases the difficulty of controller design. However, our designed SS can directly deal with this problem by the optimal control method without designing any estimator. Even though finding the solution of optimal control is tricky, we can solve it directly using the ADP method off-line.

In [10], the integral SMC (ISMC) law is divided into two parts, one is a continuous part, and the other is a discontinuous part. The continuous part of control is used to reject the mismatched uncertainties and consequently the SS is insensitive to the mismatched uncertainties. In this process, the complex zero-sum games and online complex neural network training are used in the continuous part of control, which are not conducive to practical applications and the realization of engineers. Our method is simpler in the design process and implemented off-line.

In the papers [13,14], although they all use the optimal control to design the SS, they cannot solve the problem that is making the SS insensitive to the mismatched uncertainties in the SS design process.

4. Simulation Examples

In this part, the novel sliding-mode FTC with the RADP-based SS is applied in the mismatched uncertain spacecraft attitude control systems (ACS), which are the benchmark nonlinear cascade systems. The model of this ACS [2,33] is presented as

$$\begin{array}{c}\hfill \left[\begin{array}{c}\dot{\vartheta }\\ \dot{\psi }\\ \dot{\gamma }\end{array}\right]=f\left(x_1\right)\omega +△f\left(x_1\right),\end{array}$$

(37)

and

$$J\dot{\omega }=-{\omega }^{\times }J\omega +u\left(t\right)+f_a,$$

(38)

where the inertia matrix J and $f\left(x_1\right)$ are characterized by

$$J=\left[\begin{array}{ccc}20& 0& 0.9\\ 0& 17& 0\\ 0.9& 0& 15\end{array}\right]\mathrm{kg}·{\mathrm{m}}^2,$$

(39)

$$f\left(x_1\right)=\left[\begin{array}{cc}& 0sin\gamma cos\gamma \\ & 0cos\gamma /cos\vartheta -sin\gamma /cos\vartheta \\ & 1-tan\vartheta cos\gamma -tan\vartheta sin\gamma )\end{array}\right],$$

(40)

the mismatched uncertainties $△f\left(x_1\right)$ and the actuator faults $f_a$ are given as

$$△f\left(x_1\right)=\left[\begin{array}{c}0.1\vartheta sin\left(\psi \right)cos\left(\gamma \right)\\ -0.1\gamma cos\left({\psi }^2\right)\\ 0.1\psi sin\left(\vartheta \right)\end{array}\right],$$

(41)

$$f_a=\left\{\begin{array}{cc}\hfill & {[0,0,0]}^T,t<50s\hfill \\ \hfill & {[0,0.5,0.8]}^T,t\ge 50s\hfill \end{array}\right.$$

(42)

furthermore, $\vartheta$, $\psi$, and $\gamma$ are pitch angle, yaw angle, and roll angle of a spacecraft, simply denoted as a vector $x_1={[\vartheta ,\psi ,\gamma ]}^T$, and ${\omega }_x$, ${\omega }_y$, and ${\omega }_z$ are angular velocities with respect to an inertial reference frame expressed in the body-fixed frame. In addition, the uncertainty $△f\left(x_1\right)$ is bounded, i.e., $∥△f\left(x_1\right)∥⩽∥x_1∥$. The initial value of states of this ACS are $x_1\left(0\right)={[0.5,0.5,-0.5]}^{T}rad$, and $\omega \left(0\right)={[0.1,-0.1,0.1]}^{T}rad/s$.

4.1. Traditional SMC-Based Fault-Tolerant Controller

To compare with the proposed method in this paper, a traditional SMC-based FTC is designed as follows

$$u={\left(f\left(x_1\right)J^{-1}\right)}^{-1}(-\dot{f}\left(x_1\right)\omega +f\left(x_1\right)J^{-1}{\omega }^{\times }J\omega -cf\left(x_1\right)\omega -f\left(x_1\right)J^{-1}{\widehat{f}}_a-K_{s}sat\left(s_t\right),$$

(43)

and the adaptive fault compensation is

$${\dot{\widehat{f}}}_a=K_f{J}^{-T}f{\left(x_1\right)}^T{s}_t,$$

(44)

where the designed sliding surface is $s_t=f\left(x_1\right)\omega +c{x}_1$ with $c=0.1$ and let $K_s=2$ and $K_f=100$. The simulation results are given as follows, and comments are given in the next subsection in a comparative manner.

4.2. RADP-Based Sliding-Mode Fault-Tolerant Controller

The total utility function is chosen as $\mathcal{Q}\left(x_1\right)+U(x_1,\omega )=\mathcal{Q}\left(x_1\right)+x_1^{T}Q{x}_1+{\omega }^{T}R\omega$ with $Q=I_3$ and $R=I_3$ for the uncertain ACS. A CNN is constructed to approximate the OCF as

$$\begin{array}{cc}\hfill {\widehat{J}}_n^{*}\left(x_1\right)=& {\widehat{W}}_{c1}{\vartheta }^2+{\widehat{W}}_{c2}{\psi }^2+{\widehat{W}}_{c3}{\gamma }^2+{\widehat{W}}_{c4}\vartheta \psi \hfill \\ \hfill & +{\widehat{W}}_{c5}\vartheta \gamma +{\widehat{W}}_{c6}\vartheta \psi \gamma .\hfill \end{array}$$

Note that ${\sigma }_c\left(a\right)={[{\vartheta }^2,{\psi }^2,{\gamma }^2,\vartheta \psi ,\vartheta \gamma ,\vartheta \psi \gamma ]}^T$ and ${\widehat{W}}_c={[{\widehat{W}}_{c1},{\widehat{W}}_{c2},{\widehat{W}}_{c3},{\widehat{W}}_{c4},{\widehat{W}}_{c5},{\widehat{W}}_{c6}]}^T$ are the activation function and the estimation of the ideal weight of NN, respectively. In these simulations, we set ${\alpha }_c=0.1$ and ${\sigma }_s=0.5$. Substitute the constant weight ${\widehat{W}}_c$ after convergence into the novel SS and let $K_s=2$ and $K_f=100$.

Via Figure 1, Figure 2 and Figure 3, it can be found that, under the basically same parameter settings, the proposed control scheme posses better convergence characteristics and less adjustment time and, through the boundary layer technology, no chattering phenomena. Observe the above simulation results of the proposed novel control scheme and it can be found that the states of the systems are asymptotic stability rather than UUB in Theorem 1. The reason for this behaviour is that $g_2\left(x\right)=J$ is invertible constant matrix and this makes the error $-\frac{1}{2}{R}^{-1}{g}_1^T\left(x_1\right)\nabla {\epsilon }_c\left(x_1\right)$ in (18) matched with $g_2\left(x\right)$. Therefore, if $K_s$ is large enough, the SMC (32) can guarantee its robustness against this error. However, in the SS design process of traditional SMC-based FTC, it is not considered to deal with the mismatched uncertainties, so that system states cannot eventually converge to zero, even if the SMC guarantees reachability of the SS. The evolution of the weight ${\widehat{W}}_c$ is shown in Figure 4, and finally the weight vector eventually converges to ${[0.1370,0.1467,0.1326,0.1307,-0.1204,-0.0786]}^T$.

Figure 1. Evolution of attitude of the ACS under different control schemes in the presence of mismatched uncertainties and actuator faults. (a) Traditional SMC-based FTC. (b) The proposed FTC with RADP-based sliding surface.

Figure 2. Evolution of angular velocity of the ACS under different control schemes in the presence of mismatched uncertainties and actuator faults. (a) Traditional SMC-based FTC. (b) The proposed FTC with RADP-based sliding surface.

Figure 3. Evolution of control inputs of the ACS under different control schemes in the presence of mismatched uncertainties and actuator faults. (a) Traditional SMC-based FTC. (b) The proposed FTC with RADP-based sliding surface.

Figure 4. Evolution of the critic neural network weights.

4.3. RADP-Based Sliding-Mode Fault-Tolerant Controller in Presence of Input Delay

Factors that cause delays are numerous in the ACS, such as the response time of actuators and sensors, faults of actuators and sensors, signal sampling and transmission, etc. The effects will be reflected in the input delay [34]. The existence of the input delay will cause the degradation of control performance and even destroy the stability of the system. To verify whether the proposed controller can cope with the time delay uncertainty, in the above simulation example, an unknown constant time delay is added into the original control system:

$$J\dot{\omega }=-{\omega }^{\times }J\omega +u(t-\tau )+f_a,$$

(45)

where the time delay is set as $\tau =0.1$ and other parameters are the same as the values set above. The simulation results are shown as Figure 5, Figure 6 and Figure 7.

Figure 5. Comparison of attitude of ACS without and with input delay. (a) Evolution of attitude of the ACS. (b) Evolution of attitude with input delay.

Figure 6. Comparison of angular velocity of ACS without and with input delay. (a) Evolution of angular velocity of the ACS. (b) Evolution of attitude with input delay.

Figure 7. Comparison of inputs of ACS without and with input delay. (a) Evolution of control input of the ACS. (b) Evolution of control input with input delay.

By comparing the simulation results before and after adding the input delay, it can be found that the proposed control scheme can make the time delay system bounded, but the control performance has dropped. If a better control performance is pursued, special design for time delay is essential.

5. Conclusions

In the present paper, a novel SMC based FTC scheme with RADP-based SS for a category of uncertain cascade systems is proposed. In particular, a novel idea for the ADP-based SS design is proposed to cope with the mismatched uncertainties in the SMC theory and application. For the practical implementing, the off-line design of RADP-based SS avoids the time-consuming training of neural network weights, and the adaptive SMC is also helpful to implement. In addition, the simulation part demonstrates a practical example that is the attitude control system of the spacecraft. Note that the uncertainty is also mismatched for the subsystem in this paper. Furthermore, matched uncertainties including certain faults for the subsystem are worth researching in the future. In addition, more new sliding surfaces based on the ADP algorithm treasure house could be designed, which can be combined with the powerful SMC mechanism to solve more difficult FTC problems.