Neural Adaptive H_∞ Sliding-Mode Control for Uncertain Nonlinear Systems with Disturbances Using Adaptive Dynamic Programming

Abstract

This paper focuses on a neural adaptive $H_{\infty }$ sliding-mode control scheme for a class of uncertain nonlinear systems subject to external disturbances by the aid of adaptive dynamic programming (ADP). First, by combining the neural network (NN) approximation method with a nonlinear disturbance observer, an enhanced observer framework is developed for estimating the system uncertainties and observing the external disturbances simultaneously. Then, based on the reliable estimations provided by the enhanced observer, an adaptive sliding-mode controller is meticulously designed, which can effectively counteract the effects of the system uncertainties and the separated matched disturbances, even in the absence of prior knowledge regarding their upper bounds. While the remaining unmatched disturbances are attenuated by means of $H_{\infty }$ control performance on the sliding surface. Moreover, a single critic network-based ADP algorithm is employed to learn the cost function related to the Hamilton–Jacobi–Isaacs equation, and thus, the $H_{\infty }$ optimal control is obtained. An updated law for the critic NN is proposed not only to make the Nash equilibrium achieved, but also to stabilize the sliding-mode dynamics without the need for an initial stabilizing control. In addition, we analyze the uniform ultimate boundedness stability of the resultant closed-loop system via Lyapunov’s method. Finally, the effectiveness of the proposed scheme is verified through simulations of a single-link robot arm and a power system.

1. Introduction

Within the last few decades, multifarious robust control design theories and methods have been proposed for uncertain nonlinear systems [1]. As one of the most efficient and widely used control methods, sliding-mode control (SMC) has garnered significant attention by reason of its simplicity, order reduction and inherent robustness against the matched uncertainties [2]. The classical SMC approach is to exert a discontinuous control to drive the system states onto a prescribed sliding manifold or surface [3]. As long as the sliding surface is reached, the system will become immune from the matched uncertainties and input disturbances. To remove the reaching phase, an integral SMC was developed by using the integral sliding manifold, including an integral term, which can enable the system states to reach and remain on the sliding manifold from the beginning [4,5,6]. Although towards a wide variety of actual systems, the relevant uncertainties and disturbances can be assumed to be matched in the design of control systems, there are also many physical systems, such as permanent magnet synchronous motors [7], underactuated aerial vehicles and robotic systems [8] directly affected by unmatched disturbances. Lately, several new approaches involving the integral SMC have been proposed to stabilize various systems with unmatched disturbances [9,10,11,12,13]. Among these methods, it is worth noticing that in [12,13], the impact of the separated unmatched disturbances would not be amplified after choosing a suitable projection matrix in a sliding manifold and were attenuated by the combination of the integral SMC with $H_{\infty }$ control theories. This provides a feasible and effective way to handle the unmatched disturbances and helps explore the relationships between integral SMC and $H_{\infty }$ control in nonlinear system control design.

In many instances, we expect the control policy not just to make the closed-loop system stable, but to possess certain optimality by minimizing the user-defined cost. For nonlinear systems, the settlement of associated optimal control problems requires solving the Hamilton–Jacobi–Bellman (HJB) equation. While considering $H_{\infty }$ optimal control, based on the dissipativity theory, it can be formulated as an $L_2$-gain control problem, which involves solving the Hamilton–Jacobi–Isaacs (HJI) equation [14]. However, the analytical solutions of both HJB and HJI equations are very hard or even impossible to obtain directly because of their inherent nonlinearities [15]. In recent years, a class of neural network (NN) and reinforcement learning (RL)-based intelligent optimization and control methods, referred to as adaptive dynamic programming (ADP), is becoming more and more striking and shows great application potential in solving various optimization problems, and effectively conquers the “curse of dimensionality” [15,16]. By now, many researchers have employed ADP to tackle a variety of optimal control problems for both discrete-time (DT) [17,18,19,20,21,22] and continuous-time (CT) systems [23,24,25,26,27,28]. Moreover, how to combine ADP with other robust methods to achieve better performance and stronger robustness for uncertain nonlinear systems is becoming a new research focus [29,30].

Recently, Modares et al. [31] proposed an online integral RL algorithm that incorporates a non-quadratic discounted cost function to address the constrained-input optimal tracking problem. Luo et al. [32] described an NN-based off-policy learning algorithm within the actor-critic framework to deal with the associated HJI equation, and this algorithm was later extended to find the near-optimal $H_{\infty }$ tracking control solution in [33]. Nevertheless, the influences of potential system or modeling uncertainties were not taken into account in the design. Wang et al. [34] introduced a robust neuro-optimal control approach for input-affine nonlinear systems with both matched and state-depended uncertainties. They achieved this by redesigning the cost function and selecting a suitable feedback gain, whereas the upper bound function of uncertainties is needed for redesigning the cost function to suppress these uncertainties. Mitra et al. [35] presented an optimal SMC scheme for the single-input cascade nonlinear systems with matched bounded disturbances. Fan et al. [36] investigated an adaptive actor–critic-based integral SMC strategy for CT nonlinear systems with unknown terms and input disturbances, where the initial stabilizing control requirement in the learning was quite stringent and limiting in practical applications. Qu et al. [37] developed an adaptive $H_{\infty }$ optimal SMC method in the presence of actuator faults and unmatched disturbances using the ADP algorithm, and further explored the optimal guaranteed cost SMC for constrained-input uncertain systems by formulating an auxiliary system and redefining the utility function [38]. Based on [37], combined with event-triggered mechanisms, Yang et al. [39] provided an event-triggered integral SMC design for nonlinear control-affine systems by leveraging the ADP technique. Note that these methods mentioned above rely on the availability of upper bounds for matched or unmatched disturbances, which may cause over-design and thus leads to an over-conservative control scheme. Additionally, in real-world scenarios, determining precise upper bounds of external disturbances is often a challenging task.

Inspired by the works mentioned earlier, we propose an adaptive neural $H_{\infty }$ SMC scheme for uncertain nonlinear systems subject to external disturbances using the ADP algorithm. Based on the enhanced observer system composed of the NN identifier and nonlinear disturbance observer (DO), an integral SMC is developed to counteract the impacts of the system uncertainties and the separated matched disturbances, as well as unknown approximation errors, without requiring prior knowledge of their upper bounds. While on the sliding manifold, the remaining unmatched disturbances are attenuated by $H_{\infty }$ optimal control solved by the single-network ADP algorithm. Moreover, the uniform ultimate boundedness stability of the resultant closed-loop system are guaranteed via the Lyapunov approach.

The principal contributions of this study can be enumerated as follows. First, unlike other existing schemes [34,35,36,37,38,39], based on the enhanced observer system, the proposed approach makes the designed sliding-mode controller independent from the relevant upper bounds of uncertainties and disturbances, which renders the implementation much easier and more practical and removes the assumption that the upper bounds need to be known in advance. Second, compared with the algorithms presented in [36,37], our approach can deal with both unknown nonlinear terms and unmatched external disturbances, where the single-network ADP is utilized to approximate an $H_{\infty }$ optimal control. Unlike typical actor–critic–disturbance network architectures, the single critic network structure may bring a simpler implementation, lower calculation amount, and avoid the numerical approximate errors arising from actor and disturbance networks. Third, we introduce an updated law for the critic NN, which not only achieves the Nash equilibrium, but also ensures the stability of the sliding-mode dynamics without the need for an initial stabilizing control in the learning.

The remainder of this paper is arranged as follows. Section 2 outlines the problem formulation and provides some necessary preliminaries. Section 3 describes the design of an integral SMC based on the enhanced observer system. Section 4 presents the application of the single-network ADP to obtain $H_{\infty }$ optimal control for the sliding-mode dynamics, along with stability analysis. Simulations of the robotic arm and a power system are given in Section 5, followed by a summary of this study in Section 6.

2. Problem Formulation

Consider the following uncertain perturbed nonlinear system as

$$\dot{x}=f\left(x\right)+\Delta f\left(x\right)+\left(g\left(x\right)+\Delta g\left(x\right)\right)u+d,$$

(1)

where the state vector $x\in {\mathbb{R}}^n$ is measurable, $u\in {\mathbb{R}}^m$ is the control input, $f\left(x\right)\in {\mathbb{R}}^n$ and $g\left(x\right)\in {\mathbb{R}}^{n\times m}$ are the known system drift and input dynamics, respectively; $\Delta f\left(x\right)$ and $\Delta g\left(x\right)$ denote uncertain nonlinear terms that refer to either the inherent characteristics of the system or modeling uncertainties, while $d\in {\mathbb{R}}^n$ represents the unknown external disturbances. Moreover, it is assumed that the system uncertainties $\Delta f\left(x\right)$ and $\Delta g\left(x\right)$ satisfy the matched condition, i.e., $\Delta f\left(x\right)+\Delta g\left(x\right)u=g\left(x\right)w(x,u)$, then the system (1) is rewritten in the form of

$$\dot{x}=f\left(x\right)+g\left(x\right)u+g\left(x\right)w(x,u)+d$$

(2)

with $w(x,u)$ being the bounded lumped uncertain term. Let $\Omega \subseteq {\mathbb{R}}^n$ be a compact set, and suppose that $f\left(x\right)+g\left(x\right)u$ is Lipschitz continuous over $\Omega$ with $f\left(0\right)=0$. Besides, $d\in L_2[0,\infty ]$ and its derivative $\dot{d}$ is bounded such that $\parallel \dot{d}\parallel \le d_{\mathrm{M}}$ with $d_{\mathrm{M}}>0$. To avoid any confusion, $\parallel ·\parallel$ denotes the 2-norm of a vector or the Frobenius norm of a matrix hereafter, unless otherwise specified.

Assumption 1.

The input matrix $g\left(x\right)$ has a full column rank and is norm bounded with $g_{\mathrm{M}}>0$, that is, $\parallel g\left(x\right)\parallel \le g_{\mathrm{M}}$ for any x. Moreover, the resulting left pseudoinverse $g^{+}\left(x\right)\in {\mathbb{R}}^{m\times n}$ is given by $g^{+}\left(x\right)={\left(g^{\mathsf{T}}\left(x\right)g\left(x\right)\right)}^{-1}{g}^{\mathsf{T}}\left(x\right)$, which is bounded by $\parallel g^{+}\left(x\right)\parallel \le b_{\mathrm{M}}$, where $b_{\mathrm{M}}$, $g_{\mathrm{M}}$ are known positive constants.

Based on Assumption 1, d is then decomposed into the matched and unmatched components through the projection of d onto the input matrix $g\left(x\right)$ as

$$d=g\left(x\right)g^{+}\left(x\right)d+(I-g\left(x\right)g^{+}\left(x\right))d,$$

(3)

where I denotes an identity matrix of appropriate dimensions, and $g^{+}\left(x\right)$ is the left pseudoinverse of $g\left(x\right)$. It should be noted that Assumption 1 is somewhat restrictive, which may lessen the applicability scope of the proposed approach to some extent. However, many real-world physical systems, such as the satellite dynamics, the hypersonic flight vehicle and overhead crane systems, have such a property to make this assumption valid [15,20].

To deal with the uncertain nonlinear system (1) with external disturbances, an enhanced observer system is first constructed for estimating the uncertain terms and observing the unknown disturbances simultaneously. Then, based on the reliable estimations, an integral SMC is developed to counteract the impacts of the system uncertainties and the separated matched disturbances, as well as unknown approximation errors, without requiring prior knowledge of their upper bounds. Meanwhile, the remaining unmatched disturbances are attenuated by $H_{\infty }$ optimal control on the sliding surface. Moreover, the single-network ADP algorithm is employed to learn the cost function related to the Hamilton–Jacobi–Isaacs equation, and then, the $H_{\infty }$ optimal control is obtained. What is more, a weight updating law is formulated to ensure both the achievement of Nash equilibrium and the stabilization of sliding-mode dynamics during the learning process.

3. Integral SMC Design Based on the Enhanced Observer System

Recalling the NN universal approximation property, the uncertain term $w(x,u)$ can be represented by a three-layered NN as

$$w(x,u)=W_o^{\mathsf{T}}\sigma \left(V_o^{\mathsf{T}}\overline{x}\right)+{\epsilon }_o\left(x\right),$$

(4)

where $W_o\in {\mathbb{R}}^{l_o\times m}$ and $V_o\in {\mathbb{R}}^{(n+m)\times l_o}$ denote unknown ideal weight matrices between the output and hidden, and hidden and input layers, respectively; $\overline{x}={[x^{\mathsf{T}},u^{\mathsf{T}}]}^{\mathsf{T}}\in {\mathbb{R}}^{n+m}$ is the NN input, $\sigma (·)\in {\mathbb{R}}^{l_o}$ represents the activation function with $l_o$ hidden layer neurons, and ${\epsilon }_o\left(x\right)\in {\mathbb{R}}^m$ stands for the NN reconstruction error. To simplify the learning process, only the weights of $W_o$ are adapted online, while $V_o$ is an initialized set with random values and then remains unchanged during the weight updating process [16].

The NN identifier is designed by

$$\dot{\widehat{x}}=A\widehat{x}+f\left(x\right)-Ax+g\left(x\right)u+g\left(x\right){\widehat{W}}_o^{\mathsf{T}}\sigma \left(z\right)+d,$$

(5)

where A is a Hurwitz matrix, $\widehat{x}$ is the identifier state, ${\widehat{W}}_o$ is the estimate of $W_o$, and the activation function $\sigma \left(z\right)=\sigma \left(V_o^{\mathsf{T}}\overline{x}\right)$ with $z=V_o^{\mathsf{T}}\overline{x}$. Since the unknown disturbance term d is needed in (5), inspired by [11], a nonlinear DO is introduced for obtaining $\widehat{d}$, namely, the estimated value of d.

Then, combining the NN identifier with a nonlinear DO, an enhanced observer system is constructed as

$$\begin{array}{cc}\hfill \dot{\widehat{x}}=& A\widehat{x}+f\left(x\right)-Ax+g\left(x\right)u+g\left(x\right){\widehat{W}}_o^{\mathsf{T}}\sigma \left(z\right)+\widehat{d}\hfill \\ \hfill \dot{d_0}=& -l\left(x\right)(f\left(x\right)+g\left(x\right)u+g\left(x\right){\widehat{W}}_o^{\mathsf{T}}\sigma \left(z\right)+d_0+p\left(x\right)),\hfill \end{array}$$

(6)

with $\widehat{d}=d_0+p\left(x\right)$, where $d_0$ is an auxiliary variable, and $p\left(x\right)$ is a designed state-dependent function and brings out the gain function $l\left(x\right)$ such that $l\left(x\right)={(\partial p\left(x\right)/\partial x)}^{\mathsf{T}}$. Following (6), we have

$$\dot{\widehat{d}}=-l\left(x\right)\widehat{d}+l\left(x\right)d+l\left(x\right)g\left(x\right){\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right)+l\left(x\right)g\left(x\right){\epsilon }_o\left(x\right),$$

(7)

where ${\tilde{W}}_o=W_o-{\widehat{W}}_o$ represents the NN weight estimation error. Let $\tilde{x}=x-\widehat{x}$ and $\tilde{d}=d-\widehat{d}$ be the state and disturbance estimation errors, respectively. Subtracting (5) from (2) and combining with (7), we obtain the coupled error dynamics of (6) as follows:

$$\begin{array}{cc}\hfill \dot{\tilde{x}}& =A\tilde{x}+g\left(x\right){\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right)+\tilde{d}+g\left(x\right){\epsilon }_o\left(x\right),\hfill \\ \hfill \dot{\tilde{d}}& =-l\left(x\right)\tilde{d}-l\left(x\right)g\left(x\right){\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right)+\dot{d}-l\left(x\right)g\left(x\right){\epsilon }_o\left(x\right).\hfill \end{array}$$

(8)

Before proceeding, we introduce a common assumption for stability analysis [15,16].

Assumption 2.

For the identifier NN, there are known positive constants ${\sigma }_{\mathrm{M}}$, ${\epsilon }_{\mathrm{M}}$, $W_{\mathrm{M}}$ and $V_{\mathrm{M}}$ in the sense that $\parallel \sigma \left(z\right)\parallel \le {\sigma }_{\mathrm{M}}$, $\parallel {\epsilon }_o\left(x\right)\parallel \le {\epsilon }_{\mathrm{M}}$, $\parallel W_o\parallel \le W_{\mathrm{M}}$ and $\parallel V_o\parallel \le V_{\mathrm{M}}$, respectively.

Lemma 1.

Considering the system (2) and the coupled error dynamics (8), let the identifier NN weight ${\widehat{W}}_o$ be updated by

$${\dot{\widehat{W}}}_o=-{\eta }_1\sigma \left(z\right){\tilde{x}}^{\mathsf{T}}{A}^{-1}g\left(x\right)-{\eta }_2(\parallel \tilde{x}\parallel +1){\widehat{W}}_o,$$

(9)

where ${\eta }_1$, ${\eta }_2$ are the positive updating ratios. Moreover, we select parameter matrices A, P and gain function $l\left(x\right)$ to satisfy

$$P^{\mathsf{T}}P-l\left(x\right)-l^{\mathsf{T}}\left(x\right)+l\left(x\right)g\left(x\right)g^{\mathsf{T}}\left(x\right)l^{\mathsf{T}}\left(x\right)\le -\rho I$$

(10)

with $\rho >0$. Then all the estimation errors $\tilde{x}$, $\tilde{d}$, and ${\tilde{W}}_0$ are uniformly ultimately bounded (UUB).

Proof. 

Consider the Lyapunov function candidate given by

$$L_1=\frac{1}{2}{\tilde{x}}^{\mathsf{T}}P\tilde{x}+\frac{1}{2}{\tilde{d}}^{\mathsf{T}}\tilde{d}+\frac{1}{2}\mathrm{tr}\left\{{\tilde{W}}_o^{\mathsf{T}}{\tilde{W}}_o\right\},$$

(11)

where $L_{11}={\tilde{x}}^{\mathsf{T}}P\tilde{x}/2+{\tilde{d}}^{\mathsf{T}}\tilde{d}/2$, $L_{12}=\mathrm{tr}\left\{{\tilde{W}}_o^{\mathsf{T}}{\tilde{W}}_o\right\}/2$, and $P=P^{\mathsf{T}}$ is positive definite, which together with some matrix $\Lambda >0$ satisfies $A^{\mathsf{T}}P+PA=-\Lambda$ for the Hurwitz matrix A. By taking the time derivative of $L_{11}$ and substituting the coupled error dynamics (8), we can obtain

$$\begin{array}{cc}\hfill {\dot{L}}_{11}=& \frac{1}{2}{\tilde{x}}^{\mathsf{T}}(A^{\mathsf{T}}P+PA)\tilde{x}+{\tilde{x}}^{\mathsf{T}}P\tilde{d}+{\tilde{x}}^{\mathsf{T}}Pg\left(x\right){\epsilon }_o\left(x\right)+{\tilde{x}}^{\mathsf{T}}Pg\left(x\right){\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right)-{\tilde{d}}^{\mathsf{T}}l\left(x\right)\hfill \\ \hfill & \times g\left(x\right){\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right)-\frac{1}{2}{\tilde{d}}^{\mathsf{T}}\left(l\left(x\right)+l^{\mathsf{T}}\left(x\right)\right)\tilde{d}-{\tilde{d}}^{\mathsf{T}}l\left(x\right)g\left(x\right){\epsilon }_o\left(x\right)+{\tilde{d}}^{\mathsf{T}}\dot{d}.\hfill \end{array}$$

(12)

Based on Assumption 2, together with Young’s inequality, it follows:

$$\begin{array}{cc}\hfill {\dot{L}}_{11}\le & -\frac{1}{2}{\tilde{x}}^{\mathsf{T}}\Lambda \tilde{x}+\frac{1}{2}{\tilde{x}}^{\mathsf{T}}\tilde{x}+\frac{1}{2}{\tilde{d}}^{\mathsf{T}}\left(P^{\mathsf{T}}P-l\left(x\right)-l^{\mathsf{T}}\left(x\right)+l\left(x\right)g\left(x\right)g^{\mathsf{T}}\left(x\right)l^{\mathsf{T}}\left(x\right)\right)\tilde{d}\hfill \\ \hfill & +{\tilde{d}}^{\mathsf{T}}\dot{d}+{\tilde{x}}^{\mathsf{T}}Pg\left(x\right){\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right)+{\tilde{x}}^{\mathsf{T}}Pg\left(x\right){\epsilon }_o\left(x\right)+{\sigma }_{\mathrm{M}}^2{\parallel {\tilde{W}}_o\parallel }^2+{\epsilon }_{\mathrm{M}}^2.\hfill \end{array}$$

(13)

Considering (10), (13) is rewritten as

$$\begin{array}{cc}\hfill {\dot{L}}_{11}\le & -\frac{1}{2}\tau {\tilde{x}}^{\mathsf{T}}\tilde{x}-\frac{1}{2}\rho {\tilde{d}}^{\mathsf{T}}\tilde{d}+{\tilde{x}}^{\mathsf{T}}Pg\left(x\right){\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right)+{\tilde{x}}^{\mathsf{T}}Pg\left(x\right){\epsilon }_o\left(x\right)\hfill \\ \hfill & +{\tilde{d}}^{\mathsf{T}}\dot{d}+{\sigma }_{\mathrm{M}}^2\parallel {\tilde{W}}_o{\parallel }^2+{\epsilon }_{\mathrm{M}}^2,\hfill \end{array}$$

(14)

where $\tau ={\lambda }_{\min }(\Lambda )-1>0$ ensured by properly selecting positive definite matrix $\Lambda$ and its minimum eigenvalue ${\lambda }_{\min }(\Lambda )$.

Combining with (9), ${\dot{L}}_{12}$ is derived as

$$\begin{array}{c}\hfill {\dot{L}}_{12}=\mathrm{tr}\{{\eta }_1{\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right){\tilde{x}}^{\mathsf{T}}{A}^{-1}g\left(x\right)+{\eta }_2{\tilde{W}}_o^{\mathsf{T}}\parallel \tilde{x}\parallel {\widehat{W}}_o+{\eta }_2{\tilde{W}}_o^{\mathsf{T}}{\widehat{W}}_o\}.\end{array}$$

(15)

With the inequality $\mathrm{tr}\{{\tilde{W}}_o^{\mathsf{T}}{\widehat{W}}_o\}\le \parallel W_o{\parallel }^2/2-\parallel {\tilde{W}}_o{\parallel }^2/2$, (15) becomes

$$\begin{array}{c}\hfill {\dot{L}}_{12}\le \mathrm{tr}\{{\eta }_1{\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right){\tilde{x}}^{\mathsf{T}}{A}^{-1}g\left(x\right)\}+\mathrm{tr}\{{\eta }_2{\tilde{W}}_o^{\mathsf{T}}\parallel \tilde{x}\parallel {\widehat{W}}_o\}+\frac{{\eta }_2}{2}\parallel W_o{\parallel }^2-\frac{{\eta }_2}{2}{\parallel {\tilde{W}}_o\parallel }^2.\end{array}$$

Note that the relationship $\mathrm{tr}\left\{A^{\mathsf{T}}B\right\}=B^{\mathsf{T}}A$ for all $A\in {\mathbb{R}}^n$, $B\in {\mathbb{R}}^n$ and the inequality $\mathrm{tr}\{{\tilde{W}}_o^{\mathsf{T}}(W_o-{\tilde{W}}_o)\}\le W_{\mathrm{M}}\parallel {\tilde{W}}_o\parallel -\parallel {\tilde{W}}_o{\parallel }^2$, we can have

$$\begin{array}{cc}\hfill {\dot{L}}_{12}\le & {\eta }_1{\sigma }_{\mathrm{M}}{g}_{\mathrm{M}}\parallel \tilde{x}\parallel \parallel A^{-1}\parallel \parallel {\tilde{W}}_o\parallel +{\eta }_2{W}_{\mathrm{M}}\parallel \tilde{x}\parallel \parallel {\tilde{W}}_o\parallel -{\eta }_2\parallel \tilde{x}\parallel \parallel {\tilde{W}}_o{\parallel }^2+\frac{{\eta }_2}{2}{\parallel W_o\parallel }^2\hfill \\ \hfill & -\frac{{\eta }_2}{2}{\parallel {\tilde{W}}_o\parallel }^2.\hfill \end{array}$$

(16)

By combining (14) and (16) and taking their norms, one can derive an upper bound for ${\dot{L}}_1\left(t\right)$ as

$$\begin{array}{cc}\hfill {\dot{L}}_1\le & -\frac{1}{2}\tau \parallel \tilde{x}{\parallel }^2+(g_{\mathrm{M}}{\epsilon }_{\mathrm{M}}\parallel P\parallel +\left(g_{\mathrm{M}}{\sigma }_{\mathrm{M}}\parallel P\parallel +{\eta }_1{g}_{\mathrm{M}}{\sigma }_{\mathrm{M}}\parallel A^{-1}\parallel +{\eta }_2{W}_{\mathrm{M}}\right)\parallel {\tilde{W}}_o\parallel -{\eta }_2\hfill \\ \hfill & \times \parallel {\tilde{W}}_o{\parallel }^2)\parallel \tilde{x}\parallel -\frac{1}{2}\rho \parallel \tilde{d}{\parallel }^2+d_{\mathrm{M}}\parallel \tilde{d}\parallel +\frac{{\eta }_2}{2}\parallel W_o{\parallel }^2-\frac{{\eta }_2-2{\sigma }_{\mathrm{M}}^2}{2}{\parallel {\tilde{W}}_o\parallel }^2+{\epsilon }_{\mathrm{M}}^2.\hfill \end{array}$$

(17)

Select ${\eta }_2\ge 2{\sigma }_{\mathrm{M}}^2$ and complete the square with respect to $\parallel {\tilde{W}}_o\parallel$, then (17) becomes

$$\begin{array}{cc}\hfill {\dot{L}}_1\le & -\frac{1}{2}\tau \parallel \tilde{x}{\parallel }^2-\frac{1}{2}\rho \parallel \tilde{d}{\parallel }^2+\left(g_{\mathrm{M}}{\epsilon }_{\mathrm{M}}\parallel P\parallel -{\eta }_2{\left(\parallel {\tilde{W}}_o\parallel -{\Theta }_1\right)}^2+{\eta }_2{\parallel {\Theta }_1\parallel }^2\right)\parallel \tilde{x}\parallel \hfill \\ \hfill & +d_{\mathrm{M}}\parallel \tilde{d}\left(t\right)\parallel +{\Theta }_2,\hfill \end{array}$$

(18)

where

$$\begin{array}{c}\hfill {\Theta }_1=\frac{g_{\mathrm{M}}{\sigma }_{\mathrm{M}}\parallel P\parallel +{\eta }_1{g}_{\mathrm{M}}{\sigma }_{\mathrm{M}}\parallel A^{-1}\parallel +{\eta }_2{W}_{\mathrm{M}}}{2{\eta }_2},{\Theta }_2=\frac{{\eta }_2{\parallel W_o\parallel }^2+2{\epsilon }_{\mathrm{M}}^2}{2}.\end{array}$$

Define

$$e_{xd}=\left[\begin{array}{c}\tilde{x}\\ \tilde{d}\end{array}\right],E_o=\frac{1}{2}\left[\begin{array}{c}\tau I0\\ 0\rho I\end{array}\right]$$

and $B_o=\left[g_{\mathrm{M}}{\epsilon }_{\mathrm{M}}\parallel P\parallel +{\eta }_2{\parallel {\Theta }_1\parallel }^2,d_{\mathrm{M}}\right]$, we can further derive

$${\dot{L}}_1\le -{\lambda }_{\min }\left(E_o\right)\parallel e_{xd}{\parallel }^2+\parallel B_o\parallel \parallel e_{xd}\parallel +{\Theta }_2.$$

(19)

Therefore, we can conclude that ${\dot{L}}_1<0$ only if $\parallel e_{xd}\left(t\right)\parallel$ satisfies

$$\parallel e_{xd}\parallel >\frac{\parallel B_o\parallel }{2{\lambda }_{\min }\left(E_o\right)}+\sqrt{\frac{\parallel B_o{\parallel }^2}{4{\lambda }_{\min }^2\left(E_o\right)}+\frac{{\Theta }_2}{{\lambda }_{\min }\left(E_o\right)}}.$$

Furthermore, according to the Lyapunov extension theorem [16], when the inequality (10) holds by selecting proper matrices, we can infer that all the estimation errors $\tilde{x}$, $\tilde{d}$, and ${\tilde{W}}_o$ are UUB. □

Remark 1.

The gain function matrix $l\left(x\right)$ is an important design parameter that can be chosen as linear or nonlinear functions. When the form of system function $g\left(x\right)$ is simple, it can be easy to find the function $l\left(x\right)$ that satisfies the inequality (10) by substituting appropriate functions into (10). However, if the form of system function $g\left(x\right)$ is complex, the trial and error method is employed to select appropriate function $l\left(x\right)$ that meets the inequality (10). Although there is no universal design procedure for designing $l\left(x\right)$, experience has shown that it is not difficult to find a suitable $l\left(x\right)$ for specific applications [36,37].

To effectively handle both system uncertainties and external disturbances, we propose a compound $H_{\infty }$ optimal SMC scheme that combines the integral SMC with $H_{\infty }$ control theories. This compound controller is formulated as

$$u=u_d+u_c,$$

(20)

where $u_d$ represents the discontinuous control designed to steer the system trajectories towards and maintain them on the sliding surface, thereby eliminating the effects of matched uncertainties and disturbances. $u_c$ denotes the continuous control derived to guarantee the system stability and achieve near-optimal performance under the remaining unmatched disturbances on sliding surfaces.

Accordingly, we define the integral sliding surface as follows:

$$s\left(x\right)=S_0\left(x\right)-S_0\left(x_0\right)-{\int }_0^{t}G\left(x\right)\left(f\left(x\right)+g\left(x\right)u_c\right)dv,$$

(21)

where $x_0$ denotes the initial state, $S_0\left(x\right)\in {\mathbb{R}}^m$ and $G\left(x\right)=\partial S_0\left(x\right)/\partial x\in {\mathbb{R}}^{m\times n}$. Moreover, it follows from Assumption 1 that a suitable matrix $G\left(x\right)$ can be found such that the product $G\left(x\right)g\left(x\right)$ is invertible.

Taking the time derivative of $s\left(x\right)$ as

$$\dot{s}\left(x\right)=G\left(x\right)\left(g\left(x\right)u_d+g\left(x\right)w(x,u)+d\right).$$

(22)

By incorporating the valid estimators $\widehat{d}$ and ${\widehat{W}}_o$, $u_d$ is devised as

$$\begin{array}{cc}\hfill u_d=& -{\left(G\left(x\right)g\left(x\right)\right)}^{-1}\left(G\left(x\right)\widehat{d}+G\left(x\right)g\left(x\right){\widehat{W}}_o^{\mathsf{T}}\sigma \left(z\right)+\mu \mathrm{sgn}\left(s\right)+\frac{G\left(x\right)G^{\mathsf{T}}\left(x\right)s}{\parallel s^{\mathsf{T}}G\left(x\right)\parallel }\zeta \right),\hfill \end{array}$$

(23)

where $\mu >0$, $\mathrm{sgn}\left(s\right)\in {\mathbb{R}}^m$ is the sign function, and $\zeta \in \mathbb{R}$ is generated by

$$\dot{\zeta }=\kappa \parallel s^{\mathsf{T}}G\left(x\right)\parallel$$

(24)

with $\kappa >0$. In particular, it is noted that $\zeta$ is designed to tackle the unknown bounds of the approximation errors arisen from the estimated terms $\widehat{d}$ and ${\widehat{W}}_o$.

Considering the specific implementation of $\widehat{d}$ and ${\widehat{W}}_o$ in (23), we define ${\zeta }_e=\tilde{d}+g\left(x\right){\tilde{W}}_o\sigma \left(z\right)+g\left(x\right){\epsilon }_o\left(x\right)$ to represent the approximation errors. Based on the previous analysis and the boundedness of $g\left(x\right)$, ${\zeta }_e$ is bounded as $\parallel {\zeta }_e\parallel \le {\zeta }_{\mathrm{M}}$ for an unknown positive constant ${\zeta }_{\mathrm{M}}$. To estimate ${\zeta }_{\mathrm{M}}$, we design $\zeta$ as defined in (24), and the estimation error is calculated as $\tilde{\zeta }={\zeta }_{\mathrm{M}}-\zeta$.

Theorem 1.

Considering system (2) with the sliding surface (21), the discontinuous control $u_d$ is devised by (23) with the adaptive law (24), then it can guarantee the convergence of the sliding surface s to zero from the beginning.

Proof. 

Choose the positive definite Lyapunov function candidate as

$$L_s=\frac{1}{2}{s}^{\mathsf{T}}s+\frac{1}{2}{\kappa }^{-1}{\tilde{\zeta }}^2.$$

Along with the system (2), $\dot{L_s}\left(t\right)$ is derived as

$$\begin{array}{cc}\hfill \dot{L_s}& =s^{\mathsf{T}}\dot{s}-{\kappa }^{-1}\tilde{\zeta }\dot{\zeta }\hfill \\ \hfill & =s^{\mathsf{T}}G\left(x\right)\left(g\left(x\right)u_d+g\left(x\right)w(x,u)+d\right)-{\kappa }^{-1}\tilde{\zeta }\dot{\zeta }.\hfill \end{array}$$

(25)

Substituting (23) and (24) into (25), we can have

$$\begin{array}{cc}\hfill \dot{L_s}& =s^{\mathsf{T}}G\left(x\right)(\tilde{d}+g\left(x\right){\tilde{W}}_o^{\mathsf{T}}\sigma \left(z\right)+g\left(x\right){\epsilon }_o\left(x\right))-\mu s^{\mathsf{T}}\mathrm{sgn}\left(s\right)-\parallel s^{\mathsf{T}}G\left(x\right)\parallel \zeta -\tilde{\zeta }\parallel s^{\mathsf{T}}G\left(x\right)\parallel \hfill \\ \hfill & =s^{\mathsf{T}}G\left(x\right){\zeta }_e-\mu s^{\mathsf{T}}\mathrm{sgn}\left(s\right)-\parallel s^{\mathsf{T}}G\left(x\right)\parallel \zeta -\tilde{\zeta }\parallel s^{\mathsf{T}}G\left(x\right)\parallel .\hfill \end{array}$$

Using $\parallel {\zeta }_e\parallel \le {\zeta }_{\mathrm{M}}$ and the estimation error $\tilde{\zeta }={\zeta }_{\mathrm{M}}-\zeta$ yields

$$\begin{array}{cc}\hfill \dot{L_s}& \le \parallel s^{\mathsf{T}}G\left(x\right)\parallel {\zeta }_{\mathrm{M}}-\parallel s^{\mathsf{T}}G\left(x\right)\parallel \zeta -\tilde{\zeta }\parallel s^{\mathsf{T}}G\left(x\right)\parallel -\mu s^{\mathsf{T}}\mathrm{sgn}\left(s\right)\hfill \\ \hfill & \le -\mu s^{\mathsf{T}}\mathrm{sgn}\left(s\right).\hfill \end{array}$$

(26)

Thus, it is shown from (26) that $\dot{L_s}\le -\mu {\parallel s\parallel }_1<0$ for any $s\ne 0$, where ${\parallel s\parallel }_1$ denotes the vector 1-norm. This means the asymptotic stability and convergence of sliding mode motion $s\left(x\right)=0$ can be guaranteed. Moreover, according to (21), the sliding surface $s\left(x_0\right)=0$ when $t=0$, which implies that the system states start on the sliding surface, thus avoiding the need for a separate reaching phase. □

From Theorem 1, it is clear that the stable sliding motion $s\left(x\right)=0$ exists from the initial time; that is, for all $t\ge 0$, $s\left(x\right)=0$ and $\dot{s}\left(x\right)=0$. Moreover, the equivalent control method is utilized to obtain the sliding-mode dynamics. Combining $\dot{s}\left(x\right)=0$ with (3) and (22), the equivalent control can be derived as

$$u_{\mathrm{deq}}=-{\left(G\left(x\right)g\left(x\right)\right)}^{-1}G\left(x\right)\left(I-g\left(x\right)g^{+}\left(x\right)\right)d-g^{+}\left(x\right)d-w(x,u).$$

(27)

Then, substitute $u_{\mathrm{deq}}$ into (2), the sliding-mode dynamics without matched uncertain term and disturbance component is

$$\dot{x}=f\left(x\right)+g\left(x\right)u_c+\Gamma \left(x\right)d_u,$$

(28)

where $\Gamma \left(x\right)=I-g\left(x\right){\left(G\left(x\right)g\left(x\right)\right)}^{-1}G\left(x\right)$, $d_u=(I-g\left(x\right)g^{+}\left(x\right))d$ is the unmatched component of the external disturbance in (3). In order to reduce the influence of multiplier matrix $\Gamma \left(x\right)$ and minimize the unmatched disturbance $\Gamma \left(x\right)d_u$, an optimal projection matrix $G^{*}\left(x\right)$ within $\Gamma \left(x\right)$ is provided in the following Lemma.

Lemma 2.

Considering nonlinear system (2) with Assumption 1, the optimal projection matrix $G^{*}\left(x\right)$ is selected as $G^{*}\left(x\right)=g^{+}\left(x\right)$, which not only minimizes the norm $\parallel \Gamma \left(x\right)d_u\parallel$, but also makes the relation $\Gamma \left(x\right)d_u=d_u$ hold.

Proof. 

The proof can refer to Theorem 1 in [12]. □

As a result, with the relation $\Gamma \left(x\right)d_u=d_u$, we can express (28) as

$$\dot{x}=f\left(x\right)+g\left(x\right)u_c+d_u,$$

(29)

which means that the discontinuous control $u_d$ in (23) can fully counteract the impacts of the matched uncertainties and disturbances.

Notice that in (20), $u_c$ aims not only to suppress the remaining unmatched disturbances on sliding surface, but also to achieve a near-optimal performance for sliding-mode dynamics (29). This formulation can be seen as a nonlinear $H_{\infty }$ optimal control problem, which is known to be challenging to solve directly. In the following, we will demonstrate how to find an approximate $H_{\infty }$ optimal control solution by using the single-network ADP algorithm.

4. H_∞ Control Design for Sliding-Mode Dynamics

Considering (3) and (29), the sliding-mode dynamics is represented as

$$\dot{x}=f\left(x\right)+g\left(x\right)u_c+k\left(x\right)d,$$

(30)

with $k\left(x\right)=I-g\left(x\right)g^{+}\left(x\right)$. Since $g\left(x\right)$ and $g^{+}\left(x\right)$ are bounded, it follows that the function $k\left(x\right)$ is also bounded by $\parallel k\left(x\right)\parallel \le k_{\mathrm{M}}$ with $k_{\mathrm{M}}>0$.

For attenuating the remaining unmatched disturbances $k\left(x\right)d$, the corresponding $H_{\infty }$ control problem of sliding-mode dynamics is established, which aims to seek a feedback control $u_c$ to stabilize the system and achieve $L_2$-gain no larger than $\gamma$, that is,

$${\int }_0^{\infty }\left(x^{\mathsf{T}}Qx+u_c^{\mathsf{T}}R{u}_c\right)\mathrm{d}v\le {\gamma }^2{\int }_0^{\infty }{d}^{\mathsf{T}}d\mathrm{d}v,$$

(31)

where Q and R are positive definite matrices with appropriate dimensions, and $\gamma >0$ refers to the level of the disturbance attenuation. Based on [32,33], by treating the disturbance d as the other system input, we can reframe the $H_{\infty }$ optimal control problem for system (30) as a two-player zero-sum game with the following infinite-horizon cost function:

$$V\left(x\right)={\int }_t^{\infty }\left(x^{\mathsf{T}}Qx+u_c^{\mathsf{T}}R{u}_c-{\gamma }^2{d}^{\mathsf{T}}d\right)\mathrm{d}v.$$

(32)

Assuming that $V\left(x\right)\in C^1$, the Hamiltonian function with the associated admissible control pair $(u_c,d)$ is defined as

$$H(x,\nabla V,u_c,d)=x^{\mathsf{T}}Qx+u_c^{\mathsf{T}}R{u}_c-{\gamma }^2{d}^{\mathsf{T}}d+{(\nabla V)}^{\mathsf{T}}\left(f\left(x\right)+g\left(x\right)u_c+k\left(x\right)d\right)$$

(33)

with $\nabla V=\partial V\left(x\right)/\partial x$. From Bellman’s optimality principle, it follows that the optimal cost function $V^{*}\left(x\right)$ satisfies the HJI equation

$$0=\underset{u_c}{\min }\underset{d}{\max }H(x,\nabla V^{*},u_c,d)$$

(34)

with $\nabla V^{*}=\partial V^{*}\left(x\right)/\partial x$. Moreover, according to the zero-sum game theory [16], we have the following Nash condition

$$\underset{u_c}{\min }\underset{d}{\max }H(x,\nabla V^{*},u_c,d)=\underset{d}{\max }\underset{u_c}{\min }H(x,\nabla V^{*},u_c,d),$$

(35)

which ensures the existence of saddle point $(u_c^{*},d^{*})$ of the HJI Equation (34). Then, applying the stationary condition, one can derive the optimal control $u_c^{*}$ and worst disturbance $d^{*}$ as

$$\begin{array}{cc}\hfill u_c^{*}& =-\frac{1}{2}{R}^{-1}{g}^{\mathsf{T}}\left(x\right)\nabla V^{*},\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill d^{*}& =\frac{1}{2{\gamma }^2}{k}^{\mathsf{T}}\left(x\right)\nabla V^{*}.\hfill \end{array}$$

(37)

By substituting (36) and (37) into (33), the HJI equation associated with $\nabla V^{*}$ becomes

$$\begin{array}{cc}\hfill 0=& x^{\mathsf{T}}Qx+{(\nabla V^{*})}^{\mathsf{T}}f\left(x\right)-\frac{1}{4}{(\nabla V^{*})}^{\mathsf{T}}g\left(x\right)R^{-1}{g}^{\mathsf{T}}\left(x\right)\nabla V^{*}\hfill \\ \hfill & +\frac{1}{4{\gamma }^2}{(\nabla V^{*})}^{\mathsf{T}}{k}^{\mathsf{T}}\left(x\right)k\left(x\right)\nabla V^{*}.\hfill \end{array}$$

(38)

Due to the highly nonlinear nature of the relevant HJI equation, obtaining its analytical solution is extremely difficult, if not impossible. To overcome this challenge, we propose an online optimal algorithm that learns the solution of the HJI equation and achieves $H_{\infty }$ optimal control. This is accomplished through the use of single-network ADP, where only one critic network, implemented by NN, is adopted to approximate the cost function $V^{*}$ related to (38). Therefore, by using the critic NN with $l_c$ neurons, $V^{*}$ is represented over a set $\Omega$ as follows:

$$V^{*}\left(x\right)=W_c^{\mathsf{T}}{\sigma }_c\left(x\right)+{\epsilon }_c\left(x\right)$$

(39)

with the ideal weight vector $W_c\in {\mathbb{R}}^{l_c}$ being unknown, the vector of activation functions ${\sigma }_c\left(x\right)\in {\mathbb{R}}^{l_c}$ and the reconstruction error ${\epsilon }_c\left(x\right)$. Meanwhile, we have the gradient vector

$$\nabla V^{*}={(\nabla {\sigma }_c)}^{\mathsf{T}}{W}_c+\nabla {\epsilon }_c$$

(40)

with $\nabla {\sigma }_c=\partial {\sigma }_c\left(x\right)/\partial x$ and $\nabla {\epsilon }_c=\partial {\epsilon }_c\left(x\right)/\partial x$.

By combining (36), (37) and (40), it is easy to get

$$\begin{array}{cc}\hfill u_c^{*}& =-\frac{1}{2}{R}^{-1}{g}^{\mathsf{T}}\left(x\right)\left({(\nabla {\sigma }_c)}^{\mathsf{T}}{W}_c+\nabla {\epsilon }_c\right),\hfill \end{array}$$

(41)

$$\begin{array}{cc}\hfill d^{*}& =\frac{1}{2{\gamma }^2}{k}^{\mathsf{T}}\left(x\right)\left({(\nabla {\sigma }_c)}^{\mathsf{T}}{W}_c+\nabla {\epsilon }_c\right).\hfill \end{array}$$

(42)

Substituting (41) and (42) into (33), the HJI equation becomes

$$\begin{array}{cc}\hfill 0=& H(x,\nabla V^{*},u_c^{*},d^{*})\hfill \\ \hfill =& x^{\mathsf{T}}Qx+W_c^{\mathsf{T}}\nabla {\sigma }_{c}f\left(x\right)-\frac{1}{4}{W}_c^{\mathsf{T}}\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{W}_c-{\epsilon }_{\mathrm{HJI}},\hfill \end{array}$$

(43)

where $D=g\left(x\right)R^{-1}{g}^{\mathsf{T}}\left(x\right)-k\left(x\right)k^{\mathsf{T}}\left(x\right)/{\gamma }^2$, and the approximate error ${\epsilon }_{\mathrm{HJI}}$ is defined as ${\epsilon }_{\mathrm{HJI}}=-{(\nabla {\epsilon }_c)}^{\mathsf{T}}f\left(x\right)+W_c^{\mathsf{T}}\nabla {\sigma }_{c}D\nabla {\epsilon }_c/2+{(\nabla {\epsilon }_c)}^{\mathsf{T}}D\nabla {\epsilon }_c/4$ due to the NN reconstruction error. Furthermore, taking into account $\parallel k\left(x\right)\parallel \le k_{\mathrm{M}}$ and $\parallel g\left(x\right)\parallel \le g_{\mathrm{M}}$, we can infer that there exists a positive constant $D_{\mathrm{M}}$ in the sense that $\parallel D\parallel \le D_{\mathrm{M}}$.

Because $W_c$ in (39) is unknown, the critic NN with the estimated weights approximates the cost function in the form of

$$\widehat{V}\left(x\right)={\widehat{W}}_c^{\mathsf{T}}{\sigma }_c\left(x\right),$$

(44)

where ${\widehat{W}}_c$ denotes the estimated values of $W_c$. In addition, we can obtain

$$\nabla \widehat{V}={(\nabla {\sigma }_c)}^{\mathsf{T}}{\widehat{W}}_c.$$

(45)

By using (36), (37) and (45), the approximate forms of (41) and (42) are derived as

$$\begin{array}{cc}\hfill {\widehat{u}}_c& =-\frac{1}{2}{R}^{-1}{g}^{\mathsf{T}}\left(x\right){(\nabla {\sigma }_c)}^{\mathsf{T}}{\widehat{W}}_c,\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill {\widehat{d}}_w& =\frac{1}{2{\gamma }^2}{k}^{\mathsf{T}}\left(x\right){(\nabla {\sigma }_c)}^{\mathsf{T}}{\widehat{W}}_c.\hfill \end{array}$$

(47)

Then, incorporating (46) and (47) into (43), we have the approximate Hamiltonian as follows:

$$\begin{array}{cc}\hfill H(x,{\widehat{W}}_c,{\widehat{u}}_c,{\widehat{d}}_w)& =x^{\mathsf{T}}Qx+{\widehat{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}f\left(x\right)-\frac{1}{4}{\widehat{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{\widehat{W}}_c.\hfill \end{array}$$

(48)

Subtracting (43) from (48), the corresponding Hamiltonian error is defined as

$$\begin{array}{c}\hfill e_c=H(x,{\widehat{W}}_c,{\widehat{u}}_c,{\widehat{d}}_w)-H(x,\nabla V^{*},u_c^{*},d^{*})=H(x,{\widehat{W}}_c,{\widehat{u}}_c,{\widehat{d}}_w).\end{array}$$

To effectively approximate the cost function, one needs to adjust the critic NN weight ${\widehat{W}}_c$ in a manner that minimizes the Hamiltonian error $e_c$. To this end, it is common practice to train the critic NN by minimizing the squared residual error $E_c$, where $E_c=e_c^{\mathsf{T}}{e}_c/2$. The traditional weight updating laws of critic NN based on gradient descent method can only minimize the squared error, but cannot provide any guarantee for the stability of the resulting system during the learning phase.

However, in practice, the stability is one fundamental requirement of system, and a prerequisite for achieving other higher performance. Thus, not just for minimizing the residual error, but also to guarantee the system stability and eliminate the need for an initial stabilizing control, a weight updating law is developed for the critic NN as follows:

$$\begin{array}{cc}\hfill {\dot{\widehat{W}}}_c=& -\alpha \frac{\varphi }{{({\varphi }^{\mathsf{T}}\varphi +1)}^2}{e}_c+\frac{\alpha }{4}\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{\widehat{W}}_c\frac{{\varphi }_1^{\mathsf{T}}}{{\varphi }_s}{\widehat{W}}_c-\alpha (F_2-F_1{\varphi }_1^{\mathsf{T}}){\widehat{W}}_c\hfill \\ \hfill & +\frac{\beta }{2}\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w)\nabla {\sigma }_{c}D\nabla J_a,\hfill \end{array}$$

(49)

where $\alpha$ and $\beta$ are the positive updating ratios, $\varphi =\nabla {\sigma }_c(f\left(x\right)-D{(\nabla {\sigma }_c)}^{\mathsf{T}}{\widehat{W}}_c/2)$, ${\varphi }_1=\varphi /({\varphi }^{\mathsf{T}}\varphi +1)$, ${\varphi }_s={\varphi }^{\mathsf{T}}\varphi +1$, $F_1$ and $F_2$ represent design parameter matrices with suitable dimensions, $J_a\left(x\right)$ is a Lyapunov function candidate provided in Assumption 4, and the index operator $\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w)$ is given by

$$\begin{array}{c}\hfill \Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w)=\left\{\begin{array}{ccc}\hfill & 0,& \mathrm{if}{\dot{J}}_a\left(x\right)={(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)+g\left(x\right){\widehat{u}}_c+k\left(x\right){\widehat{d}}_w)<0\hfill \\ \hfill & 1,& \mathrm{otherwise}\hfill \end{array}\right.\end{array}$$

(50)

with $\nabla J_a=\partial J_a\left(x\right)/\partial x$.

Remark 2.

Note that in (49), the first term is designed by the normalized gradient descent method for minimizing the residual error. The second term has a well-designed form for ensuring the system’s stability, which is derived from the Lyapunov stability analysis. The last term is an additional adjustment term that works or not depends on the index operator $\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w)$, which is selected based on the derivative of $J_a\left(x\right)$ along the sliding-mode dynamics (30), namely, $\dot{J_a}\left(x\right)={(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)+g\left(x\right){\widehat{u}}_c+k\left(x\right){\widehat{d}}_w)$. Once the system dynamics may become unstable, this results in $\dot{J_a}\left(x\right)\ge 0$, then $\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w)=1$ and the last term in (49) is activated. Moreover, based on the negative gradient direction of $\dot{J_a}\left(x\right)$, i.e., $-\partial ({(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)-D\nabla {\sigma }_c^{\mathsf{T}}{\widehat{W}}_c/2))/\partial {\widehat{W}}_c$, the last term is designed to reinforce the training process of the critic NN until the system dynamics become stable. This also eliminates the need for an initial stabilizing control, compared with [35,36,37,39], where the stabilizing control is required for initialization; however, in practical applications, finding an initial stabilizing control is quite challenging.

Remark 3.

Based on [14,15,16], it is necessary to satisfy the persistence of excitation (PE) requirement for updating the weights of critic NN, which enhances its ability to explore the state space and is indispensable for the weights to converge to their desired ones. To fulfill the PE requirement, a probing noise is injected into the control input [15], which may cause the instability problem during the online learning. As a result, it is important to design the last term in (49) for stabilizing the resulting system, especially when the probing signal is injected.

The schematic structure of the proposed $H_{\infty }$ SMC scheme is illustrated in Figure 1. As shown in Figure 1, this structure consists of two main modules: the $H_{\infty }$ optimal learning module and the enhanced observer module. It should be noted that, based on the deduced sliding-mode dynamics, the learning module can operate independently. However, the original system and the observer module rely on the compound control input u, which includes the approximate $H_{\infty }$ optimal control ${\widehat{u}}_c$ obtained from the learning module. Consequently, it is necessary to first run the learning module to obtain the approximate optimal control ${\widehat{u}}_c$ during the implementation process.

Considering (43), together with ${\tilde{W}}_c=W_c-{\widehat{W}}_c$, (48) is represented as

$$e_c=-{\tilde{W}}_c^{\mathsf{T}}\nabla {\sigma }_c\left(f\left(x\right)-\frac{1}{2}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{\widehat{W}}_c\right)+\frac{1}{4}{\tilde{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{\tilde{W}}_c+{\epsilon }_{\mathrm{HJI}}.$$

(51)

By means of the relation ${\dot{\tilde{W}}}_c=-{\dot{\widehat{W}}}_c$ and incorporating (51) into (49), we obtain

$$\begin{array}{cc}\hfill {\dot{\tilde{W}}}_c=& -\alpha \frac{{\varphi }_1}{{\varphi }_s}\left({\tilde{W}}_c^{\mathsf{T}}\varphi -\frac{1}{4}{\tilde{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{\tilde{W}}_c-{\epsilon }_{\mathrm{HJI}}\right)-\frac{\alpha }{4}\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{\widehat{W}}_c\frac{{\varphi }_1^{\mathsf{T}}}{{\varphi }_s}{\widehat{W}}_c\hfill \\ \hfill & +\alpha (F_2-F_1{\varphi }_1^{\mathsf{T}}){\widehat{W}}_c-\frac{\beta }{2}\Pi (x,{\widehat{u}}_c,{\widehat{d}}_w)\nabla {\sigma }_{c}D\nabla J_a.\hfill \end{array}$$

(52)

Next, the main stability theorem is presented, but before that, one basic common assumption for the critic NN is introduced [16], and the other assumption for the sliding-mode dynamics is also needed, which has been used in [34,38].

Assumption 3.

For the critic NN, there exist known positive constants ${\sigma }_{\mathrm{cM}}$, ${\sigma }_{\mathrm{dM}}$, ${\epsilon }_{\mathrm{cM}}$, ${\epsilon }_{\mathrm{dM}}$ and $W_{\mathrm{cM}}$ such that $\parallel {\sigma }_c\left(x\right)\parallel \le {\sigma }_{\mathrm{cM}}$, $\parallel \nabla {\sigma }_c\parallel \le {\sigma }_{\mathrm{dM}}$, $\parallel {\epsilon }_c\left(x\right)\parallel \le {\epsilon }_{\mathrm{cM}}$, $\parallel \nabla {\epsilon }_c\parallel \le {\epsilon }_{\mathrm{dM}}$ and $\parallel W_c\parallel \le W_{\mathrm{cM}}$, respectively. Moreover, the approximation error ${\epsilon }_{\mathrm{HJI}}$ is bounded above by ${\epsilon }_{\mathrm{H}}>0$, namely, $\parallel {\epsilon }_{\mathrm{HJI}}\parallel \le {\epsilon }_{\mathrm{H}}$.

Assumption 4.

Considering the sliding-mode dynamics (30) with the optimal control pair $(u_c^{*},d^{*})$ in (36) and (37), let $J_a\left(x\right)$ be a smooth, radially unbounded and positive definite Lyapunov candidate that satisfies $\dot{J_a}\left(x\right)={(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)+g\left(x\right)u_c^{*}+k\left(x\right)d^{*})<0$. Moreover, it is assumed that a positive definite matrix $\Psi \left(x\right)$ makes ${(\nabla J^{*})}^{\mathsf{T}}\Psi \left(x\right)\nabla J_a=x^{\mathsf{T}}Qx+u_c^{*\mathsf{T}}R{u}_c^{*}-{\gamma }^2{d}^{*\mathsf{T}}{d}^{*}$ hold. Then, one can derive

$${(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)+g\left(x\right)u_c^{*}+k\left(x\right)d^{*})=-{(\nabla J_a)}^{\mathsf{T}}\Psi \left(x\right)\nabla J_a.$$

(53)

Remark 4.

Note that the plausibility of Assumption 4 depends on the boundedness of optimal sliding-mode dynamics, which is usually assumed to be bounded by a function of system state x. For more details, refer to [34,38]. Furthermore, it is impossible to solve (53) directly for getting the form of $J_a\left(x\right)$. Based on [34], one can obtain $J_a\left(x\right)$ by selecting an appropriate form, such as a quadratic polynomial.

Theorem 2.

Considering the sliding-mode dynamics (30) and its associated cost function (32), the control input and disturbance policy are designed by (46) and (47), respectively, along with the critic weight updating law as given by (49). Then, both the sliding-mode state x and the weight estimation error ${\tilde{W}}_c$ are ensured to be UUB. Furthermore, the obtained control input ${\widehat{u}}_c$ can be proven to converge to a neighborhood of the optimum control $u_c^{*}$ with a small adjustable bound.

Proof. 

Consider the following Lyapunov function candidate

$$L=\frac{1}{2}{\tilde{W}}_c^{\mathsf{T}}{\alpha }^{-1}{\tilde{W}}_c+{\beta }_1{J}_a\left(x\right),$$

where ${\beta }_1=\beta /\alpha >0$. By calculating the time derivative of L along the sliding-mode dynamics (30), we have

$$\dot{L}={\tilde{W}}_c^{\mathsf{T}}{\dot{\tilde{W}}}_c+{\beta }_1{(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)+g\left(x\right){\widehat{u}}_c+k\left(x\right){\widehat{d}}_w).$$

(54)

Substituting (52) into (54) and making some adjustments, one can get

$$\begin{array}{cc}\hfill \dot{L}=& -{\tilde{W}}_c^{\mathsf{T}}{\varphi }_1{\varphi }_1^{\mathsf{T}}{\tilde{W}}_c+{\beta }_1{(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)+g\left(x\right){\widehat{u}}_c+k\left(x\right){\widehat{d}}_w)+\frac{1}{4}{\tilde{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{W}_c\frac{{\varphi }_1^{\mathsf{T}}}{{\varphi }_s}{\tilde{W}}_c\hfill \\ \hfill & -\frac{1}{4}{\tilde{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{W}_c\frac{{\varphi }_1^{\mathsf{T}}}{{\varphi }_s}{W}_c-\frac{1}{4}{\tilde{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{\tilde{W}}_c\frac{{\varphi }_1^{\mathsf{T}}}{{\varphi }_s}{W}_c\hfill \\ \hfill & +{\tilde{W}}_c^{\mathsf{T}}\frac{{\varphi }_1^{\mathsf{T}}}{{\varphi }_s}{\epsilon }_{\mathrm{HJI}}-\frac{\beta }{2}\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w){\tilde{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}D\nabla J_a\hfill \\ \hfill & +{\tilde{W}}_c^{\mathsf{T}}{F}_2{\widehat{W}}_c-{\tilde{W}}_c^{\mathsf{T}}{F}_1{\varphi }_1^{\mathsf{T}}{\widehat{W}}_c.\hfill \end{array}$$

(55)

Using ${\widehat{W}}_c=W_c-{\tilde{W}}_c$, the last two terms in (55) become

$${\tilde{W}}_c^{\mathsf{T}}{F}_2{\widehat{W}}_c-{\tilde{W}}_c^{\mathsf{T}}{F}_1{\varphi }_1^{\mathsf{T}}{\widehat{W}}_c={\tilde{W}}_c^{\mathsf{T}}{F}_2{W}_c-{\tilde{W}}_c^{\mathsf{T}}{F}_2{\tilde{W}}_c-{\tilde{W}}_c^{\mathsf{T}}{F}_1{\varphi }_1^{\mathsf{T}}{W}_c+{\tilde{W}}_c^{\mathsf{T}}{F}_1{\varphi }_1^{\mathsf{T}}{\tilde{W}}_c.$$

(56)

Defining ${\rm Y}={[{\tilde{W}}_c^{\mathsf{T}}{\varphi }_1,{\tilde{W}}_c^{\mathsf{T}}]}^{\mathsf{T}}$, and substituting (55) into (56), it can be rewritten as

$$\begin{array}{cc}\hfill \dot{L}=& -{{\rm Y}}^{\mathsf{T}}M{\rm Y}+{{\rm Y}}^{\mathsf{T}}\delta +{\beta }_1{(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)+g\left(x\right){\widehat{u}}_c+k\left(x\right){\widehat{d}}_w)\hfill \\ \hfill & -\frac{\beta }{2}\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w){\tilde{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}D\nabla J_a,\hfill \end{array}$$

(57)

where

$$\begin{array}{cc}\hfill M& =\left[\begin{array}{cc}I& -{\displaystyle \frac{\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{W}_c}{4{\varphi }_s}}-{\displaystyle \frac{F_1}{2}}\\ -{\displaystyle \frac{\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{W}_c}{4{\varphi }_s}}-{\displaystyle \frac{F_1}{2}}& F_2\end{array}\right],\hfill \\ \hfill \delta & =\left[\begin{array}{c}{\displaystyle \frac{1}{{\varphi }_s}}{\epsilon }_{\mathrm{HJI}}\\ -{\displaystyle \frac{\nabla {\sigma }_{c}D{(\nabla {\sigma }_c)}^{\mathsf{T}}{W}_c}{4{\varphi }_s}}+F_2{W}_c-F_1{\varphi }_1^{\mathsf{T}}{W}_c\end{array}\right].\hfill \end{array}$$

With Assumption 3 in mind, and recalling the boundedness of ${\varphi }_1$ and D, in particular $\parallel {\varphi }_1\parallel <1$ and $\parallel D\parallel \le D_{\mathrm{M}}$, we can infer that there exists a positive constant ${\delta }_{\mathrm{M}}$ in the sense that $\parallel \delta \parallel \le {\delta }_{\mathrm{M}}$. For guaranteeing $M>0$, the appropriate parameters $F_1$ and $F_2$ need to be selected in design. Then, one can upper bound $\dot{L}$ as follows:

$$\begin{array}{cc}\hfill \dot{L}\le & -{\lambda }_{\min }{\left(M\right)\parallel {\rm Y}\parallel }^2+{\delta }_{\mathrm{M}}\parallel {\rm Y}\parallel +{\beta }_1{(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)+g\left(x\right){\widehat{u}}_c+k\left(x\right){\widehat{d}}_w)\hfill \\ \hfill & -\frac{{\beta }_1}{2}\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w){\tilde{W}}_c^{\mathsf{T}}\nabla {\sigma }_{c}D\nabla J_a\hfill \end{array}$$

(58)

with ${\lambda }_{\min }\left(M\right)$ being the minimum eigenvalue of M.

According to (50), there are two cases to consider: $\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w)=0$ and $\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w)=1$ for (58) in the following analysis.

$Case$ 1: For $\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w)=0$, it follows from (50) that ${\dot{J}}_a\left(x\right)<0$, i.e., ${(\nabla J_a)}^{\mathsf{T}}\dot{x}<0$, which, together with the PE condition, can ensure that there exists a positive constant $\varrho$ such that $\parallel \dot{Z}\parallel >\varrho$. This implies that ${(\nabla J_a)}^{\mathsf{T}}\dot{x}<-\varrho \parallel \nabla J_a\parallel <0$. Then, (58) becomes

$$\begin{array}{cc}\hfill \dot{L}& \le {\beta }_1{(\nabla J_a)}^{\mathsf{T}}\dot{x}-{\lambda }_{\min }\left(M\right){\parallel {\rm Y}\parallel }^2+{\delta }_{\mathrm{M}}\parallel {\rm Y}\parallel \hfill \\ \hfill & <-{\beta }_1\varrho \parallel \nabla J_a\parallel -{\lambda }_{\min }\left(M\right){\left(\parallel {\rm Y}\parallel -{\displaystyle \frac{{\delta }_{\mathrm{M}}}{2{\lambda }_{\min }\left(M\right)}}\right)}^2+\frac{{\delta }_{\mathrm{M}}^2}{4{\lambda }_{\min }\left(M\right)}.\hfill \end{array}$$

(59)

Focus on (59), only if the following inequalities:

$$\parallel \nabla J_a\parallel >\frac{{\delta }_{\mathrm{M}}^2}{4{\lambda }_{\min }\left(M\right)\varrho }\triangleq {\mathcal{A}}_1$$

or

$$\parallel {\rm Y}\parallel >\frac{{\delta }_{\mathrm{M}}^2}{{\lambda }_{\min }\left(M\right)}$$

hold, then $\dot{L}<0$. Moreover, based on the relation $\parallel {\rm Y}\parallel \le \sqrt{\parallel {\varphi }_1{\parallel }^2+1}\parallel {\tilde{W}}_c\parallel$ with $\parallel {\varphi }_1\parallel <1$, we can derive

$$\parallel {\tilde{W}}_c\parallel >\frac{{\delta }_{\mathrm{M}}^2}{\sqrt{2}{\lambda }_{\min }\left(M\right)}\triangleq {\mathcal{B}}_1.$$

$Case$ 2: For $\Sigma (x,{\widehat{u}}_c,{\widehat{d}}_w)=1$, in light of (41) and (42), by adding and subtracting ${\beta }_1{(\nabla J_a)}^{\mathsf{T}}D\nabla {\epsilon }_c/2$ into (58), we can derive

$$\begin{array}{cc}\hfill \dot{L}\le & -{\lambda }_{\min }\left(M\right){\left(\parallel {\rm Y}\parallel -\frac{{\delta }_{\mathrm{M}}}{2{\lambda }_{\min }\left(M\right)}\right)}^2+\frac{{\delta }_{\mathrm{M}}^2}{4{\lambda }_{\min }\left(M\right)}+{\beta }_1{(\nabla J_a)}^{\mathsf{T}}(f\left(x\right)+g\left(x\right)u_c^{*}\hfill \\ \hfill & +k\left(x\right)d^{*})+\frac{{\beta }_1}{2}{(\nabla J_a)}^{\mathsf{T}}D\nabla {\epsilon }_c.\hfill \end{array}$$

(60)

Then, using (53) in Assumption 4, and recalling the boundedness of D and $\nabla {\epsilon }_c$, (60) is upper bounded as

$$\dot{L}\le -{\lambda }_{\min }\left(M\right){\left(\parallel {\rm Y}\parallel -\frac{{\delta }_{\mathrm{M}}}{2{\lambda }_{\min }\left(M\right)}\right)}^2-\frac{{\beta }_1}{2}{\lambda }_{\min }(\Psi ){\parallel \nabla J_a\parallel }^2+\Phi ,$$

(61)

where $\Phi ={\delta }_{\mathrm{M}}^2/\left(4{\lambda }_{\min }\left(M\right)\right)+{\beta }_1{D}_{\mathrm{M}}^2{\epsilon }_{\mathrm{dM}}^2/\left(8{\lambda }_{\min }(\Psi )\right)$, ${\lambda }_{\min }(\Psi )$ denotes the minimum eigenvalue of $\Psi \left(x\right)$. Hence, provided the following inequalities:

$$\parallel \nabla J_a\parallel >\sqrt{\frac{2\Phi }{{\beta }_1{\lambda }_{\min }(\Psi )}}\triangleq {\mathcal{A}}_2$$

or

$$\parallel {\rm Y}\parallel >\sqrt{\frac{\Phi }{{\lambda }_{\min }\left(M\right)}}+\frac{{\delta }_{\mathrm{M}}}{2{\lambda }_{\min }\left(M\right)}$$

hold, one has $\dot{L}<0$. Further, by the relation $\parallel {\rm Y}\parallel \le \sqrt{2}\parallel {\tilde{W}}_c\parallel$, we have

$$\parallel {\tilde{W}}_c\parallel >\sqrt{\frac{\Phi }{2{\lambda }_{\min }\left(M\right)}}+\frac{{\delta }_{\mathrm{M}}}{2\sqrt{2}{\lambda }_{\min }\left(M\right)}\triangleq {\mathcal{B}}_2.$$

To sum up, for both $Case$ 1 and $Case$ 2, with proper parameters $F_1$ and $F_2$ satisfying $M>0$, the inequality $\parallel \nabla J_a\parallel \ge \max \{{\mathcal{A}}_1,{\mathcal{A}}_2\}=\overline{\mathcal{A}}$ or $\parallel {\tilde{W}}_c\parallel \ge \max \{{\mathcal{B}}_1,{\mathcal{B}}_2\}=\overline{\mathcal{B}}$ holds, then, we have $\dot{L}<0$. From the Lyapunov extension theorem [16], it is found that $\parallel \nabla J_a\parallel$ and $\parallel {\tilde{W}}_c\parallel$ are bounded by $\overline{\mathcal{A}}$ and $\overline{\mathcal{B}}$, respectively. Based on Assumption 4, the Lyapunov candidate $J_a\left(x\right)$ is radially unbounded, which implies that the boundedness of $\parallel \nabla J_a\parallel$ leads to the boundedness of the system state $\parallel x\parallel$. In particular, $\parallel x\parallel$ is bounded by ${\overline{\mathcal{A}}}_x=\max \{{\mathcal{A}}_{1x},{\mathcal{A}}_{2x}\}$, where ${\mathcal{A}}_{1x}$ and ${\mathcal{A}}_{2x}$ are determined by ${\mathcal{A}}_1$ and ${\mathcal{A}}_2$, respectively. So far, we can conclude that both x and ${\tilde{W}}_c$ are guaranteed to be UUB.

Next, we will prove ${\widehat{u}}_c$ converges to a small neighborhood of $u_c^{*}$ with an adjustable bound, i.e., $\parallel {\widehat{u}}_c-u_c^{*}\parallel \le {ϵ}_u$. Considering (41) and (46), we have

$${\widehat{u}}_c-u_c^{*}=-\frac{1}{2}{R}^{-1}{g}^{\mathsf{T}}\left(x\right)({(\nabla {\sigma }_c)}^{\mathsf{T}}{\tilde{W}}_c+\nabla {\epsilon }_c).$$

Noticing that ${\tilde{W}}_c$ is UUB together with the associated bound $\overline{\mathcal{B}}=\max \{{\mathcal{B}}_1,{\mathcal{B}}_2\}$, and invoking $\parallel g\left(x\right)\parallel \le g_{\mathrm{M}}$, $\parallel \nabla {\sigma }_c\parallel \le {\sigma }_{\mathrm{dM}}$, $\parallel \nabla {\epsilon }_c\parallel \le {\epsilon }_{\mathrm{dM}}$ and boundedness of R, it follows that

$$\begin{array}{cc}\hfill \parallel {\widehat{u}}_c-u_c^{*}\parallel & \le \frac{1}{2}{\lambda }_{\max }\left(R^{-1}\right)g_{\mathrm{M}}({\sigma }_{\mathrm{dM}}\overline{\mathcal{B}}+{\epsilon }_{\mathrm{dM}})\triangleq {ϵ}_u.\hfill \end{array}$$

(62)

 □

Remark 5.

From the expression of ${\mathcal{B}}_1$ and ${\mathcal{B}}_2$, it is seen that $\overline{\mathcal{B}}$ can be kept small with ${\lambda }_{\min }\left(M\right)$ being larger enough. In view of (57), we can enlarge the value of ${\lambda }_{\min }\left(M\right)$ by adjusting the corresponding design parameters $F_1$ and $F_2$. Moreover, we can make the approximate error ${\epsilon }_c$ and its upper bound ${\epsilon }_{\mathrm{dM}}$ sufficiently small when the neuron number $l_c$ is large enough. Therefore, we can make the convergence errors ${ϵ}_u$ in (62) as small as possible in the design.

5. Simulation Results

To validate the effectiveness of the proposed $H_{\infty }$ optimal SMC scheme, two simulation examples are provided. The first example focuses on a single-link robot arm, while the second example deals with a power system.

5.1. Single-Link Robot Arm

Considering a nonlinear single-link robot arm [23] and its dynamics given by

$$J\ddot{\theta }=-MgL\sin \left(\theta \right)-D\dot{\theta }+u+w,$$

(63)

where $\theta$ is the joint rotation angle of robot arm in radians, u refers to the control torque applied to the joint in $\mathrm{Nm}$, and w denotes the lumped uncertain term. Select the system parameters as follows: the arm length $L=0.5\mathrm{m}$, the payload mass $M=1\mathrm{kg}$, the local gravity acceleration $g=9.81\mathrm{m}/{\mathrm{s}}^2$. the rotational inertia $J=1\mathrm{kg}·{\mathrm{m}}^2$ and the viscous friction $D=2\mathrm{Nm}·\mathrm{s}/\mathrm{rad}$. With the system states defined as $x_1=\theta$ and $x_2=\dot{\theta }$, and considering the presence of exogenous disturbances, then the dynamics (63) in state-space form can be represented as

$$\left[\begin{array}{c}{\dot{x}}_1\\ {\dot{x}}_2\end{array}\right]=\left[\begin{array}{c}{x}_2\\ -4.905\sin \left(x_1\right)-2x_2\end{array}\right]+\left[\begin{array}{c}0\\ 1\end{array}\right](u+w)+d,$$

(64)

where d represents the unknown disturbances. Moreover, it is assumed that the initial state is set as $x_0={[1,-0.5]}^{\mathsf{T}}$, the lumped uncertainty term is $w(x,u)=x_2\sin \left(x_1\right)+0.1\sin \left(x_1\right)u$, and the disturbance term is chosen as $d={[0.5e^{-t}\sin \left(t\right),0.5\sin \left(t\right)]}^{\mathsf{T}}$ in the simulation.

The enhanced observer system, consisting of an NN identifier and a nonlinear DO, can be designed as shown in (6), where the identifier NN is selected as a three-layered feedforward NN with one hidden layer containing six neurons, and the hyperbolic activation function $\tanh (·)$ is utilized. The updating ratios are set as ${\eta }_1=30$ and ${\eta }_2=2.5$, while the weights ${\widehat{W}}_o$ and ${\widehat{V}}_o$ are initialized with random values chosen from the interval $[-0.1,0.1]$. The initial observer state is set as ${\widehat{x}}_0={[0.5,0]}^{\mathsf{T}}$. Moreover, based on Lemma 1, select the Hurwitz matrix $A=[-15,0;0,-15]$, $p\left(x\right)=[10x_1;10x_2]$ and $l\left(x\right)=[10,0;0,10]$ to ensure that the inequality (10) holds. The integral sliding surface function is determined by (21), together with $G\left(x\right)=g^{+}\left(x\right)=[0,1]$ and $S_0\left(x\right)=x_2$. Accordingly, the discontinuous SMC $u_d$ is given by (23) and (24). For the propose of eliminating the chattering phenomenon, an arctangent function $\mathrm{atan}(s/ϵ)$ with a small positive scalar $ϵ=0.005$ is employed to replace the sign function $\mathrm{sgn}\left(s\right)$ in (23).

By considering the SMC law $u_d$, the sliding-mode dynamics can be obtained as

$$\dot{x}=f\left(x\right)+g\left(x\right)u_c+k\left(x\right)d,$$

(65)

where $k\left(x\right)=I-g\left(x\right)g^{+}\left(x\right)=[1,0;0,0]$. We choose the associated cost function as the form of (32), together with $Q=\mathrm{diag}(1,1)$, $R=1$ and $\gamma =1.5$. For the critic NN, the activation function is chosen as ${\sigma }_c\left(x\right)={[x_1^2,x_1{x}_2,x_2^2,x_1^3{x}_2,x_1^2{x}_2^2,x_1{x}_2^3]}^{\mathsf{T}}$, which results in ${\widehat{W}}_c={[{\widehat{W}}_{c1},{\widehat{W}}_{c2},\dots ,{\widehat{W}}_{c6}]}^{\mathsf{T}}$. Select the updating ratios $\alpha =1$, $\beta =0.5$, the design parameters $F_1=F_2=10I$, $l_c=6$ and $J_a\left(x\right)$ as a quadratic polynomial. Furthermore, the weight vector ${\widehat{W}}_c$ is initialized to zero, which leads to the initial control input of zero. Noticing that the zero initial control cannot make the system (65) stable, it is thus clear that no initial stabilizing control strategy is necessary when implementing the proposed algorithm.

During the learning process, a damped decreasing probing noise is injected into the control input for satisfying PE condition. This noise comprises sinusoids of diverse frequencies and is applied for the first 450 s. Figure 2 shows the trajectories of the critic weights, which eventually converge to ${\widehat{W}}_c={[1.0420,0.0856,-0.0603,-0.2174,0.2948,-0.0358]}^{\mathsf{T}}$. Figure 3 describes the trajectories of system states in the learning. From Figure 3, one can see that without an initial stabilizing control, the system states stay at or near zero after the probing noise is removed, which indicates that ${\widehat{u}}_c$ generated by the learning module can effectively stabilize the system. With the converged weights, the approximate $H_{\infty }$ optimal control ${\widehat{u}}_c$ can be calculated by (46).

Next, we substitute ${\widehat{u}}_c$ into (21) to obtain an available sliding surface. Subsequently, integrating with the enhanced observer system, the SMC law $u_d$ is implemented by using (23) and (24) with the reliable estimations of uncertainties and disturbances. Figure 4 depicts the estimates of disturbances $d_1=0.5e^{-t}\sin \left(t\right)$ and $d_2=0.5\sin \left(t\right)$, along with small estimation errors. Figure 5 presents the identifications of system states using the identifier NN. It can be observed that the identified states rapidly track the real states, illustrating the effectiveness and efficiency of the identifier NN. Note that the valid estimations $\widehat{d}$ and ${\widehat{W}}_o$ are used to design the SMC law $u_d$, which helps to reduce the sliding-mode gain and alleviate the chattering phenomenon. Figure 6 displays the state trajectories of the robot arm under the compound $H_{\infty }$ sliding-mode control $u=u_d+{\widehat{u}}_c$. Figure 7 depicts the compound control u, while the $H_{\infty }$ control ${\widehat{u}}_c$ and the SMC law $u_d$ are given in Figure 8. These results presented in Figure 6, Figure 7 and Figure 8 confirm that the compound control u successfully renders the robot arm system stable and exhibits satisfactory performance against both system uncertainties and external disturbances.

5.2. Power Plant System

To further validate the effectivity of the proposed scheme, we consider an electric power system comprised of a gas turbine generator, a system load, and an automatic generation control [34]. To model this system, the incremental frequency deviation $\Delta f_G$, the generator output power variation $\Delta P_m$, and the valve position change of the governor $\Delta v$ are taken into consideration. The control input is represented by the speed change $\Delta P_c$ in position deviation. By defining the state vector $x={[\Delta v,\Delta P_m,\Delta f_G]}^{\mathsf{T}}\in {\mathbb{R}}^3$, we can express the reduced power system model in state-space form as

$$\dot{x}=\left[\begin{array}{ccc}-{\displaystyle \frac{1}{T_g}}& 0& {\displaystyle \frac{1}{R_g{T}_g}}\\ {\displaystyle \frac{K_t}{T_t}}& -{\displaystyle \frac{1}{T_t}}& 0\\ 0& {\displaystyle \frac{K_p}{T_p}}& -{\displaystyle \frac{1}{T_p}}\end{array}\right]x+\left[\begin{array}{c}{\displaystyle \frac{1}{T_g}}\\ 0\\ 0\end{array}\right](u+\vartheta )+d$$

(66)

where $g\left(x\right)={[1/T_g,0,0]}^{\mathsf{T}}$, $\vartheta$ represents the modeling uncertainty, and d stands for the exterior disturbances. Assume that the uncertain term is $\vartheta =x_2\sin \left(x_1\right)$, and the disturbance term is defined as $d\left(t\right)={[\sin \left(2\pi t\right)e^{-t},0,0.2{\sin }^2\left(t\right)e^{-t}]}^{\mathsf{T}}$ in the simulation. Let the regulation constant $R_g$ = 2.5 Hz/MW, the turbine gain constant $K_t$ = 1 s and the generator gain constant $K_p$ = 120 Hz/MW. Moreover, the corresponding time constants are set as $T_g$ = 0.08 s, $T_t$ = 0.1 s and $T_p$ = 20 s, respectively.

For estimating the unknown uncertainty and disturbance terms, the enhanced observer system is constructed as (6) with a three-layered feedforward NN containing eight hidden neurons and the Hurwitz matrix $A=[-12,0,0;0,-12,0;0,0,-12]$. The activation function, the initial weights, and the updating ratios are the same as in Section 5.1. Let $p\left(x\right)={[10x_1,0,10x_3]}^{\mathsf{T}}$, $l\left(x\right)=[10,0,0;0,0,0;0,0,10]$, $G\left(x\right)=g^{+}\left(x\right)=[0.08,0,0]$ and $S_0\left(x\right)=0.08x_1$. Similarly, an arctangent function $\mathrm{atan}(s/ϵ)$ is used for designing the SMC law $u_d$ instead of the sign function $\mathrm{sgn}\left(s\right)$.

Without the matched uncertainties and disturbances, we can derive the sliding-mode dynamics from (66), wherein $k\left(x\right)=[0,0,0;0,1,0;0,0,1]$, and the initial state $x_0={[0.2,-0.2,0.1]}^{\mathsf{T}}$. Let the associated cost function be of the form (32) along with $Q=\mathrm{diag}(1,1,1)$, $R=1$ and $\gamma =3$. The critic NN is designed as (44) and its corresponding parameters are $\alpha =15$, $\beta =0.5$, $\sigma \left(x\right)=[x_1^2,x_1{x}_2,x_1{x}_3,x_2^2,x_2{x}_3,x_3^2,x_1^2{x}_2{x}_3,x_1{x}_2^2{x}_3,x_1{x}_2{x}_3^2{]}^{\mathsf{T}}$ and ${\widehat{W}}_c={[{\widehat{W}}_{c1},{\widehat{W}}_{c2},\dots ,{\widehat{W}}_{c9}]}^{\mathsf{T}}$. Similar to Section 5.1, $J_a\left(x\right)=x^{\mathsf{T}}x/2$, the initial weight vector is set to zero, and a similar probing noise is injected into the control input before 550 s. The evolving trajectories of the critic weights are shown in Figure 9, while the trajectories of system states in the learning are depicted in Figure 10. After 550 s, the critic weights converge to ${\widehat{W}}_c={[0.0830,0.1245,0.2284,0.1616,0.4883,0.5488,0.1154,0.0563,0.0564]}^{\mathsf{T}}$, then we can derive ${\widehat{u}}_c$ using (46) with the converged weights.

Then, we substitute ${\widehat{u}}_c$ into the integral sliding surface (21), and we design the SMC law $u_d$ by (23) and (24). Consequently, the compound control is constructed as $u=u_d+{\widehat{u}}_c$. After simulation, Figure 11 shows the trajectories of the power system states under this compound control for 15 s. Figure 12 presents the compound control u. From Figure 11 and Figure 12, we can conclude that the compound control effectively stabilizes the system states to the equilibrium point, even in the presence of modeling uncertainties and exterior disturbances. These results undeniably demonstrate the viability and efficiency of the proposed approach.

6. Conclusions

In this paper, we develop a neural adaptive $H_{\infty }$ sliding-mode control scheme for uncertain nonlinear systems subject to external disturbances. Based on the enhanced observer system composed of the NN identifier and nonlinear DO, an integral SMC is designed for suppressing the influences of the uncertain term and the matched disturbance component, as well as unknown approximation errors, with no prior knowledge of their upper bounds. Meanwhile, on the sliding surface, the remaining unmatched disturbances are attenuated using the $H_{\infty }$ optimal control solved by the single critic network-based ADP algorithm. Furthermore, uniform ultimate boundedness stability of the resultant closed-loop system can be proven by Lyapunov’s method. In addition to the theoretical analysis, two simulation examples are provided to further validate the proposed approach. Recently, the growing interest in saving communication resources or reducing the calculation amount of networked control systems makes the event-triggering mechanism gain more and more attention and undergo rapid development. Hence, how to combine the optimal SMC strategy with the event-triggering mechanism for more complex physical systems, not just for control-affine systems, will be our future research topic.