Dynamic Event-Triggering Surrounding Control for Multi-USVs Under FDI Attacks via Adaptive Dynamic Programming

Abstract

This paper investigates the surrounding control problem of multiple unmanned surface vehicles (USVs) against false data injection (FDI) attacks and proposes a learning-based prescribed performance control (PPC) integrated with a dynamic event-triggering (DET) mechanism. First, a predefined-time observer (PTO) is designed to estimate the injected false data. Then, the constrained surrounding tracking error of multi-USVs is first formulated based on an exponential prescribed performance function. To facilitate the control law design, the constrained surrounding problem is transformed into an unconstrained space using a hyperbolic tangent function. Based on adaptive dynamic programming (ADP) and the DET mechanism, a prescribed performance time-varying surrounding control scheme is developed. Finally, the effectiveness and superiority of the proposed control strategy are demonstrated through rigorous theoretical analysis and simulation experiments, while Zeno behavior in the event-triggered mechanism is excluded.

1. Introduction

In recent years, multi-agent systems (MASs) control has emerged as a significant research focus due to the inherent flexibility, scalability, and intelligent emergence characteristics of MASs. The MASs control technology has been widely used in the formation control of unmanned systems [1], surrounding control of unmanned surface vehicles (USVs) [2], and reconnaissance missions of multiple unmanned aerial vehicles (UAVs) [3].

Multi-USV systems have attracted significant research attention globally, owing to the advantages of their low cost, high maneuverability, and operational reliability. An equal-distance surrounding control strategy was proposed to address the encirclement tasks of multi-USVs with collision avoidance in [4]. Skjetne et al. [5] designed an adaptive recursive control scheme for nonlinear USV systems with parameter uncertainties. In [6], a distributed prescribed performance formation control method with obstacle avoidance was presented for multiple USVs under parameter uncertainties and external disturbances. However, the above-mentioned control methods, such as sliding mode control, adaptive control, and backstepping control, fail to account for the trade-off between control performance and control cost. Moreover, in practical applications, the communication networks of multi-USVs may be vulnerable to cyber-attacks, and the closed-loop systems (CLSs) are required to achieve good steady-state and transient performance. Therefore, it is significant to design a robust, secure, and optimal control scheme with prescribed performance to address the surrounding control problem of multiple USVs under network attacks.

In [7], a fixed-time extended state observer (FTESO) was designed to estimate the velocities and disturbances of multi-USVs, and a FTESO-based fixed-time distributed formation control method was developed. In [8], a leader–follower formation control strategy based on event-triggered mechanism and artificial potential function was proposed for multiple autonomous underwater vehicles (AUVs) in a three-dimensional environment. The authors investigated the formation control problem of multiple USVs under constraints on line-of-sight range and angular measurements, and a fault-tolerant finite-time formation control method based on time-varying tangent barrier Lyapunov function was proposed in [9]. Formation control methods are generally used to realize cooperative tasks. However, the surrounding control and containment control methods can be utilized to address the MASs control problem involving non-cooperative targets [10]. Xu et al. [11] developed an event-triggered control method to address the surrounding-formation control problem with multiple dynamic targets; Zeno behavior did not occur in the event-triggered mechanism. Reference [12] proposed a sampled-data-based containment control strategy to solve the second-order multi-agent control problem involving a dynamic leader with nonzero input, ultimately guaranteeing that all followers converge within the convex hull formed by the dynamic leader. In practical applications, USVs are typically powered by lithium batteries, which are limited in capacity due to the constraints of the USV’s size and structure. Therefore, designing control schemes that minimize energy consumption while maintaining USV performance is of significant value. In [13], an optimal tracking control scheme was designed for USV dead-zone input nonlinearity and external disturbances, which can balance the control cost and performance. In [14], the authors addressed the control problem of fixed-wing UAVs subjected to both matched and unmatched disturbances, and an adaptive optimal control method based on the adaptive gain generalized super-twisting algorithm was proposed to mitigate the negative impact of disturbances on the CLSs.

The aforementioned works primarily focus on designing control methods to ensure the steady-state performance of control systems. However, these control methods do not simultaneously account for both steady-state and transient performance, including factors such as convergence accuracy, convergence speed, and overshoot in the CLSs. In [15], a prescribed performance control (PPC) method was first designed for MIMO nonlinear systems, which can ensure that the state of CLSs remains within the range defined by a prescribed performance function. A reinforcement learning (RL)-based prescribed performance optimal control scheme was presented to deal with the constrained tracking control problem of USV [16]. An adaptive fault-tolerant control scheme based on PPC techniques for spacecraft in the presence of external disturbances and actuator failures in [17]. In [18], a RL-based prescribed performance attitude control method was designed for spacecraft attitude control systems under disturbances and state constraints, and a neural network (NN) weight update law without persistent excitation (PE) conditions was proposed in the RL algorithm. In [19], a leader–follower prescribed performance formation control approach was developed to solve second-order MASs output feedback control problems without velocity measurement. In [20], a FTESO-based PPC method was proposed to handle the path-tracking problem of USVs via sliding mode control (SMC) techniques. An event-triggered-based reinforcement learning PPC method was presented for the underactuated USV in [21]. The event-triggered strategies in the aforementioned works primarily are mainly static triggering mechanisms. To further reduce the number of event triggers, reference [22] employed a dynamic event-triggered (DET) mechanism to design a finite-time formation control method for USVs.

Multi-UAVs are one branch of cyber-physical systems (CPSs); malicious cyberattacks can have severe negative impacts on their security and stability [23,24]. The cyberattacks are mainly categorized into denial of service (DoS) attacks [25] that destroy data availability, and false data injection (FDI) attacks [26] that damage data integrity. Compared to DoS attacks, FDI attacks are more covert and destructive. Considering the consensus control problem of multiple UAVs under deception attacks, a memory-based event-triggered control strategy was proposed by incorporating an averaging mechanism to suppress event mis-triggering caused by deception attacks [27]. For smart grid systems subjected to FDI attacks, a deep-learning-based intelligent detection algorithm was designed to identify the behavioral characteristics of false data using historical measurement data in [28]. He et al. [29] proposed a pulse-synchronization-based secure control method for MASs under deception attacks. For MASs under composite attacks combining FDI and information replay, an adaptive observer-based distributed leader–follower resilient control scheme was proposed in [30]. A periodic event-triggered resilient control strategy was developed for the consensus control problem of linear MASs under FDI and DoS attacks [31]. In [32], a fully distributed resilient control protocol was developed for the formation-containment control problem of higher-order heterogeneous MASs under unbounded FDI attacks. In [33], a distributed resilient containment control method was proposed for MASs against FDI attacks. Zhang et al. [34] designed a predictive sliding mode control method to address the output tracking problem of higher-order fully actuated systems in the presence of FDI attacks. In [35], considering the higher-order fully actuated MASs under FDI attacks, a predictive sliding mode control approach was proposed to ensure the stability and security of CLSs.

By analyzing the aforementioned works, it can be observed that there is relatively little research on the surrounding control problem of multi-USVs under FDI cyberattacks. Inspired by the above studies, this paper proposes a learning-based resilient DET surrounding control strategy for multi-USVs under FDI attacks. The main contributions of this work are as follows:

1. A novel predefined-time observer (PTO) is proposed to estimate false data in the communication network of multi-USVs, and a PTO-based resilient control scheme is designed. In contrast to the asymptotic convergence FDI observer presented in [27], the proposed PTO can estimate the false data within a user-predefined time and does not require the assumption that the false data are bounded.

2. Based on the information from the PTO, a prescribed performance time-varying surrounding control method is designed for multi-USVs within the adaptive dynamic programming (ADP) framework. In [16], an RL-based control method for the tracking control problem of USVs was proposed using the NN weight update law under the persistent excitation condition (PE). In contrast, the NN weight update law in this paper relaxes the PE requirement by incorporating the system’s state into the learning rate, thereby accelerating the convergence of the weight update law.

3. Compared to these control schemes for USVs in [4,11,16], this work proposes an ADP-based prescribed performance controller integrated with the DET mechanism. In contrast to the time-sampling controller in [4,11], and the static event-triggering strategy in [16], the proposed DET scheme further reduces the event-triggering numbers to reduce energy consumption. Moreover, by combining the ADP and PPC techniques, the proposed optimal surrounding controller effectively balances control cost and performance while considering both the steady-state and transient performance of the CLSs.

2. Materials and Methods

This paper investigates the surrounding control problem of multiple unmanned surface vehicles (USVs) against false data injection (FDI) attacks and the control structure diagram of the multi-USVs against FDI attacks is given in Figure 1. The false data are injected into the communication network between USVs to degrade the performance of CLSs. The proposed FDI compensation observer can effectively detect false signals to mitigate the impact of FDI attacks. Ultimately, combining the framework of adaptive dynamic programming and the dynamic event-triggered mechanism, a resilient prescribed performance surrounding control method is designed for multi-USVs.

2.1. Communication Network Diagram of USV

The time-varying surrounding control problem of multi-USVs is investigated in this work. Considering the multi-USVs with N agents, the directed communication network of USVs is defined as $G=\left(\mathcal{V},\epsilon \right)$, where $\mathcal{V}=\{v_1,v_2,\dots v_N\}$ denotes the node set and $\epsilon \subseteq \mathcal{V}\times \mathcal{V}$ is the edge set among the directed graph $G$. The symbol $\mathcal{A}=[a_{ij}]\in {\mathit{R}}^{N\times N}$ is defined as the adjacency matrix of multi-USVs, where $a_{ij}$ denotes the connection weight between the ith USV and jth USV. If $\left(v_j,v_i\right)\in \epsilon$, that means the ith USV can acquire the state of jth USV, $a_{ij}=1$, otherwise $a_{ij}=0$. The diagonal matrix $D=\mathrm{diag}\{d_i\}$ is denoted as the in-degree matrix of USVs, where $d_i={\displaystyle \sum _{j=1}^N{a}_{ij}}$. The Laplacian matrix is given by $L=D-\mathcal{A}$.

Consider the state of target USV as ${\mathit{\eta }}_0$, if the state information of target USV can flow to the ith USV, $b_{i0}=1$, otherwise $b_{i0}=0$. For a digraph $\overline{G}$ with $N+1$ agents, the directed graph $\overline{G}$ contains the target USV and N pursuer USVs. The Laplacian matrix of digraph $\overline{G}$ is defined as $H=L+B$, where $B=\mathrm{diag}\{b_{i0}\}$.

Assumption 1. 

For the digraph $\overline{G}$, there exists at least one path from the target USV to any pursuer USV. Also, a directed spanning tree is rooted at the target USV in the $\overline{G}$.

2.2. The Dynamical Model of USV

The simplified schematic diagram of the unmanned surface vehicle (USV) is shown in Figure 2, where $F_e{O}_e{X}_e{Y}_e$ represents the Earth-fixed coordinate and $F_B{O}_B{X}_B{Y}_B$ denotes the body-fixed coordinate of the USV.

According to references [6,16], the kinematic and dynamic models of the USVs are given as follows:

$$\left\{\begin{array}{l}{\dot{\mathit{\eta }}}_i={\mathit{R}}_i({\phi }_i){\mathit{v}}_i\\ {\dot{\mathit{v}}}_i=-{\mathit{M}}_i^{-1}\left[{\mathit{C}}_i({\mathit{v}}_i){\mathit{v}}_i+{\mathit{D}}_i({\mathit{v}}_i){\mathit{v}}_i-{\mathit{\tau }}_i\right]\end{array}\right.$$

(1)

where ${\mathit{\eta }}_i={\left[x_i,y_i,{\phi }_i\right]}^{\mathrm{T}}$ denotes the state of the ith USV, $(x_i,y_i)$ is the position state and ${\phi }_i$ is the yaw angle. ${\mathit{v}}_i={\left[u_i,v_i,r_i\right]}^{\mathrm{T}}$ is the velocity vector, $u_i, v_i, r_i$ are the surge, sway, yaw velocities, respectively. ${\mathit{\tau }}_i={\left[{\tau }_{u_i},{\tau }_{{\mathit{v}}_i},{\tau }_{r_i}\right]}^{\mathrm{T}}$ is the control input. ${\mathit{M}}_i\in {\mathit{R}}^{3\times 3}$ represents the inertia matrix, satisfying ${\mathit{M}}_i={\mathit{M}}_i^{\mathrm{T}}$, ${\mathit{C}}_i({\mathit{v}}_i)\in {\mathit{R}}^{3\times 3}$ is the skew-symmetric Coriolis force matrix, ${\mathit{C}}_i({\mathit{v}}_i)=-{\mathit{C}}_i{({\mathit{v}}_i)}^{\mathrm{T}}$, and ${\mathit{D}}_i({\mathit{v}}_i)\in {\mathit{R}}^{3\times 3}$ is the damping matrix. ${\mathit{R}}_i({\phi }_i)$ is the rotation matrix from $F_B{O}_B{X}_B{Y}_B$ to $F_e{O}_e{X}_e{Y}_e$, defined as follows:

$${\mathit{R}}_i({\phi }_i)=\left[\begin{array}{ccc}\cos {\phi }_i& -\sin {\phi }_i& 0\\ \sin {\phi }_i& \cos {\phi }_i& 0\\ 0& 0& 1\end{array}\right]$$

(2)

where the rotation matrix has the property ${\mathit{R}}_i^{\mathrm{T}}({\phi }_i){\mathit{R}}_i({\phi }_i)=\mathit{I}$. The matrices ${\mathit{M}}_i$, ${\mathit{C}}_i({\mathit{v}}_i)$, ${\mathit{D}}_i({\mathit{v}}_i)$ are expressed as follows:

$${\mathit{M}}_i=\left[\begin{array}{ccc}{m}_{11i}& 0& 0\\ 0& m_{22i}& m_{23i}\\ 0& m_{23i}& m_{33i}\end{array}\right], {\mathit{C}}_i({\mathit{v}}_i)=\left[\begin{array}{ccc}0& 0& -c_{13i}\\ 0& 0& c_{23i}\\ c_{13i}& -c_{23i}& 0\end{array}\right], {\mathit{D}}_i({\mathit{v}}_i)=\left[\begin{array}{ccc}{d}_{11i}& 0& 0\\ 0& d_{22i}& 0\\ 0& 0& d_{33i}\end{array}\right]$$

(3)

with $m_{11i}=m_i-X_{\dot{u}i}$, $m_{22i}=m_i-Y_{\dot{v}i}$, $m_{33i}=I_{zi}-N_{\dot{r}i}$, $m_{23i}=m_i{x}_{gi}-Y_{\dot{r}i}$, $m_i$ is the mass of the ith USV, $I_{zi}$ denotes the inertia term in the z-axis, $x_{gi}$ is the gravity of the USV, and these variables $X_{\dot{u}i},Y_{\dot{v}i},Y_{\dot{r}i},N_{\dot{r}i}$ are the added masses. $c_{13i}=m_{22i}{v}_i+m_{23i}{r}_i$, $c_{23i}=m_{11i}{u}_i$. $d_{11i}=-(X_{ui}+X_{u\left|u\right|i}\left|u_i\right|)$, $d_{22i}=-(Y_{vi}+Y_{v\left|v\right|i}\left|v_i\right|)$, $d_{33i}=-(N_{ri}+N_{r\left|r\right|i}\left|r_i\right|)$, $X_{ui},Y_{vi},N_{ri}$ are the linear damping coefficients of USV, and $X_{u\left|u\right|i},Y_{v\left|v\right|i},N_{r\left|r\right|i}$ denote the quadratic damping coefficients.

Next, according to the Euler-Lagrange modeling method [36,37], the dynamical model (1) of the USV is transformed as follows:

$${\mathit{v}}_i={\mathit{R}}_i^{-1}{\dot{\mathit{\eta }}}_i$$

(4)

Taking the derivative of Equation (4), we obtain the following equation:

$${\dot{\mathit{v}}}_i={\dot{\mathit{R}}}_i^{-1}{\dot{\mathit{\eta }}}_i+{\mathit{R}}_i^{-1}{\ddot{\mathit{\eta }}}_i$$

(5)

Substituting (5) into (1), we can obtain the following:

$${\mathit{J}}_i{\ddot{\mathit{\eta }}}_i+{\mathit{H}}_i{\dot{\mathit{\eta }}}_i={\mathit{\tau }}_i$$

(6)

where ${\mathit{J}}_i={\mathit{M}}_i{\mathit{R}}_i^{-1}$, ${\mathit{H}}_i=\left({\mathit{M}}_i+{\mathit{C}}_i+{\mathit{D}}_i\right){\mathit{R}}_i^{-1}$.

In this work, the control inputs under the FDI attacks are represented as follows:

$${\tilde{\mathit{\tau }}}_i={\mathit{\tau }}_i+{\mathit{E}}_i{\mathbf{\Gamma }}_i$$

(7)

where ${\mathbf{\Gamma }}_i$ denotes the false information injected by the attacker, ${\mathit{E}}_i=\mathrm{diag}\{{\gamma }_{i1}, {\gamma }_{i2}, {\gamma }_{i3}\}$ is the FDI attacks probability matrix, ${\gamma }_{i1}$, ${\gamma }_{i2}$, and ${\gamma }_{i3}$ denote random variables that follow the Bernoulli distribution. The probability distribution of the random variable satisfies the following:

$$\begin{array}{l}Prob\{{\gamma }_{i1}=1\}={\overline{\gamma }}_{i1},\\ Prob\{{\gamma }_{i2}=1\}={\overline{\gamma }}_{i2},\\ Prob\{{\gamma }_{i3}=1\}={\overline{\gamma }}_{i3},\end{array}$$

(8)

where ${\overline{\gamma }}_{i1}\in [0,1]$, ${\overline{\gamma }}_{i2}\in [0,1]$, and ${\overline{\gamma }}_{i3}\in [0,1]$.

Substituting (7) into (6), we can obtain the dynamic systems of multiple USVs under FDI attacks as follows:

$${\mathit{J}}_i{\ddot{\mathit{\eta }}}_i+{\mathit{H}}_i{\dot{\mathit{\eta }}}_i={\mathit{\tau }}_i+{\mathit{E}}_i{\mathbf{\Gamma }}_i$$

(9)

2.3. The Surrounding Control for Multi-USVs

Definition 1 

[38]. There exists a set  $\mathit{X}=\{x_1,x_2,\dots x_n\}$ that belongs to a convex set  $\mathcal{C}$. In the convex set  $\mathcal{C}$, there exists a convex hull  $Co(\mathit{X})=\left\{{\displaystyle {\sum }_{i=1}^n{\delta }_i{x}_i}|x_i\in \mathit{X},{\displaystyle {\sum }_{i=1}^n{\delta }_i=1}\right\}$.

Definition 2. 

The time-varying surrounding control of multi-USVs is achieved, if for any pursuer USV satisfying the following equation:

$$\underset{t\to \infty }{\lim }‖{\mathit{\eta }}_i-{\mathit{\eta }}_0-{\mathit{\rho }}_{i0}‖=0$$

(10)

where ${\mathit{\eta }}_0$ denotes the state of target USV, the time-varying surrounding function ${\mathit{\rho }}_{i0}={[c_1\cos (c_{2}t+2\pi i/N),c_1\sin (c_{2}t+2\pi i/N),0]}^{\mathrm{T}}$, $c_1,c_2>0$, $i=1,2,\dots N$.

Figure 3 illustrates the process of multiple USVs moving from their initial states to uniformly encircling the target. It can be observed that the target USV is finally distributed into the convex hull spanned by the pursuer USVs.

Lemma 1 

[39]. Consider the dynamic system  $\mathit{x}=\mathit{f}(\mathit{x},\mathit{u})$, where  $\mathit{x}$ represents the system state and  $\mathit{u}$ is the control input. For any positive constants  $a$ and  $0<p<1$, there exists a positive-definite Lyapunov function  $V(t)$ satisfying the following:

$$\dot{V}(t)\le -{\displaystyle \frac{1}{ap{T}_c}}{(1+aV{(t)}^p)}^{2}V{(t)}^{1-p}$$

(11)

with $T_c>0$. The system state $\mathit{x}$ will converge to the origin within the user-predefined time $T_c$.

2.4. Design of a Preset Time Controller Based on ADP and Dynamic Event-Triggering Mechanism

2.4.1. Predefined-Time FDI Attack Observer Design

This work considers the surrounding control problem of multi-USVs under FDI attacks. If the false information in the communication network of USVs is not handled properly, it will lead to the performance degradation of the closed-loop system. To mitigate the impact of FDI attacks, a predefined-time attack compensation observer will be designed for the multi-USVs. One auxiliary variable is defined as follows:

$${\mathit{S}}_i=k_i{\displaystyle {\int }_{t0}^t\left({\mathit{\tau }}_i-{\mathit{H}}_i{\dot{\mathit{\eta }}}_i+{\dot{\mathit{J}}}_i{\dot{\mathit{\eta }}}_i-{\mathit{S}}_i(s)\right)}\mathrm{d}s-k_i{\mathit{J}}_i{\dot{\mathit{\eta }}}_i$$

(12)

where $k_i$ is a positive constant. Taking the time derivative of Equation (12), we obtain the following expression:

$${\dot{\mathit{S}}}_i=-k_i\left({\mathit{S}}_i+{\mathit{F}}_i\right)$$

(13)

where ${\mathit{F}}_i={\mathit{E}}_i{\mathbf{\Gamma }}_i$.

Theorem 1. 

Based on the auxiliary variable ${\mathit{S}}_i$ and its derivative ${\dot{\mathit{S}}}_i$, which are specifically defined in Equations (12) and (13), a predefined-time observer is designed as follows:

$${\dot{\widehat{\mathit{S}}}}_i={\displaystyle \frac{1}{2a_i{p}_i{T}_i}}{\left(1+a_i{L}_i^{p_i}\right)}^2{L}_i^{-p_i}{\tilde{\mathit{S}}}_i+{\dot{\mathit{S}}}_i$$

(14)

with $a_i>0$, $0<p_i<1$, $L_i={\displaystyle \frac{1}{2}}{\tilde{\mathit{S}}}_i^{\mathrm{T}}{\tilde{\mathit{S}}}_i$, ${\tilde{\mathit{S}}}_i={\mathit{S}}_i-{\widehat{\mathit{S}}}_i$. According to Equations (13) and (14), to alleviate the impact of FDI attacks on the closed-loop system of USVs, the following predefined-time FDI observer is given as follows:

$${\widehat{\mathit{F}}}_i=-{\displaystyle \frac{1}{k_i}}\left(k_i{\widehat{\mathit{S}}}_i+{\dot{\mathit{S}}}_i\right)$$

(15)

Under the observer (15), the estimation error ${\tilde{\mathit{F}}}_i={\mathit{F}}_i-{\widehat{\mathit{F}}}_i$ will converge to zero within the predefined time $T_i$.

Proof. 

The function $L_i={\displaystyle \frac{1}{2}}{\tilde{\mathit{S}}}_i^{\mathrm{T}}{\tilde{\mathit{S}}}_i$ is selected as the Lyapunov function, and the ${\dot{L}}_i$ can be obtained as follows:

$$\begin{array}{cc}\hfill {\dot{L}}_i& ={\tilde{\mathit{S}}}_i^{\mathrm{T}}{\dot{\tilde{\mathit{S}}}}_i\hfill \\ & ={\tilde{\mathit{S}}}_i^{\mathrm{T}}\left(-{\displaystyle \frac{1}{2a_i{p}_i{T}_i}}{\left(1+a_i{L}_i^{p_i}\right)}^2{L}_i^{-p_i}{\tilde{\mathit{S}}}_i\right)\hfill \\ & \le -{\displaystyle \frac{1}{a_i{p}_i{T}_i}}{\left(1+a_i{L}_i^{p_i}\right)}^2{L}_i^{1-p_i}\hfill \end{array}$$

(16)

According to Lemma 1, it can be derived that $‖{\tilde{\mathit{S}}}_i‖\to 0$ as the time $t\to T_i$. When $t\ge T_i$, $‖{\tilde{\mathit{S}}}_i‖=0$ can be obtained. Based on Equations (13) and (15), we can obtain the following:

$$\begin{array}{cc}\hfill {\tilde{\mathit{F}}}_i& ={\mathit{F}}_i-{\widehat{\mathit{F}}}_i\hfill \\ & ={\mathit{F}}_i+{\displaystyle \frac{1}{k_i}}\left(k_i{\widehat{\mathit{S}}}_i-k_i\left({\mathit{S}}_i+{\mathit{F}}_i\right)\right)\hfill \\ & =-{\tilde{\mathit{S}}}_i\hfill \end{array}$$

(17)

Based on the above analysis, it can be concluded that the estimation error $‖{\tilde{\mathit{F}}}_i‖$ of the false data will converge to zero within the predefined time $T_i$. □

2.4.2. Tracking Error Subsystem of USVs

Considering the second-order Euler-Lagrange dynamic model of the USV in Equation (6), the following controller is given as follows:

$${\mathit{\tau }}_i={\mathit{\tau }}_{ri}+{\mathit{\tau }}_{fi}$$

(18)

where the ${\mathit{\tau }}_{ri}$ denotes the ADP-based controller, the feedforward control law ${\mathit{\tau }}_{fi}$ is defined as ${\mathit{\tau }}_{fi}={\mathit{H}}_i{\dot{\mathit{\eta }}}_i-{\widehat{\mathit{F}}}_i+{\mathit{J}}_i{\ddot{\mathit{h}}}_{i0}$, and ${\mathit{h}}_{i0}$ is the surrounding position vector of the ith USV relative to the target.

The position vector of the target USV is set as ${\mathit{\eta }}_d={\left[x_d,y_d,{\phi }_d\right]}^{\mathrm{T}}$, and ${\dot{\mathit{v}}}_d={\mathit{\eta }}_d$ is the velocity vector of the target. Next, the surrounding tracking error model for the multi-USVs will be established. The position error ${\mathit{e}}_{i,1}$ of the ith USV relative to its neighbor is formulated as follows:

$${\mathit{e}}_{i,1}={\displaystyle \sum _{j=0}^N{a}_{ij}\left({\mathit{\eta }}_i-{\mathit{\eta }}_j-{\mathit{h}}_{ij}\right)}$$

(19)

where ${\mathit{h}}_{ij}={\mathit{h}}_{i0}-{\mathit{h}}_{j0}$ denotes the relative position between ith USV and jth USV, ${\mathit{\eta }}_0={\mathit{\eta }}_d$.

The velocity tracking error ${\mathit{e}}_{i,2}$ is defined as follows:

$${\dot{\mathit{e}}}_{i,1}={\mathit{e}}_{i,2}={\displaystyle \sum _{j=0}^N{a}_{ij}\left({\dot{\mathit{\eta }}}_i-{\dot{\mathit{\eta }}}_j-{\dot{\mathit{h}}}_{ij}\right)}$$

(20)

Take the derivative of Equation (20) with respect to time, it can yield the following:

$$\begin{array}{cc}\hfill {\dot{\mathit{e}}}_{i,2}& ={\displaystyle \sum _{j=0}^N{a}_{ij}\left({\ddot{\mathit{\eta }}}_i-{\ddot{\mathit{\eta }}}_j-{\ddot{\mathit{h}}}_{ij}\right)}\hfill \\ & ={\displaystyle \sum _{j=0}^N{a}_{ij}\left({\ddot{\mathit{\eta }}}_i-{\ddot{\mathit{\eta }}}_j-{\ddot{\mathit{h}}}_{ij}\right)}\hfill \\ & ={\displaystyle \sum _{j=0}^N{a}_{ij}\left({\mathit{J}}_i^{-1}{\mathit{\tau }}_{ri}-{\mathit{J}}_j^{-1}{\mathit{\tau }}_{rj}+{\mathit{J}}_i^{-1}{\mathit{F}}_i-{\mathit{J}}_i^{-1}{\widehat{\mathit{F}}}_i-\left({\mathit{J}}_j^{-1}{\mathit{F}}_j-{\mathit{J}}_j^{-1}{\widehat{\mathit{F}}}_j\right)\right)}\hfill \\ & ={\displaystyle \sum _{j=0}^N{a}_{ij}\left({\mathit{\tau }}_{Ri}-{\mathit{\tau }}_{Rj}+{\mathit{J}}_i^{-1}{\tilde{\mathit{F}}}_i-{\mathit{J}}_j^{-1}{\tilde{\mathit{F}}}_j\right)}\hfill \end{array}$$

(21)

with ${\mathit{\tau }}_{Ri}={\mathit{J}}_i^{-1}{\mathit{\tau }}_{ri}$, ${\mathit{\tau }}_{Rj}={\mathit{J}}_j^{-1}{\mathit{\tau }}_{rj}$, ${\mathit{\tau }}_{R0}={\ddot{\mathit{\eta }}}_d$. One can determine that ${\tilde{\mathit{F}}}_i=0_3$ and ${\tilde{\mathit{F}}}_j=0_3$ when time $t\ge T_i$. According to (20) and (21), a combined dynamic system is defined as follows:

$${\dot{\mathit{e}}}_i=A_i{\mathit{e}}_i+B_i{\mathit{\tau }}_{Ri}-B_i{\mathit{\tau }}_{Rj}$$

(22)

where ${\mathit{e}}_i={\left[{\mathit{e}}_{i,1}^{\mathrm{T}},{\mathit{e}}_{i,2}^{\mathrm{T}}\right]}^{\mathrm{T}}$, ${\mathit{e}}_{i,1}={\left[e_{i1},e_{i2},e_{i3}\right]}^{\mathrm{T}}$, ${\mathit{e}}_{i,2}={\left[e_{i4},e_{i5},e_{i6}\right]}^{\mathrm{T}}$, $A_i=\left[\begin{array}{cc}{0}_{3\times 3}& I_{3\times 3}\\ 0_{3\times 3}& 0_{3\times 3}\end{array}\right]$, $B_i=\left[\begin{array}{c}{0}_{3\times 3}\\ I_{3\times 3}{\displaystyle \sum _{j=0}^N{a}_{ij}}\end{array}\right]$.

In this paper, to ensure that the surrounding tracking error of USVs has both great steady-state and transient performance, the following prescribed performance function is designed:

$${\mathit{\lambda }}_i(t)=\left({\mathit{\lambda }}_i(0)-{\mathit{\lambda }}_{i,\infty }\right)\exp (-l_{i}t)+{\mathit{\lambda }}_{i,\infty }$$

(23)

where ${\mathit{\lambda }}_i(t)={\left[{\lambda }_{i1},{\lambda }_{i2},\dots {\lambda }_{i6}\right]}^{\mathrm{T}}$, $l_i>0$, $0<\left|e_{ik}(0)\right|<{\lambda }_{ik}(0)$, $0<{\lambda }_{ik,\infty }<{\lambda }_{ik}(0)$, $k=1, \dots 6$. The surrounding tracking error ${\mathit{e}}_i$ can be expressed as follows:

$${\mathit{e}}_i(t)={\mathit{\lambda }}_i(t){\mathbf{\Gamma }}_i({\mathit{z}}_i)$$

(24)

where ${\mathbf{\Gamma }}_i({\mathit{z}}_i)$ is a monotonically increasing smooth function such that $\underset{z_i\to +\infty }{\lim }{\mathbf{\Gamma }}_i({\mathit{z}}_i)=\mathbf{1}$, $\underset{z_i\to -\infty }{\lim }{\mathbf{\Gamma }}_i({\mathit{z}}_i)=-\mathbf{1}$. The mapping function ${\mathbf{\Gamma }}_i({\mathit{z}}_i)$ is designed as follows:

$${\mathbf{\Gamma }}_i({\mathit{z}}_i)={\displaystyle \frac{\exp ({\mathit{z}}_i)-\exp (-{\mathit{z}}_i)}{\exp ({\mathit{z}}_i)+\exp (-{\mathit{z}}_i)}}$$

(25)

Because $-\mathbf{1}<{\mathbf{\Gamma }}_i({\mathit{z}}_i)<\mathbf{1}$, it can be deduced that $-{\mathit{\lambda }}_i(t)<{\mathit{\lambda }}_i(t){\mathbf{\Gamma }}_i({\mathit{z}}_i)<{\mathit{\lambda }}_i(t)$ from Equation (25). Furthermore, we can obtain the following:

$$-{\mathit{\lambda }}_i(t)<{\mathit{e}}_i(t_i)<{\mathit{\lambda }}_i(t)$$

(26)

From (26), the inverse function of the variable ${\mathbf{\Gamma }}_i({\mathit{z}}_i)$ is given as follows:

$${\mathit{z}}_i={\displaystyle \frac{1}{2}}\ln {\displaystyle \frac{1+{\mathbf{\Gamma }}_i({\mathit{z}}_i)}{1-{\mathbf{\Gamma }}_i({\mathit{z}}_i)}}={\displaystyle \frac{1}{2}}\ln {\displaystyle \frac{1+({\mathit{e}}_i/{\mathit{\lambda }}_i)}{1-({\mathit{e}}_i/{\mathit{\lambda }}_i)}}$$

(27)

The time derivative of Equation (27) can be given as follows:

$${\dot{\mathit{z}}}_i={\displaystyle \frac{1}{2}}\left({\displaystyle \frac{{\dot{\mathit{\lambda }}}_i+{\dot{\mathit{e}}}_i}{{\mathit{\lambda }}_i+{\mathit{e}}_i}}-{\displaystyle \frac{{\dot{\mathit{\lambda }}}_i-{\dot{\mathit{e}}}_i}{{\mathit{\lambda }}_i-{\mathit{e}}_i}}\right)={\mathit{G}}_{i1}({\dot{\mathit{e}}}_i-{\mathit{G}}_{i2}{\mathit{e}}_i)$$

(28)

with ${\mathit{G}}_{i1}=\mathrm{diag}\left\{G_{i11},G_{i12},\dots ,G_{i16}\right\}$, $G_{i1k}={\lambda }_{ik}/({\lambda }_{ik}^2-e_{ik}^2)$, ${\mathit{G}}_{i2}=\mathrm{diag}\left\{G_{i21},G_{i22},\dots ,G_{i26}\right\}$, $G_{i2k}={\dot{\lambda }}_{ik}/{\lambda }_{ik}$, $k=1, \dots 6$. Substituting Equation (20) into (28), we obtain the following:

$${\dot{\mathit{z}}}_i=\mathcal{F}({\mathit{z}}_i)+{\overline{B}}_i{\mathit{\tau }}_{Ri}-{\overline{B}}_i{\mathit{\tau }}_{Rj}$$

(29)

where $\mathcal{F}({\mathit{z}}_i)={\mathit{G}}_{i1}(A_i-{\mathit{G}}_{i2}){\mathit{e}}_i$, ${\overline{B}}_i={\mathit{G}}_{i1}{B}_i$. According to (24) and (27), $\mathcal{F}({\displaystyle \mathbf{0}})={\displaystyle \mathbf{0}}$ can be derived.

2.4.3. ADP-Based Optimal Time-Varying Surrounding Control

To save the energy of the USV, this paper adopts a dynamic event-triggered control strategy to reduce communication frequency and state sampling times between USVs. The sampling time sequence for the ith USV is defined as $t_s^i$, satisfying $t_s^i<t_{s+1}^i$, $s=0, \dots \infty$. The sampling error ${\mathit{q}}_i(t)$ of ith USV is designed as follows:

$${\mathit{q}}_i(t)={\mathit{z}}_i(t_s^i)-{\mathit{z}}_i(t),t\in \left[t_s^i,t_{s+1}^i\right]$$

(30)

where ${\mathit{z}}_i(t_s^i)$ represents the previous sampling time $t_s^i$ of ith USV. To facilitate the design of the event-triggered control strategy in the following discussion, we define ${\widehat{\mathit{z}}}_i={\mathit{z}}_i(t_s^i)$. The performance index function for the ith USV is designed as follows:

$${\mathcal{J}}_i({\mathit{z}}_i)={\displaystyle {\int }_0^{\infty }\left[{\mathit{z}}_i^{\mathrm{T}}{Q}_i{\mathit{z}}_i+{\mathit{\tau }}_{Ri}^{\mathrm{T}}{R}_i{\mathit{\tau }}_{Ri}\right]} \mathrm{d}t$$

(31)

where $Q_i=Q_i^{\mathrm{T}}>0$ and $R_i=R_i^{\mathrm{T}}>0$. If the ith USV has an optimal control scheme ${\mathit{\tau }}_{Ri}^{\ast }$, the cost function in Equation (27) can be transformed into the following form:

$${\mathcal{J}}_i^{\ast }({\mathit{z}}_i)={\displaystyle {\int }_0^{\infty }\left[{\mathit{z}}_i^{\mathrm{T}}{Q}_i{\mathit{z}}_i+{({\mathit{\tau }}_{Ri}^{\ast })}^{\mathrm{T}}{R}_i{\mathit{\tau }}_{Ri}^{\ast }\right]} \mathrm{d}t$$

(32)

We conduct the time derivative of the optimal cost function ${\mathcal{J}}_i^{\ast }({\mathit{z}}_i)$ in (32), it yields the following:

$$\begin{array}{cc}\hfill H\left({\mathit{z}}_i,{\mathit{\tau }}_{Ri}^{\ast },{\nabla }_{z_i}{\mathcal{J}}_i^{\ast }({\mathit{z}}_i)\right)& ={\mathit{z}}_i^{\mathrm{T}}{Q}_i{\mathit{z}}_i+{({\mathit{\tau }}_{Ri}^{\ast })}^{\mathrm{T}}{R}_i{\mathit{\tau }}_{Ri}^{\ast }+{\nabla }_{z_i}^{\mathrm{T}}{\mathcal{J}}_i^{\ast }({\mathit{z}}_i)(\mathcal{F}({\mathit{z}}_i)+{\overline{B}}_i{\mathit{\tau }}_{Ri}^{\ast }-{\overline{B}}_i{\mathit{\tau }}_{Rj}^{\ast })\hfill \\ & =0\hfill \end{array}$$

(33)

where ${\nabla }_{z_i}{\mathcal{J}}_i^{\ast }({\mathit{z}}_i)=\partial {\mathcal{J}}_i^{\ast }({\mathit{z}}_i)/\partial {\mathit{z}}_i$ is the partial derivative of ${\mathcal{J}}_i^{\ast }({\mathit{z}}_i)$ with respect to ${\mathit{z}}_i$. Using Equation (33), the optimal control strategy ${\mathit{\tau }}_{Ri}^{\ast }$ is obtained as follows:

$${\mathit{\tau }}_{Ri}^{\ast }=-{\displaystyle \frac{1}{2}}{R}_i^{-1}{\overline{B}}_i^T{\nabla }_{z_i}{\mathcal{J}}_i^{\ast }({\mathit{z}}_i)$$

(34)

According to the definition of ${\mathit{z}}_i(t_s^i)$ in (30), the event-triggered optimal controller is obtained as follows:

$${\widehat{\mathit{\tau }}}_{Ri}^{\ast }({\widehat{\mathit{z}}}_i)=-{\displaystyle \frac{1}{2}}{R}_i^{-1}{\overline{B}}_i^T{\nabla }_{z_i}{\mathcal{J}}_i^{\ast }({\widehat{\mathit{z}}}_i)$$

(35)

Substituting (35) into (33), the HJB equation of ith USV under the event-triggered strategy is given as follows:

$$\begin{array}{l}{\mathit{z}}_i^{\mathrm{T}}{Q}_i{\mathit{z}}_i+{\displaystyle \frac{1}{4}}{(\nabla {\mathcal{J}}_i^{\ast }({\widehat{\mathit{z}}}_i))}^{\mathrm{T}}{\overline{B}}_i{R}_i^{-1}{\overline{B}}_i^{\mathrm{T}}\nabla {\mathcal{J}}_i^{\ast }({\widehat{\mathit{z}}}_i)+{(\nabla {\mathcal{J}}_i^{\ast }({\mathit{z}}_i))}^{\mathrm{T}}\mathcal{F}({\mathit{z}}_i)\\ -{\displaystyle \frac{1}{2}}{(\nabla {\mathcal{J}}_i^{\ast }({\mathit{z}}_i))}^{\mathrm{T}}{\overline{B}}_i{R}_i^{-1}{\overline{B}}_i^{\mathrm{T}}\nabla {\mathcal{J}}_i^{\ast }({\widehat{\mathit{z}}}_i)+{\displaystyle \frac{1}{2}}\nabla {\mathcal{J}}_i^{\ast }({\mathit{z}}_i){)}^{\mathrm{T}}{\overline{B}}_i{R}_j^{-1}{\overline{B}}_j^{\mathrm{T}}\nabla {\mathcal{J}}_j^{\ast }({\mathit{z}}_j)=0\end{array}$$

(36)

Based on the above analysis, we can obtain the event-triggered optimal control strategy ${\widehat{\mathit{\tau }}}_{Ri}^{\ast }({\widehat{\mathit{z}}}_i)$ by solving the HJB Equation (36). However, the HJB equation is highly nonlinear and contains partial differential terms, making it difficult to obtain an analytical solution. Therefore, we use a neural network to approximate the optimal cost function, which can be formulated as follows:

$${\mathcal{J}}_i^{\ast }({\mathit{z}}_i)={({\mathbb{W}}_i^{\ast })}^{\mathrm{T}}{\mathit{\varphi }}_i({\mathit{z}}_i)+{\epsilon }_i({\mathit{z}}_i)$$

(37)

where ${\mathbb{W}}_i\in {ℝ}^{6\times 1}$ is the optimal weight vector of the NN, ${\mathit{\varphi }}_i({\mathit{z}}_i)\in {ℝ}^{6\times 1}$ represents the activation function, and ${\epsilon }_i({\mathit{z}}_i)$ is the approximation error of the NN. Utilizing (37), the optimal event-triggered control strategy in (35) is re-expressed as follows:

$${\widehat{\mathit{\tau }}}_{Ri}^{\ast }({\widehat{\mathit{z}}}_i)=-{\displaystyle \frac{1}{2}}{R}_i^{-1}{\overline{B}}_i^T\left(\nabla {\mathit{\varphi }}_i({\widehat{\mathit{z}}}_i){\mathbb{W}}_i^{\ast }+\nabla {\epsilon }_i({\widehat{\mathit{z}}}_i)\right)$$

(38)

The optimal weight vector ${\mathbb{W}}_i^{\ast }$ is unknown, which makes it impossible to directly obtain the optimal cost function ${\mathcal{J}}_i^{\ast }({\mathit{z}}_i)$. Then, we design the ${\widehat{\mathbb{W}}}_i$ to estimate the ${\mathbb{W}}_i^{\ast }$, and ${\widehat{\mathcal{J}}}_i({\mathit{z}}_i)={\widehat{\mathbb{W}}}_i^{\mathrm{T}}{\mathit{\varphi }}_i({\mathit{z}}_i)$ is defined as the estimation of ${\mathcal{J}}_i^{\ast }({\mathit{z}}_i)$. Therefore, the actual optimal control strategy ${\widehat{\mathit{\tau }}}_{Ri}({\widehat{\mathit{z}}}_i)$ is given as follows:

$${\widehat{\mathit{\tau }}}_{Ri}({\widehat{\mathit{z}}}_i)=-{\displaystyle \frac{1}{2}}{R}_i^{-1}{\overline{B}}_i^T\nabla {\mathit{\varphi }}_i({\widehat{\mathit{z}}}_i){\widehat{\mathbb{W}}}_i$$

(39)

According to (33) and ${\widehat{\mathcal{J}}}_i({\mathit{z}}_i)={\widehat{\mathbb{W}}}_i^{\mathrm{T}}{\mathit{\varphi }}_i({\mathit{z}}_i)$, the Hamiltonian function can be rewritten as follows:

$$H\left({\mathit{z}}_i,{\widehat{\mathit{\tau }}}_{Ri},{\widehat{\mathbb{W}}}_i\right)={\mathit{z}}_i^{\mathrm{T}}{Q}_i{\mathit{z}}_i+{({\widehat{\mathit{\tau }}}_{Ri})}^{\mathrm{T}}{R}_i{\widehat{\mathit{\tau }}}_{Ri}+{\widehat{\mathbb{W}}}_i^{\mathrm{T}}{\nabla }_{z_i}^{\mathrm{T}}{\mathit{\varphi }}_i({\mathit{z}}_i){\dot{\mathit{z}}}_i={𝔼}_i$$

(40)

Furthermore, based on Equations (33), (37) and (40), we can obtain the following:

$$H\left({\mathit{z}}_i,{\widehat{\mathit{\tau }}}_{Ri},{\mathbb{W}}_i^{\ast }\right)={\mathit{z}}_i^{\mathrm{T}}{Q}_i{\mathit{z}}_i+{({\widehat{\mathit{\tau }}}_{Ri})}^{\mathrm{T}}{R}_i{\widehat{\mathit{\tau }}}_{Ri}+{({\mathbb{W}}_i^{\ast })}^{\mathrm{T}}{\nabla }_{z_i}^{\mathrm{T}}{\mathit{\varphi }}_i({\mathit{z}}_i){\dot{\mathit{z}}}_i={𝔼}_{iH}$$

(41)

where ${𝔼}_{iH}=-{(\nabla {\epsilon }_i({\mathit{z}}_i))}^{\mathrm{T}}{\dot{\mathit{z}}}_i$. According to (40) and (41), we can obtain the following:

$${𝔼}_{iH}-{𝔼}_i={\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\varpi }_i$$

(42)

with ${\varpi }_i={\nabla }_{z_i}^{\mathrm{T}}{\mathit{\varphi }}_i({\mathit{z}}_i){\dot{\mathit{z}}}_i$ and ${\tilde{\mathbb{W}}}_i={\mathbb{W}}_i^{\ast }-{\widehat{\mathbb{W}}}_i$.

Assumption 2. 

The activation function of the NN and the approximation error are both bounded, satisfying $‖\nabla {\mathit{\varphi }}_i‖\le {\overline{\varphi }}_i$, $‖\nabla {\epsilon }_i‖\le {\overline{\epsilon }}_i$, where ${\overline{\varphi }}_i$ and ${\overline{\epsilon }}_i$ are constant values.

Assumption 3. 

The activation function $\nabla {\mathit{\varphi }}_i^T({\mathit{z}}_i)$ satisfies the Lipschitz continuous [40]; we obtain the following:

$$‖\nabla {\mathit{\varphi }}_i^T({\mathit{z}}_i)-\nabla {\mathit{\varphi }}_i^T({\widehat{\mathit{z}}}_i)‖\le c_i‖{\mathit{q}}_i(t)‖$$

(43)

where $c_i$ is a positive constant.

Theorem 2. 

Considering the transformed subsystem (29) of the USVs, an event-triggered control strategy (39) based on reinforcement learning is designed. The dynamic event-triggering condition is designed as follows:

$$t_{s+1}^i=\sup \left\{t>t_s^i\left|{\mathcal{l}}_4({\mathcal{l}}_3{‖{\mathit{q}}_i‖}^2-\sigma {\mathcal{l}}_1{‖{\mathit{z}}_i‖}^2-\sigma {\mathcal{l}}_2{‖{\widehat{\mathit{\tau }}}_{Ri}‖}^2-\exp (-‖{\mathit{z}}_i‖))\le {\delta }_i\right.\right\}$$

(44)

where ${\mathcal{l}}_1={\lambda }_{\min }(Q_i)$, ${\mathcal{l}}_2={\lambda }_{\min }(R_i)$, ${\mathcal{l}}_4>0$, ${\mathcal{l}}_3=c_i^2‖R_i^{-1}‖{‖{\overline{B}}_i^{\mathrm{T}}‖}^2{‖{\widehat{\mathbb{W}}}_i‖}^2$, $\sigma \in \left(0,1\right)$. The dynamic variable ${\delta }_i$ in (44) is given as follows:

$${\dot{\delta }}_i=-{\beta }_i{\delta }_i+{\xi }_i(\sigma {\mathcal{l}}_1{‖{\mathit{z}}_i‖}^2+\sigma {\mathcal{l}}_2{‖{\widehat{\mathit{\tau }}}_{Ri}‖}^2+\exp (-‖{\mathit{z}}_i‖)-{\mathcal{l}}_3{‖{\mathit{q}}_i‖}^2)$$

(45)

where ${\delta }_i(0)>0$, ${\beta }_i>0$, ${\xi }_i\in \left[0,1\right]$. The NN weight update law without PE condition is given as follows:

$${\dot{\widehat{\mathbb{W}}}}_i=-{\displaystyle \frac{{\varsigma }_2}{{\varsigma }_1}}\left(\exp (-{\varsigma }_3/‖{\mathit{z}}_i‖)+{\varsigma }_4\right){\widehat{\mathbb{W}}}_i-{\displaystyle \frac{{\varsigma }_5{𝔼}_i{\varpi }_i}{{\varsigma }_1{({\varpi }_i^{\mathrm{T}}{\varpi }_i+1)}^2}}$$

(46)

with ${\varsigma }_1$, ${\varsigma }_2$, ${\varsigma }_3$, ${\varsigma }_4$ and ${\varsigma }_5$ being positive constants. The state tracking error ${\mathit{z}}_i$ and the NN weight estimation error ${\tilde{\mathbb{W}}}_i={\mathbb{W}}_i^{\ast }-{\widehat{\mathbb{W}}}_i$ are ensured to be uniformly ultimately bounded (UUB) under the optimal control law (39) and the weight update law (46). Furthermore, the proposed dynamic event-triggered control strategy does not exhibit Zeno behavior.

Proof. 

According to Equations (44) and (45), we can obtain ${\dot{\delta }}_i\ge -{\beta }_i{\delta }_i-{\displaystyle \frac{{\xi }_i}{{\mathcal{l}}_4}}{\delta }_i=-({\beta }_i+{\displaystyle \frac{{\xi }_i}{{\mathcal{l}}_4}}){\delta }_i$. Furthermore, from one, we obtain ${\delta }_i(t)\ge 0$. Based on the above analysis, a positive-definite Lyapunov function for the ith USV is chosen as follows:

$$V_i={\mathcal{J}}_i^{\ast }+{\displaystyle \frac{1}{2}}{\varsigma }_1{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\tilde{\mathbb{W}}}_i+{\delta }_i$$

(47)

The following analysis will be divided into two parts: (1) the stability analysis; (2) the exclusion of Zeno behavior.

(1) Stability Analysis: The time derivative of the Lyapunov function $V_i$ is given as follows:

$${\dot{V}}_i={(\nabla {\mathcal{J}}_i^{\ast })}^{\mathrm{T}}(\mathcal{F}({\mathit{z}}_i)+{\overline{B}}_i{\widehat{\mathit{\tau }}}_{Ri}-{\overline{B}}_i{\widehat{\mathit{\tau }}}_{Rj})+{\varsigma }_1{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\dot{\tilde{\mathbb{W}}}}_i+{\dot{\delta }}_i$$

(48)

According to Equations (33), (46) and (48) can be further derived into the following form:

$$\begin{array}{cc}\hfill {\dot{V}}_i=& -{\mathit{z}}_i^{\mathrm{T}}{Q}_i{\mathit{z}}_i+{({\mathit{\tau }}_{Ri}^{\ast })}^{\mathrm{T}}{R}_i{\mathit{\tau }}_{Ri}^{\ast }-2{({\mathit{\tau }}_{Ri}^{\ast })}^{\mathrm{T}}{R}_i{\hat{\mathit{\tau }}}_{Ri}+{(\nabla {\mathcal{J}}_i^{\ast })}^{\mathrm{T}}{\overline{B}}_i({\mathit{\tau }}_{Rj}^{\ast }-{\hat{\mathit{\tau }}}_{Rj})\hfill \\ & -{\varsigma }_1{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\dot{\hat{\mathbb{W}}}}_i+{\dot{\delta }}_i\hfill \end{array}$$

(49)

According to [18], the ideal weight ${\mathbb{W}}_i^{\ast }$ is bounded. Furthermore, we can obtain $‖{(\nabla {\mathcal{J}}_i^{\ast })}^{\mathrm{T}}{\overline{B}}_i({\mathit{\tau }}_{Rj}^{\ast }-{\widehat{\mathit{\tau }}}_{Rj})‖\le {\upsilon }_j$, where ${\upsilon }_j>0$. Based on the weight update law (46) and Equation (49), the following inequality can be obtained:

$$\begin{array}{cc}\hfill -{\varsigma }_1{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\dot{\widehat{\mathbb{W}}}}_i& ={\chi }_{1i}{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\widehat{\mathbb{W}}}_i+{\chi }_{2i}{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{𝔼}_i{\varpi }_i\hfill \\ & ={\chi }_{1i}{\tilde{\mathbb{W}}}_i^{\mathrm{T}}\left({\mathbb{W}}_i^{\ast }-{\tilde{\mathbb{W}}}_i\right)+{\chi }_{2i}{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\varpi }_i\left({𝔼}_{iH}-{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\varpi }_i\right)\hfill \\ & \le -{\displaystyle \frac{{\chi }_{1i}}{2}}{‖{\tilde{\mathbb{W}}}_i‖}^2-{\displaystyle \frac{{\chi }_{2i}}{2}}{‖{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\varpi }_i‖}^2+{\displaystyle \frac{{\chi }_{1i}}{2}}{‖{\mathbb{W}}_i^{\ast }‖}^2+{\displaystyle \frac{{\chi }_{2i}}{2}}{‖{𝔼}_{iH}‖}^2\hfill \\ & \le -{\displaystyle \frac{{\chi }_{1i}}{2}}{‖{\tilde{\mathbb{W}}}_i‖}^2-{\displaystyle \frac{{\chi }_{2i}}{2}}{‖{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\varpi }_i‖}^2+{\displaystyle \frac{{\chi }_{1i}}{2}}{\overline{\mathbb{W}}}_i^2+{\displaystyle \frac{{\chi }_{2i}}{2}}{‖{\overline{𝔼}}_{iH}‖}^2\hfill \end{array}$$

(50)

where ${\chi }_{1i}={\varsigma }_2\left(\exp (-{\varsigma }_3/‖{\mathit{z}}_i‖)+{\varsigma }_4\right)$ and ${\chi }_{2i}={\varsigma }_5/{({\varpi }_i^{\mathrm{T}}{\varpi }_i+1)}^2$. ${\overline{\mathbb{W}}}_i$ is the upper bound of the ideal weight $‖{\mathbb{W}}_i^{\ast }‖$, and ${\overline{𝔼}}_{iH}$ is the upper bound of ${𝔼}_{iH}$ Substituting (50) into (49), we obtain the following:

$$\begin{array}{c}{\dot{V}}_i\le -{\mathit{z}}_i^{\mathrm{T}}{Q}_i{\mathit{z}}_i+{({\mathit{\tau }}_{Ri}^{\ast }-{\widehat{\mathit{\tau }}}_{Ri})}^{\mathrm{T}}{R}_i({\mathit{\tau }}_{Ri}^{\ast }-{\widehat{\mathit{\tau }}}_{Ri})-{({\widehat{\mathit{\tau }}}_{Ri})}^{\mathrm{T}}{R}_i{\widehat{\mathit{\tau }}}_{Ri}\\ -{\displaystyle \frac{{\chi }_{1i}}{2}}{‖{\tilde{\mathbb{W}}}_i‖}^2-{\displaystyle \frac{{\chi }_{2i}}{2}}{‖{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\varpi }_i‖}^2+{\displaystyle \frac{{\chi }_{1i}}{2}}{\overline{\mathbb{W}}}_i^2+{\displaystyle \frac{{\chi }_{2i}}{2}}{‖{\overline{𝔼}}_{iH}‖}^2+{\dot{\delta }}_i\end{array}$$

(51)

According to Equations (34) and (37), the time-sampling-based control strategy ${\mathit{\tau }}_{Ri}^{\ast }$ is obtained as follows:

$${\mathit{\tau }}_{Ri}^{\ast }=-{\displaystyle \frac{1}{2}}{R}_i^{-1}{\overline{B}}_i^T\left[\nabla {\mathit{\varphi }}_i^{\mathrm{T}}({\mathit{z}}_i){\mathbb{W}}_i^{\ast }+\nabla {\epsilon }_i({\mathit{z}}_i)\right]$$

(52)

And further from (39) and (52), the following inequality can be derived as follows:

$$\begin{array}{cc}\hfill {({\mathit{\tau }}_{Ri}^{\ast }-{\widehat{\mathit{\tau }}}_{Ri})}^{\mathrm{T}}{R}_i({\mathit{\tau }}_{Ri}^{\ast }-{\widehat{\mathit{\tau }}}_{Ri})& \le ‖R_i‖{‖{\mathit{\tau }}_{Ri}^{\ast }-{\widehat{\mathit{\tau }}}_{Ri}‖}^2\hfill \\ & \le ‖R_i‖{‖R_i^{-1}{\overline{B}}_i^{\mathrm{T}}\left(\nabla {\mathit{\varphi }}_i^{\mathrm{T}}({\mathit{z}}_i)-\nabla {\mathit{\varphi }}_i^{\mathrm{T}}({\widehat{\mathit{z}}}_i)\right){\widehat{\mathbb{W}}}_i‖}^2\hfill \\ & +‖R_i‖{‖R_i^{-1}{\overline{B}}_i^{\mathrm{T}}\left(\nabla {\mathit{\varphi }}_i^{\mathrm{T}}({\mathit{z}}_i){\tilde{\mathbb{W}}}_i+\nabla {\epsilon }_i({\mathit{z}}_i)\right)‖}^2\hfill \\ & \le c_i^2‖R_i^{-1}‖{‖{\overline{B}}_i^{\mathrm{T}}‖}^2{‖{\widehat{\mathbb{W}}}_i‖}^2{‖{\mathit{q}}_i‖}^2+2‖R_i^{-1}‖{‖{\overline{B}}_i^{\mathrm{T}}‖}^2{\overline{\varphi }}_i^2{‖{\tilde{\mathbb{W}}}_i‖}^2\hfill \\ & +2‖R_i^{-1}‖{‖{\overline{B}}_i^{\mathrm{T}}‖}^2{\overline{\epsilon }}_i^2\hfill \end{array}$$

(53)

Substituting (53) into (52), Equation (51) can be reformulated as follows:

$$\begin{array}{cc}\hfill {\dot{V}}_i& \le -{\lambda }_{\min }(Q_i){‖{\mathit{z}}_i‖}^2+c_i^2‖R_i^{-1}‖{‖{\overline{B}}_i^{\mathrm{T}}‖}^2{‖{\widehat{\mathbb{W}}}_i‖}^2{‖{\mathit{q}}_i‖}^2+2‖R_i^{-1}‖{‖{\overline{B}}_i^{\mathrm{T}}‖}^2{\overline{\varphi }}_i^2{‖{\tilde{\mathbb{W}}}_i‖}^2+2‖R_i^{-1}‖{‖{\overline{B}}_i^{\mathrm{T}}‖}^2{\overline{\epsilon }}_i^2\hfill \\ & -{\lambda }_{\min }(R_i){‖{\widehat{\mathit{\tau }}}_{Ri}‖}^2-{\displaystyle \frac{{\chi }_{1i}}{2}}{‖{\tilde{\mathbb{W}}}_i‖}^2-{\displaystyle \frac{{\chi }_{2i}}{2}}{‖{\tilde{\mathbb{W}}}_i^{\mathrm{T}}{\varpi }_i‖}^2+{\displaystyle \frac{{\chi }_{1i}}{2}}{\overline{\mathbb{W}}}_i^2+{\displaystyle \frac{{\chi }_{2i}}{2}}{‖{\overline{𝔼}}_{iH}‖}^2+{\dot{\delta }}_i\hfill \\ & \le -{\mathcal{l}}_1{‖{\mathit{z}}_i‖}^2-{\mathcal{l}}_2{‖{\widehat{\mathit{\tau }}}_{Ri}‖}^2+{\mathcal{l}}_3{‖{\mathit{q}}_i‖}^2-{\vartheta }_1{‖{\tilde{\mathbb{W}}}_i‖}^2+{\vartheta }_2+{\dot{\delta }}_i\hfill \end{array}$$

(54)

where ${\vartheta }_1$ and ${\vartheta }_2$ are shown as follows:

$$\left\{\begin{array}{l}{\vartheta }_1={\displaystyle \frac{{\chi }_{1i}}{2}}+{\displaystyle \frac{{\chi }_{2i}{\varpi }_i^{\mathrm{T}}{\varpi }_i}{2}}-2‖R_i^{-1}‖{‖{\overline{B}}_i^{\mathrm{T}}‖}^2{\overline{\varphi }}_i^2>0\\ {\vartheta }_2=2‖R_i^{-1}‖{‖{\overline{B}}_i^{\mathrm{T}}‖}^2{\overline{\epsilon }}_i^2+{\displaystyle \frac{{\chi }_{1i}}{2}}{\overline{\mathbb{W}}}_i^2+{\displaystyle \frac{{\chi }_{2i}}{2}}{\overline{𝔼}}_{iH}^2\end{array}\right.$$

(55)

Based on the dynamic event-triggering conditions (44) and (45), Equation (54) can be re-expressed in the following form:

$$\begin{array}{cc}\hfill {\dot{V}}_i& \le -(1-{\xi }_i\sigma ){\mathcal{l}}_1{‖{\mathit{z}}_i‖}^2-(1-{\xi }_i\sigma ){\mathcal{l}}_2{‖{\widehat{\mathit{\tau }}}_{Ri}‖}^2+(1-{\xi }_i\sigma ){\mathcal{l}}_3{‖{\mathit{q}}_i‖}^2\hfill \\ & +{\xi }_i\exp (-‖{\mathit{z}}_i‖)-{\vartheta }_1{‖{\tilde{\mathbb{W}}}_i‖}^2+{\vartheta }_2-{\beta }_i{\delta }_i\hfill \\ & \le -(1-\sigma ){\mathcal{l}}_1{‖{\mathit{z}}_i‖}^2-(1-\sigma ){\mathcal{l}}_2{‖{\widehat{\mathit{\tau }}}_{Ri}‖}^2+\left({\displaystyle \frac{1-{\xi }_i}{{\mathcal{l}}_4}}-{\beta }_i\right){\delta }_i-{\vartheta }_1{‖{\tilde{\mathbb{W}}}_i‖}^2+{\overline{\vartheta }}_2\hfill \end{array}$$

(56)

where ${\mathcal{l}}_4>(1-{\xi }_i)/{\beta }_i$, ${\xi }_i\exp (-‖{\mathit{z}}_i‖)+{\vartheta }_2\le {\overline{\vartheta }}_2$. Based on the above analysis, it is known that the variables ${\mathit{z}}_i$ and ${\tilde{\mathbb{W}}}_i$ are UUB. It is reasonable to assume that the position tracking error is $\underset{t\to 0}{\lim }‖{\mathit{e}}_{i,1}‖\le k_{i1}$, where $k_{i1}$ is a positive constant. Therefore, we can obtain the following:

$${\displaystyle \sum _{i=1}^N\frac{1}{N}‖{\mathit{\eta }}_i-{\mathit{\eta }}_0-{\mathit{\rho }}_{i0}‖}=‖{\mathit{\eta }}_0-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N{\mathit{\eta }}_i}‖\le k_{i1}$$

(57)

According to the definition of encirclement control and Equation (53), it can be concluded that multiple unmanned ships will successfully complete the encirclement control task for the target.

(2) Exclusion of Zeno behavior: Taking the derivative of the event-triggered sampling error ${\mathit{q}}_i(t)$ with respect to time $t\in \left(t_s^i,t_{s+1}^i\right)$, we obtain the following:

$$\begin{array}{cc}\hfill ‖{\dot{\mathit{q}}}_i(t)‖& =‖-{\dot{\mathit{z}}}_i(t)‖\hfill \\ & =‖\mathcal{F}({\mathit{z}}_i)+{\overline{B}}_i{\mathit{\tau }}_{Ri}-{\overline{B}}_i{\mathit{\tau }}_{Rj}‖\hfill \end{array}$$

(58)

From the proof results given above, it is known that these variables ${\mathit{z}}_i,{\mathit{\tau }}_{Ri},{\mathit{\tau }}_{Rj}$ are bounded. Therefore, Equation (58) can be written in the following form:

$$‖{\dot{\mathit{q}}}_i(t)‖\le k_{i2}$$

(59)

where $k_{i2}\ge 0$. Since time is within the interval $t\in \left(t_s^i,t_{s+1}^i\right)$, integrating both sides of Equation (59) yields the following expression:

$$t_{s+1}^i-t_s^i\ge t_{s+1}^i-t\ge {\displaystyle \frac{‖{\mathit{q}}_i(t)‖}{k_{i2}}}>0$$

(60)

According to (60), there is no Zeno behavior in the proposed DETC. □

Remark 1. 

The predefined-time FDI attacks observer was first proposed in this work for multi-USVs under FDI network attacks. Compared to the asymptotic convergence, finite-time, and fixed-time observers [1,7,27], the proposed observer allows for a predefined convergence time while maintaining a simple structure. In [41], the prescribed-time observer may encounter singularity issues at the prescribed time point. This occurs because measurement errors in the system state can lead to an infinite gain. Compared to [41], the predefined-time approach proposed in this paper overcomes the singularity problem.

Remark 2. 

Compared to these multi-USVs surrounding control strategies in [4,11,42], this paper proposes a prescribed-performance time-varying surrounding control scheme based on ADP for the first time. In the proposed control strategy, multi-USVs can achieve time-varying even encirclement of the target, meaning that all the pursuer USVs are evenly distributed on a circle with the encirclement radius and move in a circular motion around the target.

Remark 3. 

Reference [16] proposed a reinforcement learning-based prescribed-performance tracking control strategy for USV, which requires the assumption of a weight update law with persistent excitation. Compared to [16], this paper introduces an NN weight update law that does not require the persistently excited condition. Additionally, a system state is incorporated into the NN weight update law, which accelerates the convergence of the NN weights.

Remark 4. 

Compared to the static event-triggered control strategy proposed in [21] for the USV tracking problem, this paper introduces an improved dynamic event-triggered mechanism (IDETM) to further reduce the number of triggers. Moreover, through rigorous theoretical proof, Zeno behavior is eliminated. The proposed reinforcement learning control method based on IDETM significantly reduces the energy consumption of multi-USVs caused by the controller and communication.

3. Results and Discussion

In this section, numerical simulations are used to verify the effectiveness of the proposed control strategy. The computer specifications for this simulation are as follows: an AMD Ryzen 9-7945HX processor running at 2.5 GHz and 16 GB of RAM. The simulation is implemented in MATLAB/Simulink (R2016B) and MATLAB/Simscape (R2023B) with a fixed-step sampling method, where the step size is 0.01 and the total simulation time is 100 s. In the simulation, four pursuer USVs ($i=1, 2, 3, 4$) and one target are designed. Figure 4 shows the communication topology of the multi-USVs.

The settings for the inertia matrix ${\mathit{M}}_i$, Coriolis term ${\mathit{C}}_i$, and damping matrix ${\mathit{D}}_i$ of the dynamics model of the unmanned surface vehicle refer to [5], with specific values listed in

Table 1. Parameters of USVs.

Parameters	Values	Parameters	Values
$m_{i11}$	25.8 kg	$d_{i11}$	$0.7225+1.3274\left|u_i\right|+5.8664u_i^2$
$m_{i22}$	33.8 kg	$d_{i22}$	$0.8612+36.2823\left|v_i\right|+0.805\left|r_i\right|$
$m_{i23}$	1.0948 kg	$d_{i23}$	$-0.1079+0.845\left|v_i\right|+3.45\left|r_i\right|$
$m_{i32}$	1.0948 kg	$d_{i32}$	$-0.1052-5.0437\left|v_i\right|-0.13\left|r_i\right|$
$m_{i33}$	2.76 kg	$d_{i33}$	$1.9-0.08\left|v_i\right|+0.75\left|r_i\right|$
$c_{i13}$	$-m_{i22}{v}_i-m_{i23}{r}_i$	$c_{i23}$	$m_{i11}{u}_i$

. The parameters for the prescribed performance function are set as ${\lambda }_i(0)=4$, ${\lambda }_{i,\infty }=0.001$, $l_i=0.5$. Additionally, the initial values of USV states, NN weights, the weight matrix, control parameters, and the parameters of the predefined-time FDI observer are provided in

Table 2. Parameters for simulation.

Parameters	Values	Parameters	Values
${\mathit{\eta }}_1(0)$	${[0,3,2]}^{\mathrm{T}}$	$a_i$	1
${\mathit{\eta }}_2(0)$	${[-3,0,1.2]}^{\mathrm{T}}$	$p_i$	0.2
${\mathit{\eta }}_3(0)$	${[0,-3,1.5]}^{\mathrm{T}}$	$k_i$	10
${\mathit{\eta }}_4(0)$	${[3,0,1.8]}^{\mathrm{T}}$	$T_i$	10 s
${\mathit{v}}_1(0)$	${[0,0,2]}^{\mathrm{T}}$	$Q_i$	$15{\mathit{I}}_6$
${\mathit{v}}_2(0)$	${[0,0,1.2]}^{\mathrm{T}}$	$R_i$	$8{\mathit{I}}_3$
${\mathit{v}}_3(0)$	${[0,0,1.5]}^{\mathrm{T}}$	${\varsigma }_1$	1.1
${\mathit{v}}_4(0)$	${[0,0,1.8]}^{\mathrm{T}}$	${\varsigma }_2$	1
${\widehat{\mathbb{W}}}_1(0)$	${[16,30,35,22,28,33]}^{\mathrm{T}}$	${\varsigma }_3$	1.2
${\widehat{\mathbb{W}}}_2(0)$	${[20,28,32,22,28,33]}^{\mathrm{T}}$	${\varsigma }_4$	1.5
${\widehat{\mathbb{W}}}_3(0)$	${[18,26,33,22,28,33]}^{\mathrm{T}}$	${\varsigma }_5$	0.5
${\widehat{\mathbb{W}}}_4(0)$	${[17,31,32,22,27.5,33]}^{\mathrm{T}}$	$\sigma$	0.01
${\mathcal{l}}_4$	1	${\beta }_i$	1.1
${\delta }_i(0)$	1	${\xi }_i$	0.5

. The vector ${\widehat{\mathit{z}}}_i$ is expanded as ${\widehat{\mathit{z}}}_i={[{\widehat{z}}_{i1},\dots ,{\widehat{z}}_{i6}]}^{\mathrm{T}}$, and ${\mathit{\varphi }}_i({\widehat{\mathit{z}}}_i)={[{\widehat{z}}_{i1}^2,{\widehat{z}}_{i2}^2,{\widehat{z}}_{i3}^2,{\widehat{z}}_{i1}{\widehat{z}}_{i4},{\widehat{z}}_{i2}{\widehat{z}}_{i5},{\widehat{z}}_{i3}{\widehat{z}}_{i6}]}^{\mathrm{T}}$ is set as the NN activation function. The desired motion trajectory for the target is set as ${\mathit{\eta }}_d(t)={[3+0.2t,3+0.2t,0.785]}^{\mathrm{T}}$, while the time-varying surrounding configuration of the pursuer USVs relative to the target is given in

Table 3. Time-varying surrounding function.

Parameters	Values
${\mathit{h}}_{10}(t)$	${[3\cos (0.15t+2\pi /4), 3\sin (0.15t+2\pi /4),0]}^{\mathrm{T}}$
${\mathit{h}}_{20}(t)$	${[3\cos (0.15t+4\pi /4), 3\sin (0.15t+4\pi /4),0]}^{\mathrm{T}}$
${\mathit{h}}_{30}(t)$	${[3\cos (0.15t+6\pi /4), 3\sin (0.15t+6\pi /4),0]}^{\mathrm{T}}$
${\mathit{h}}_{40}(t)$	${[3\cos (0.15t+8\pi /4), 3\sin (0.15t+8\pi /4),0]}^{\mathrm{T}}$

. Assuming the false data are ${\Gamma }_i(t)={[0.1\sin (0.1t),0.1\cos (0.1t),0.1\sin (0.1t)]}^{\mathrm{T}}$, the attack probability constants are set as ${\overline{\gamma }}_{i1}=0.3$, ${\overline{\gamma }}_{i2}=0.26$, ${\overline{\gamma }}_{i3}=0.23$.

Considering the surrounding control problem of multi-USVs under FDI attacks, this work proposes an ADP-based prescribed performance time-varying encirclement control strategy. This subsection verifies the effectiveness and superiority of the proposed scheme through numerical simulations and comparative experiments.

Figure 5 shows the 2D trajectory of multi-USVs. From Figure 5, it can be seen that at the initial moment, the target is located outside the four pursuer USVs. Under the proposed control method, the four pursuer USVs successfully form a square configuration to encircle the target unmanned ship. Furthermore, from Figure 5, it can be observed that the formation of the configuration of pursuer USVs is time-varying, meaning that each pursuer USV performs a circular motion around the target unmanned ship, ultimately resulting in the target being enclosed within the convex hull formed by the pursuing unmanned ships.

To demonstrate the superior performance of the proposed ADP-based prescribed-performance control method (PPCADP), Figure 6 presents comparative simulation results with three different control approaches, including the traditional ADP control method (TADP) in [38], the model predictive control strategy (MPC) in [43], and the sliding mode control method (SMC) in [44]. Figure 6 illustrates the simulation results of the state tracking error under different controllers. From Figure 6, it can be seen that the state tracking errors under the PPCADP strategy remain strictly within the prescribed bounds, indicating the superior control performance of the proposed PPCADP. Figure 6b–d show the simulation results of the state tracking errors under the TADP, MPC, and SMC methods, respectively. The three control methods without prescribed performance constraints fail to confine the state tracking errors within the predefined bounds.

To sum up, it can be observed from Figure 6 that the proposed PPCADP approach achieves faster convergence, higher convergence accuracy, and smaller overshoot compared with the other three control strategies.

In order to quantitatively compare the energy consumption results, a unified control energy cost function has been designed in the manuscript for different control methods: $J_C={\displaystyle {\int }_0^{T_0}{\displaystyle \sum _{i=1}^4\left[{\mathit{z}}_i^{\mathrm{T}}Q{\mathit{z}}_i+{({\mathit{\tau }}_i-{\mathit{\tau }}_{fi})}^{\mathrm{T}}R({\mathit{\tau }}_i-{\mathit{\tau }}_{fi})\right]\mathrm{d}t}}$, where $Q={\mathit{I}}_6$, $R={\mathit{I}}_3$, $T_0=100 \mathrm{s}$. Figure 7 presents comparative simulation results of energy consumption under different control strategies.

As observed from Figure 7, the PPCADP strategy achieves the lowest energy consumption, reducing energy cost by 12.8%, 49.3%, and 71.6% compared to the TADP, MPC, and SMC methods, respectively. This improvement is attributed to the incorporation of both control performance and control cost in the design of the cost function for the proposed PPCADP.

Figure 8 shows the control input curves. It can be observed that the resilient control strategy designed in this work ensures the stable performance of the multi-USVs under FDI attacks, and the control inputs converge to the origin within 10 s.

Figure 9 illustrates the time response of the NN weights. It can be seen that the weights converge from initial values to a constant value, indicating that the weights remain stable as the state convergence of CLSs.

The time response results of the attack variable that satisfy the Bernoulli probability distribution are shown in Figure 10.

Figure 11 shows the estimation error of the predefined-time FDI observer for the injected false data. It can be observed that the FDI observer’s estimation error converges to zero within 10 s (the predefined convergence time), verifying the effectiveness and rapidity of the proposed predefined-time FDI observer.

Figure 12 presents a comparison of the number of sampling between the proposed dynamic event-triggered control scheme (DETC), and the event-triggered reinforcement learning control (ETC) method proposed in [40]. The ETC method [40] is applied to the USV model in simulation, and all parameters of the multi-USVs are kept consistent with the settings mentioned above. In Figure 12, the blue bar represents the results under the DETC, while the orange-red bar is the results under the ETC. It can be observed that the four USVs have fewer event-trigger numbers under DETC. The dynamic event-triggering strategy proposed in this paper reduces the total number of triggers by 31.5% compared to the ETC in [40].

Figure 13 shows the time response curves of the dynamic event-triggering condition $q_i(t)$ (44) and trigger threshold $q_{iT}$. It can be observed that the value of the event-triggering condition is always less than or equal to the trigger threshold, and the $q_i(t)$ and $q_{iT}$ gradually converge to zero as the closed-loop system state stabilizes.

In summary, the theoretical proofs and simulation results given above validate the effectiveness and superiority of the proposed control method (18). This method can effectively solve the time-varying surrounding control problem of multi-USVs under FDI attacks.

To further validate the engineering applicability of the proposed PPCADP control method, a 3D semi-physical simulation model of multiple USVs was developed using the Simscape simulation platform. The mass is $1.884 \mathrm{kg}$, the inertial matrix of USV is $I=\mathrm{diag}\{0.098,0.017,0.75\} \mathrm{kg}·{\mathrm{m}}^2$, and the control parameters were kept consistent with those used in the previous Simulink simulations. Figure 14 and Figure 15 show the motion trajectories of the USVs and the time responses of the state tracking errors in the Simscape environment, respectively. These results demonstrate the effectiveness of the proposed PPCADP in the Simscape-based multi-USVs simulation.

Although the above simulation results verify the effectiveness and superiority of the proposed control method, sensitivity analysis was not conducted [45,46]. In future work, we plan to further investigate the reliability of the algorithm under varying simulation parameters through box plots and Monte Carlo simulations.

4. Conclusions

This paper integrated an IDETM with the ADP algorithm and proposed a prescribed-performance time-varying surrounding control scheme for multiple unmanned surface vehicles (multi-USVs) under false data injection (FDI) attacks. First, a predefined-time FDI observer was developed to estimate the injected false data within a user-specified time, thereby removing the conventional assumption of bounded false data. Subsequently, an ADP-based control strategy utilizing the IDETM was designed. Within the learning framework of ADP, a weight update law was constructed without requiring the persistency of the excitation condition, enhancing the generality of the proposed method. Finally, theoretical analysis and numerical simulations demonstrated that the proposed approach enables the pursuer USVs to achieve time-varying surrounding configuration of the target while avoiding Zeno behavior in the DETC. Future work will explore the surrounding control problem of multi-USVs in the presence of actuator failures.