Adaptive Fault-Tolerant Formation Control of Heterogeneous Multi-Agent Systems under Directed Communication Topology

This paper investigates the adaptive fault-tolerant formation control scheme for heterogeneous multi-agent systems consisting of unmanned aerial vehicles (UAVs) and unmanned surface vehicles (USVs) with actuator faults, parameter uncertainties and external disturbances under directed communication topology. Firstly, the dynamic models of UAVs and USVs are introduced, and a unified heterogeneous multi-agent system model with actuator faults is established. Then, a distributed fault-tolerant formation controller is proposed for the unified model of UAVs and USVs in the XY plane by using adaptive updating laws and radial basis function neural network. After that, a decentralized formation-tracking controller is designed for the altitude control system of UAVs. Based on the Lyapunov stability theory, it can be proved that the formation errors and tracking errors are uniformly ultimately bounded which means that the expected time-varying formation is achieved. Finally, a simulation study is given to demonstrate the effectiveness of the proposed scheme.


Introduction
In the last three decades, the formation control of multi-agent systems (MASs) has drawn lots of researchers' attention in both theoretical research and practical applications, such as forest fire monitoring, power grid inspection, search and rescue, and so on [1][2][3][4][5][6]. The purpose of formation control is to form a specific formation to complete the tasks. One of the fundamental problem in MASs is consensus problem. Consensus problems of MASs have been investigated extensively and results have been achieved [7][8][9][10][11][12][13]. Based on the consensus theory, the formation control problem of MASs can be solved. In [14], the output-feedback formation control protocol of tracking a desired trajectory for a set of UAVs is developed. In [15], a distributed time-varying output formation control scheme is introduced for general linear MASs with directed topology. In [16], a distributed leader-following formation control for multiple quadrotors is studied by using nonsmooth backstepping method. In [17], a distributed relative position-based formation control by using backstepping approach is studied for leader-follower MASs.
However, the above results for formation control are used for homogeneous multiagent systems. In the actual applications, for the heterogeneous multi-agent systems (HMASs) in which each agent may have different structures, dynamics and even the information perceptions and decision-making capabilities are considered. The time-varying formation-containment control for homogeneous and heterogeneous MASs is studied in [18]. In [19], the time-varying output formation-tracking scheme for linear HMASs is developed with directed topologies. In [20], the coherent formation control for HMASs is introduced. The distributed cooperative synchronization control problem for a networked HMASs is developed in [21]. The HMASs can achieve more complex and variable tasks through information interaction. For example, the HMASs consisting of multiple UAVs and USVs have a larger search radius and attack range, which is of great significance in both military and civil fields. Due to the heterogeneous characteristics of UAVs and USVs, as well as the influence of parameter uncertainties and external environment, the air-sea heterogeneous formation faces several important challenges which are shown as follows: • UAVs and USVs differ notably in structure, model parameters, and state dimensions. Furthermore, the task changing is random, making it difficult to complete air-sea coordinated formation tasks. • During the process of formation, the UAVs and USVs may inevitably be subjected to internal and external uncertainties, such as parameter uncertainties and external disturbances caused by modeling techniques and external environment. • Unsatisfactory faults caused by a lot of damage to the actuator of the multiple UAVs and USVs system may affect the tracking performance.
In practical applications, faults may occur in the MASs. Faults may lead to system performance degradation, or even more serious consequences. Fault-tolerant control (FTC) is a useful method that has drawn wide attention [22][23][24][25]. In [26], a distributed adaptive leader-following formation control problem is studied for nonlinear second-order MASs in the presence of actuator faults. The distributed fault-tolerant time-varying formation control scheme for second-order MASs is presented subject to actuator faults in [27]. In [28], the active FTC problem is illustrated for the high-order HMASs with network disconnections and actuator faults. In [29], an adaptive FTC strategy by using a virtual actuator framework is presented for nonlinear HMASs with actuator faults. In addition, external disturbances and parameter uncertainties can also impact the performance of the systems. In [30], the time-varying formation tracking control scheme for the linear MASs is presented with external disturbances under directed graph. In [31], the time-varying anti-disturbance formation control scheme is introduced for nonlinear MASs under switched directed topologies. An adaptive tracking controller by adjusting the coupling weight is designed for the leader-follower linear MASs with external disturbances in [32]. In [33], an internal model approach is studied for the leader-following rendezvous with external disturbances and parameter uncertainties. During the process of formation, unmanned vehicles inevitably suffer from actuator faults, parameter uncertainties, and external disturbances. For example, due to modeling techniques and unpredictable marine environments, USVs are subject to internal and external uncertainties consisting of parameter uncertainties, nonparametric uncertainties, and external disturbances [34]. In addition, the actuator faults and uncertainties in a single vehicle can spread unevenly to neighboring vehicles through the directed communication topology in the process of formation. Hence, it is necessary to study the fault-tolerant formation control for HMASs consisting of multiple UAVs and USVs with parameter uncertainties and external disturbances. Although the research on FTC of MASs has achieved some results, the adaptive fault-tolerant formation control under directed communication topology for air-sea systems is still open, and needs to be further studied.
Motivated by the abovementioned results, this paper presents the adaptive faulttolerant formation control scheme for multiple UAVs and USVs with actuator faults, parameter uncertainties, and external disturbances under directed communication topology. In order to handle the problems, a unified dynamic model of UAVs and USVs, which includes the XY plane and Z axis, are presented. To deal with the parameter uncertainties and external disturbances, a distributed fault-tolerant formation controller by combining adaptive control method and the radial basis function neural network is proposed in the XY plane and a decentralized formation-tracking controller is designed for the altitude control system of UAVs in the Z axis. The main contributions of this paper are summarized as 1.
The adaptive fault-tolerant formation control is developed for the HMASs with parameter uncertainties, external disturbances, and actuator faults including loss of effectiveness and bias under directed communication topology. Some existing results have been researched for time-varying formation control, such as [16,[34][35][36][37]. However, these results only consider the formation control of single vehicle or the HMASs consisting of a UAV and a USV. In this paper, the fault-tolerant time-varying formation for multiple UAVs and USVs can be achieved.

2.
In order to handle parameter uncertainties and external disturbances, the adaptive control method and the radial basis function neural network (RBFNN) are combined to ensure that the formation errors and tracking errors of the closed-loop system are uniformly ultimately bounded.

3.
Compared with the works [20,38], in which the height of each UAV is the same and time-invariant, this paper presents a decentralized tracking controller for the altitude control system of UAVs to track the reference signal to meet the requirements of practical tasks.
The rest of this paper is organized as follows. In Section 2, some preliminaries and problem formulation are given. In Section 3, the distributed adaptive fault-tolerant formation control scheme is introduced for multiple UAVs and USVs in the presence of actuator faults, parameter uncertainties and external disturbances. In Section 4, a decentralized formation tracking controller is proposed for the altitude control system of UAVs. In Section 5, a simulation study is given and some conclusions are drawn in Section 6. Notation 1. * denotes the Euclidean norm. For a matrix A, λ max (A) represents the maximum eigenvalue of matrix A, λ 2 (A) represents the minimum nonzero eigenvalue, and A F denotes the Frobenius norm of the matrix. Let 1 denote 1 = col{1, . . . , 1} ∈ R N .

Graph Theory
Let G = (V, E ) be a directed graph, in which V = {1, 2, . . . , N} represents the set of nodes and E ⊆ V × V represents the set of edges. The neighbour set of node i is denoted by N i = {j ∈ V |(j, i) ∈ E }. If there exists a directed path between two arbitrary nodes, the graph G is strongly connected. A = [a ij ] ∈ R N×N denotes the adjacency matrix of G, where a ij = 1 if (j, i) ∈ E and a ij = 0 otherwise. D = diag{d 1 , d 2 , . . . , d N } denotes the degree matrix of G, where d i = ∑ N j=1 a ij . The Laplacian matrix of G is defined as L = D − A.

Problem Formulation
In this subsection, the HMASs consisting of M UAVs and N − M USVs are considered. The dynamic models of the UAVs and USVs are given firstly. Based on them, a unified dynamic model for the HMASs is demonstrated. For convenience, let Π 1 = {1, 2, . . . , M}, Unmanned aerial vehicle model.The structure of the quadrotor UAV is shown in Figure 1. The dynamic model of the i-th (i ∈ Π 1 ) quadrotor UAV is given as [41] where [p aix , p aiy , p aiz ] denotes the position state, [φ i , θ i , ψ i ] denotes the attitude state, u pi denotes the control thrust of the quadrotor, τ φi , τ θi , τ ψi denote the three control torques of the quadrotor, m ai is the mass of the quadrotor UAV, g is the gravitational acceleration,d i denotes the overall residual rotor angle, d ix , d iy , d iz , d iφ , d iθ , d iψ denote the translational drag coefficients, J ax , J ay , J az are the moments of the inertia, J ar is the moment of rotor's inertia, ∆ aix , ∆ aiy , ∆ aiz denote the external disturbances encountered by the quadrotor UAV.
x e y e z e From the dynamic model (1) and (2), we can found that the motion of the quadrotor consists of translational dynamics with respect to positions and rotational dynamics with respect to angles. Our formation goal is to locate the center of mass in a predefined position, while the rotational dynamics can be stabilized separately. Hence, the position dynamics of the i-th (i ∈ Π 1 ) UAV based on (1) can be rewritten as where p ai = [p aix , p aiy , p aiz ] denotes the position of the i-th UAV, Unmanned surface vehicle model. The kinematic and dynamic equation of the i-th (i ∈ Π 2 ) USV in the horizontal plane is described as [35] and where (x si , y si ) is the position of the i-th USV; ψ si is the yaw angle of the i-th USV; α i = [µ si , ν si , r si ] are the surge, sway, and yaw velocity, respectively; m µsi , m νsi , m rusi are the inertial mass; Since the motion model of the USVs described by (4) and (5) is underactuated, a hand position approach is used to deal with it. We define the front point (p six , p siy ) of the USVs as the hand point which can be formulated as where L si is the distance between the actual position (x si , y si ) and the new defined hand point (p six , p siy ), which is shown in Figure 2. By taking the second derivative of (7), one can obtain Substituting (6) into (8) yields that where and Based on (9), the position dynamics of the i-th (i ∈ Π 2 ) USV can be described as where Actuator fault model.The actuator fault model for the h-th actuator of the i-th (i ∈ Π) HMASs is given as where u F ih is the actual actuation input, u ih is the applied control signal to be designed, 0 < ρ ih ≤ 1 is the unknown effectiveness factor and u ihb represents the unknown bias. For the actuator fault model (13), ρ ih = 1 and u ihb = 0 denote that there is no fault; 0 < ρ ih < 1 and u ihb = 0 denote the loss of effectiveness fault; ρ ih = 0 and u ihb = 0 denote bias fault; 0 < ρ ih < 1 and u ihb = 0 denote both loss of effectiveness and bias faults.
From (13), the fault model for all actuators can be described as where ρ i = diag{ρ i1 , ρ i2 , . . . , ρ in } and u ib = [u ib1 , u ib2 , . . . , u ibn ] with n representing the dimension of the control input signal. Unified model. The UAV model with actuator faults (14) in the XY plane can be rewritten as x ai2 = f aixy + g aixy (ρ ai u aixy + u aib ) + ∆ aixy = F aixy + G aixy u aixy + ∆ aixy , where x ai1 = [p aix , p aiy ] is the position of the i-th UAV in the XY plane, u aixy = [u aix , u aiy ] , Similarly, the UAV model with actuator faults (14) in the Z axis can be rewritten as where p aiz is the altitude of the i-th UAV, v aiz is the velocity of the i-th UAV in the Z axis, f aiz = −d izṗaiz /m ai − g, ρ aiz is the effectiveness factor, u aibz is the bias, F aiz = f aiz + u aibz /m ai , G aiz = ρ ai /m ai .
The USV model with actuator faults (14) can be rewritten as where x si1 = p si , ρ si = diag{ρ six , ρ siy } is the effectiveness factor, u sib = [u sibx , u siby ] is the bias, g si = Ω si (ψ si ) si , F si = f sixy + g si u sib , G si = g si ρ si . Combining (15) and (17), the unified model of UAVs and USVs in the XY plane can be obtained where x i1 = x ai1 ∈ R 2 , x i2 = x ai2 ∈ R 2 are the position and velocity of the i-th (i ∈ Π 1 ) UAV in the XY plane, F xi = F aixy , G xi = G aixy , ∆ xi = ∆ aixy , u xi = u aixy ; x i1 = x si1 ∈ R 2 , x i2 = x si2 ∈ R 2 are the position and velocity of the i-th (i ∈ Π 2 ) USV, Hence, we are under the situation to design the fault-tolerant time-varying formation control scheme for the HMASs consisting of multiple UAVs and USVs in the presence of unknown actuator faults, parameter uncertainties and external disturbances under directed topology.

Remark 1.
In the inertial frame O − XYZ, the USVs only move in the XY plane. The UAVs move in three-dimensional space, but their motion in the Z axis can be decoupled from that in the XY plane. Hence, the height of the UAVs can be controlled independently. Therefore, the fault-tolerant time-varying formation control for multiple UAVs and USVs is considered in the XY plane. At the same time, the formation-tracking controller is designed for the UAVs so that the height of the UAVs can track the reference signal. In this way, the fault-tolerant time-varying formation control for multiple UAVs and USVs can be achieved. Assumption 1. The effectiveness factor ρ ih and bias u ihb are unknown but bounded. There exist positive constants ρ ih ,ρ ih andū ihb satisfying 0 < ρ ih ≤ ρ ih ≤ρ ih ≤ 1 and u ihb ≤ u ihb , respectively. Assumption 2. The aerodynamic drag coefficients d ix , d iy and d iz are unknown but bounded. Assumption 3. The external disturbances ∆ aix , ∆ aiy , ∆ aiz encountered by the quadrotor UAV are bounded and satisfying ∆ aix ≤∆ aix , ∆ aiy ≤∆ aiy , ∆ aiz ≤∆ aiz , where∆ aix ,∆ aiy and∆ aiz are unknown positive constants. Assumption 4. The external disturbances w µsi , w νsi , w rsi encountered by the USV are bounded and satisfying w µsi ≤w µsi , w νsi ≤w νsi , w rsi ≤w rsi , wherew µsi ,w νsi andw rsi are unknown positive constants. Assumption 5. In (4), the sway velocity ν si of the underactuated USV is passively-bounded.

Remark 2.
Assumption 1 is standard to handle the actuator faults in the existing literature [43,44]. Due to the unpredictable aerodynamics produced by the complex operation environment, it is difficult to obtain accurate system parameters, so Assumption 2 is reasonable and realistic. Since UAVs and USVs will not be used under extreme whether conditions, Assumptions 3 and 4 are also reasonable. Superficially, Assumption 5 seems to be restrictive. However, it is easy to verify that Assumption 5 is always satisfied in most practical applications of the USVs [45].

Radial Basis Function Neural Network
Since the radial basis function neural network can approximate any continuous function with arbitrary precision, in this paper, the RBFNN is used to approximate the nonlinear function. The RBFNN can be expressed as where f (Z) is a smooth nonlinear function, Z = [Z 1 , Z 2 , . . . , Z n ] ∈ Ω Z ⊂ R n denotes the input vector, w(Z) = [w 1 (Z), w 2 (Z), . . . , w p (Z)] ∈ R p is the basis function vector, p is the number of the RBFNN nodes. (Z) is the approximation error, bounded by (Z) ≤¯ . θ * = [θ 1 , θ 2 , . . . , θ p ] ∈ R p is the ideal weighting vector, which is denoted as where θ denotes the weighting vector.

Control Objective
The control objective of this paper is to design an adaptive fault-tolerant formation control scheme for multiple UAVs and USVs (16) and (18) to achieve the time-varying formation of the HMASs in the presence of unknown actuator faults, parameter uncertainties and external disturbances under directed topology, that is, for any given bounded initial states, where χ 1 , χ 2 , χ 3 and χ 4 are small enough positive constants, being piecewise continuously differentiable. h i1 (t) and h i2 (t) are the position and velocity, respectively. c ip and c iv are the desired height and velocity reference signal, respectively.

Distributed Fault-Tolerant Formation Control Scheme and Performance Analysis
In this section, a distributed fault-tolerant formation control scheme is designed to achieve the time-varying formation control of HMASs composed of UAVs and USVs with actuator faults, parameter uncertainties and external disturbances under directed topology in the XY plane.
The schematic of the control system is illustrated in Figure 3.

Performance Analysis
The performance of the time-varying formation error system is given as follows.
Theorem 1. Consider a heterogeneous multi-agent system (18). Suppose that Assumptions 1-5 hold and the feasibility condition h i2 −ḣ i1 = 0 of the time-varying formation is satisfied, the fault-tolerant control scheme is designed as (26) and the adaptive laws are developed as (27)-(29), then the time-varying formation errorsz and e in the XY plane are uniformly ultimately bounded.

Proof of Theorem 1. Consider the following Lyapunov candidate function
The time derivative of (30) is given aṡ Tr(G xiĠ xi ) Substituting the control input (26) into (31), we can obtaiṅ Then, the RBFNN is used to approximate the unknown nonlinear function F xi , which is shown as where i1 ≤¯ i1 .
In terms of Young's inequality, we can obtain where κ i11 = θ * i1 θ * i1 and κ i12 = (¯ i1 +∆ xi ) (¯ i1 +∆ xi ). Substituting (34) and (35) into (32), we can obtaiṅ According to Lemma 1, we can obtain In terms of Young's inequality, we can obtain Substituting (37)-(39) into (36) yields thaṫ where δ = min{δ 1 , δ 2 , . . . , ). In terms of the boundedness theorem, the time-varying formation errorsz and e are uniformly ultimately bounded. Recalling the definition ofz i , s i and e i , we can conclude that s i is uniformly ultimately bounded. According to the control objective, the time-varying formation of the HMASs in the XY plane is achieved. This completes the proof.

Remark 3.
The time-varying formation feasibility condition h i2 −ḣ i1 = 0 indicates that there exists a constraint on the formation that can be achieved. Due to their dynamic limitations, the HMASs are unable to achieve any formations. Therefore, it is necessary to judge whether the feasibility condition is met when a formation is given. If the condition is met, the pre-designed time-varying formation can be achieved.

Decentralized Formation Controller Design and Performance Analysis
In this section, a decentralized formation controller is designed to achieve the height tracking control of UAVs in the Z axis.

Decentralized Formation Controller Design
The altitude error system is defined as where c ip is the desired position signal and c iv is the desired velocity signal. The formation tracking control input and adaptive laws are designed as 0, ifĜ aiz =Ḡ aiz and S ≥ 0 orĜ aiz = G aiz and S ≤ 0, S, otherwise, where S = ι 23 (−k 23Ĝaiz + e iζ u aiz ), G aiz andḠ aiz are the lower bound and upper bound of the parameter G xi , respectively, where σ i , k 21 , k 22 , k 23 , k iζ , ι 21 , ι 22 , ι 23 , ς i3 , ς i4 are positive parameters to be designed.

Performance Analysis
The performance of the altitude error system is given as follows.

Theorem 2.
Consider the UAVs' altitude control system (16). Suppose that Assumptions 1-3 hold and the feasibility condition c iv −ċ ip = 0 is satisfied, the formation tracking control scheme is designed as (43) and the adaptive laws are developed as (44)-(46), then the UAVs' altitude trajectory can track the reference signal and the tracking error e izp is uniformly ultimately bounded.

Proof of Theorem 2.
If the feasibility condition c iv −ċ ip = 0 is satisfied, then the time derivative of the altitude error system (41) and (42) can be written aṡ In the system (47), e izv can be regarded as the virtual control input. By designing the virtual control input ζ i = −k iζ e izp , the stability of the system (47) can be ensured.
Consider a positive Lyapunov function as The time derivative of (50) is given bẏ Define a new error as Taking the time derivative of (51) and substituting the control input (43) into it, we can obtainė Consider the following Lyapunov candidate function where V iz is given as Taking the time derivative of (54) yields thaṫ V iz ≤ −k iζ e izp e izp − σ i e iζ e iζ + e iζ F aiz + e iζ ∆ aiz Similarly, the RBFNN is used to approximate the unknown nonlinear function F aiz , which is shown as where i2 ≤¯ i2 .
In terms of Young's inequality, one can obtain where κ i21 = θ * i2 θ * i2 and κ i22 = (¯ i2 +∆ aiz ) (¯ i2 +∆ aiz ). Substituting (57) and (58) into (55), we can obtaiṅ V iz ≤ −k iζ e izp e izp − σ i e iζ e iζ + e iζ e iζ κ i21 ω i2 ω i2 where 2 . According to (59), the derivative of (53) can be obtaineḋ where In terms of the boundedness theorem, the solution of the closed-loop system is uniformly ultimately bounded. Recalling the definitions of e izp and e iζ , it can be concluded that the altitude tracking error and velocity error of UAVs is uniformly ultimately bounded. According to control objective, the pre-defined time-varying formation h(t) is achieved for multiple UAVs and USVs. This completes the proof.

Remark 4.
For simplicity, it is assumed that the height of each UAV in the existing literature [20] and [38] is the same and time-invariant when achieving a formation. However, in practical applications, the height of UAVs may vary according to the tasks. For the altitude control system of UAVs, a decentralized tracking controller is designed to track the reference signal to meet the requirements of practical tasks in the presence of actuator faults, parameter uncertainties and external disturbances in this paper, which has more application value.

Simulation Study
In order to verify the effectiveness of the proposed fault-tolerant time-varying formation control scheme, a HMAS composed of two UAVs (i = 1, 2) and two USVs (i = 3, 4) is selected in this section. Table 1 shows the system parameters of UAVs and USVs. The directed communication topology is shown in Figure 4, in which the weights are selected as one.  The actuator faults of UAVs and USVs are given as The actuator faults of UAVs in the Z axis are given as t < 13 s, ρ a1z = 1, u a1bz = 0; t ≥ 13 s, ρ a1z = 0.9, u a1bz = 0.2; t < 10 s, ρ a2z = 1, u a2bz = 0; t ≥ 10 s, ρ a2z = 0.8, u a2bz = 0.1.
The external disturbances are given as ∆ a1xy = [0.2cos(0.5t), 0.8cos(t)] , ∆ a1z = 0.3cos(0.2t), ∆ a2xy = [0.6sin(t), 0.3] , ∆ a2z = 0.5sin(0.4t), w d3xy = [1.1cos(0.5t), −0.2sin(2t)] , w d4xy = [0.6, −0.2cos(t)] . The desired time-varying formations are described as h i1 = [3cos(t + (i − 1)π/2), 3sin(t + (i − 1)π/2)] , h i2 = [−3sin(t + (i − 1)π/2), 3cos(t + (i − 1)π/2)] , i = 1, 2, 3, 4. c ip = 0.1t, c iv = 0.1, i = 1, 2. The performance of the proposed adaptive fault-tolerant time-varying formation tracking scheme is compared with the robust backstepping sliding-mode control scheme in [41]. To quantitatively evaluate the fault-tolerant time-varying formation tracking performance, X-axis position tracking error metric (XPTEM), Y-axis position tracking error metric (YPTEM), and Z-axis position tracking error metric (ZPTEM) are defined as The position snapshots of the HMASs in the XY plane at different time instants is shown in Figure 5. The time-varying formation errors of all agents in the XY plane are depicted in Figure 6. The altitude tracking error of UAVs is shown in Figure 7.   From these curves, it can be seen that the tracking performance by the proposed control scheme is better than the robust backstepping sliding-mode control scheme in [41]. Furthermore, under the proposed control scheme, all tracking errors of the systems are uniformly ultimately bounded. From Figures 6 and 7, it can be seen that the tracking errors under the scheme in [41] have sharp deviations from the small region containing zero when the actuator fault occurs. The position tracking error metrics are shown in Figure 8. It can be seen from these curves that the proposed control scheme has better performance than the scheme in [41]. From Figures 5-8, we can know that the expected time-varying formation is achieved for the HMASs in the presence of actuator faults, parameter uncertainties and external disturbances. The effectiveness of the proposed fault-tolerant formation control scheme is verified by the simulation study.

Conclusions
In this paper, an adaptive fault-tolerant time-varying formation control scheme is designed for a heterogeneous multi-agent system composed of multiple UAVs and USVs with actuator faults, parameter uncertainties and external disturbances under directed communication topology. Based on the unified dynamic model of UAVs and USVs in the XY plane, a distributed fault-tolerant formation controller utilizing adaptive control and RBFNN is proposed. At the same time, a decentralized formation tracking controller is designed for the altitude control system of UAVs. Based on Lyapunov stability theory, the time-varying formation errors and tracking errors are uniformly ultimated bounded, and the pre-defined time-varying formation for multiple UAVs and USVs can be realized. Simulation results verify the effectiveness of the proposed scheme. Nevertheless, the communication topology is fixed in this study. Hence, a fault-tolerant formation control for multiple UAVs and USVs under switch topology will be investigated in our future works.