A Novel Decoupled Synchronous Control Method for Multiple Autonomous Unmanned Linear Systems: Bounded L₂-Gain for Coupling Attenuation

Featured Application

This proposed method is suitable for the cooperative consensus control of various homogeneous Multiple Autonomous Unmanned linear systems, such as an underwater robot swarm and aerial UAV swarm.

Abstract

This paper addresses the distributed optimal decoupling synchronous control of multiple autonomous unmanned linear systems (MAUS) subject to complex network dynamic coupling. The leader–follower mechanism based on neighborhood error dynamics is established and the network coupling term is regarded as the external disturbance to realize the decoupling cooperative control of each agent. The Bounded L₂-Gain problem for the network coupling term is formulated into a multi-player zero-sum differential game. It is shown that the solution to the multi-player zero-sum differential game requires the solution to coupled Hamilton–Jacobi (HJ) equations. The coupled HJ equations are transformed into an algebraic Riccati equation (ARE), which can be solved to obtain the Nash equilibrium of a multi-player zero-sum game. It is shown that the bounded L₂-Gain for coupling attenuation can be realized by applying the zero-sum game solution as the control protocol and the ultimately uniform boundedness (UUB) of a local neighborhood error vector under conservative conditions is proved. A simulation example is provided to show the effectiveness of the proposed method.

1. Introduction

In the field of multi-agent distributed control, synchronous cooperative control is one of the most popular research topics because of its wide application prospects in many engineering systems, such as the cooperative control of autonomous underwater vehicles, wind farm and unmanned aerial vehicles. A great deal of research has been developed on the distributed control methods for multi-agent synchronization [1,2,3,4]. The purpose of distributed synchronous control is to design a control protocol for each agent, depending only on the states of neighboring agents, to ensure that the states of all agents in the communication digraph finally achieve synchronization. A leader–follower mechanism is the most popular one for distributed synchronous control because of its simplicity and scalability. The basic idea is that a leader agent is set as the reference node, and other agents are set as the follower nodes whose goal is to track the reference node to achieve the ultimate synchronization of the entire communication network.

On the basis of this mechanism, ref. [5] defines the local neighborhood error of each agent and deduces that this is a dynamical system with multiple control inputs, from itself and all of its neighbors. This means that the local neighborhood error of each agent is the result of coupling from adjacent node agents, which brings considerable complexity to the design of the control protocol. Ref. [6] formulated this intricate relationship as a differential game, namely, a multi-agent differential graphical game, by defining a local performance index for each agent. Optimal control and game theory [7,8] has been successful utilized to formulate strategic behavior, where the dynamic of each agent relies on the actions of itself and its neighbors. In an optimal control and differential graphical game, each agent minimizes the performance objective (cost value function) by adjusting its control strategy to optimal. In [9], the finite-time optimal coordination problem of multi-agent systems (MASs) is investigated. The authors in [10] addressed the centralized optimal coordination problem under tree formation constraints. In [11], the robust optimal formation control of heterogeneous MASs is studied. These published graphical games and optimal control methods for consensus and synchronization are achieved based on the solution of coupled Hamilton–Jacobi–Isaacs (HJI) equations and Hamilton–Jacobi–Bellman (HJB) equations, respectively. In practice, coupled HJI and coupled HJB equation are difficult to be solved by analytical methods due to their inherent nonlinearity and uncertainty.

The reinforcement learning (RL) method is often regarded as the effective method to solve the coupled HJI and coupled HJB equation. RL is the branch of machine learning concerned with how to methodically adjust the control strategy of agents based on the rewards from the environment [12,13,14,15]. In [16], an online distributed optimal adaptive algorithm is proposed for a differential graphical game, the intelligent identifier is designed to find the unknown dynamic and the neural actor–critic network structure is introduced to find the solutions of the Nash equilibrium. In [17], the bounded L₂-gain consensus problem for the MASs with external disturbance is formulated into the zero-sum differential game by introducing a specific performance index and a policy iteration (PI) algorithm-based RL is provided to find the solution to the coupled HJI equations. In [18], the optimal synchronization control problem is studied for homogeneous MASs with input saturation by using the RL methods. This research utilizes the neural network as an approximator and a design-specific update law so that the neural network approximates the optimal value function and optimal control strategy with certain precision. However, the strict asymptotic convergence proof of the neural network is not given in these works, and only the boundedness of approximate errors is guaranteed. In addition, the neural network approximator-based RL needs to satisfy the persistence of the excitation condition (PE) [19,20,21,22,23], which also limits the practical engineering application of these methods.

The quadratic optimal control problem of a linear single system can be solved by solving an algebraic Riccati equation (ARE) [24], but the optimal control problem of MASs is far more complicated than that of a single system owing to the state coupling in the control design. At present, some optimal control methods of MASs are accompanied by a huge amount of calculations and strong assumptions.

Motivated by the above discussion, this paper focuses on the optimal cooperative control of Multiple Autonomous Unmanned linear systems (MAUS) from a new perspective, i.e., the adjacent nodes’ input coupling part is regarded as the external disturbance. Thus, the complex distributed multi-agent error dynamics are decoupled into centralized multi-input dynamics. Inspired by the idea of a zero-sum game in [17], this paper formulates these centralized multi-input dynamics into multiple independent multi-player zero-sum differential games. The motivation is to realize the decoupled optimal synchronous control of MASs and the main contributions of this paper are listed in the following:

(1)

The coupling among the distributed multi-agents is equivalent to the disturbance from different channels, and the local neighborhood error dynamics of each agent are modeled as an independent centralized multi-player game.

(2)

The bounded L₂-gain problem for coupling attenuation is introduced and is formulated into a multi-player zero-sum game by defining a modified performance index. Different from the L₂-gain problem of [17], concerning disturbance rejection, the motive of the bounded L₂-gain for the coupling attenuation studied here is to suppress the coupling effect on the performance.

(3)

It is proved that the solution of the zero-sum game requires the solution of the coupled Hamilton–Jacobi (HJ) equation. The coupled HJ equation of each agent is transformed into an independent equivalent algebraic Riccati equation, which simplifies the solution process effectively.

This paper is organized as follows. Section 2 provides the mathematical background and derives the local error dynamics of each node that is coupled by its own control protocol and those of its neighbors. Section 3 proposes the problem formulation of the bounded L₂-gain for coupling attenuation and its equivalent multi-player zero-sum differential game. Section 4 transforms this zero-sum differential game into the solution of an algebraic Riccati equation and proves the ultimately uniform boundedness of the local neighborhood error, conservatively. The simulation results and conclusion is presented in Section 5 and Section 6, respectively.

2. Preliminaries and Problem Formulation

2.1. Graph Theory

In this paper, the multi-agent directed communication network is depicted. A directed connected graph is defined as $G(V,E,A)$, where $V=\left\{v_1,v_2,\cdots ,v_n\right\}$ represents a finite non-empty set of nodes, $E\subseteq V\times V$ is the ordered set of nodes pairs and $A=[a_{ij}]$ is the adjacency matrix. If node $v_i$ can receive the information from node $v_j$, then the node pairs ${\overline{v}}_{ij}=(v_i,v_j)\in E$, and node $v_j$ is called a neighbor of node $v_i$. The neighbor set of node $v_i$ is represented by $N_i=\left\{ v_j|(v_i,v_j)\in E \right\}$. Correspondingly, the adjacency matrix element $a_{ij}=1$ when $v_j\in N_i$, otherwise $a_{ij}=0$. The graph Laplacian matrix is defined as $L=D-A$, whose row sums are equal to zero [25]. Diagonal matrix $D=diag(d_i)$ is the in-degree matrix, where $d_i={\displaystyle {\sum }_{j\in N_i}{a}_{ij}}$ is the in-degree of node $v_i$.

Definition 1.

A directed graph is called as strongly connected if there is a directed path for any a pair of distinct nodes$(v_i,v_j)$, where the directed path is the edge sequence$(v_{i1},v_{i2})$, $(v_{i2},v_{i3})$,...,$(v_{ik},v_j)$.

Definition 2 [17]. 

A directed tree is a connected graph where every node except the root node, has an in-degree equal to one. The graph is called to have a spanning tree if a subset of the node pairs constructs a directed tree.

In this paper, $\overline{\lambda }(B)$ and $\underset{\_}{\lambda }(B)$ represent the maximum and minimum singular values of the matrix $B$, respectively.

2.2. Problem Formulation

Considering the Multiple Autonomous Unmanned linear systems (MAUS) constructed by the directed communication graph $G(V,E,A)$ having N agents, the dynamics of each agent is described in the following:

$${\dot{x}}_i=A{x}_i+B{u}_i$$

(1)

where $x_i\in R^n, u_i\in R^{m_i}$ are states and control inputs of node i, respectively. The cooperative control of homogeneous systems is investigated in this paper and the leader node $x_0\in R^n$ is set to satisfy the following dynamic

$${\dot{x}}_0=A{x}_0$$

(2)

The problem of MAUS synchronization is designing control protocols $u_i$ for each agent so that states of each node track the leader node, i.e., $\Vert x_0-x_i\Vert \to 0, \forall i$.

The neighborhood error for each node is defined as [26]

$${\delta }_i={\displaystyle \sum _{j\in N_i}{a}_{ij}(x_i-x_j)+g_i(x_i-x_0)}$$

(3)

where $g_i\ge 0$ denotes the pinning gain and there is at least one node that has a link to the leader node.

For the neighborhood error (3), the overall neighborhood error vector of graph $G(V,E,A)$ is given by

$$\delta =((L+\overline{G})\otimes I_n)(x_i-{\underset{\_}{x}}_0)$$

(4)

where $x={[x_1^T{x}_2^T \cdots x_N^T]}^T$ and $\delta ={[{\delta }_1^T{\delta }_2^T \cdots {\delta }_N^T]}^T$ denote the global state vector and global error vector, respectively. Moreover, for ${\underset{\_}{x}}_0=\underset{\_}{I}{x}_0\in R^{nN}$ with $\underset{\_}{I}=1_N\otimes I_n\in R^{nN\times n}$, $I_n$ denotes the n dimensional identity matrix and $1_N$ denotes the N-vector of ones. The symbol $\otimes$ is the Kronecker product [27]. $\overline{G}=diag(g_1,g_2,\cdots ,g_N)$ as a diagonal matrix represents the connection between all agents and the leader node.

The overall synchronization error is

$$\epsilon =(x-{\underset{\_}{x}}_0)\in R^{nN}$$

(5)

Assumption 1.

The communication graph is strongly connected, i.e., there is a directed path for any a pair of distinct nodes.

On the basis of Assumption 1, if $\overline{G}\ne 0$, then $g_i\ne 0$ for at least one. In this case, the matrix $L+\overline{G}$ is non-singular and the real parts of all eigenvalues are positive [26]. The following lemma can be obtained, which shows that the overall neighborhood error vector $\delta$ is positively correlated with the overall synchronization error $\epsilon$.

Lemma 1.

If the communication graph is strongly connected and$\overline{G}\ne 0$, the synchronization errors are bounded, as follows

$$\delta /\overline{\lambda }(L+\overline{G})\le \Vert \epsilon \Vert \le \delta /\underset{\_}{\lambda }(L+\overline{G})$$

(6)

Furthermore, $\delta \equiv 0$ if and only if all nodes are synchronized, i.e.,

$$\epsilon =(x-{\underset{\_}{x}}_0)=0$$

(7)

The dynamics of the local neighborhood tracking errors are given as

$${\dot{\delta }}_i={\displaystyle \sum _{j\in N_i}{a}_{ij}({\dot{x}}_i-{\dot{x}}_j)+g_i({\dot{x}}_i-{\dot{x}}_0)}$$

(8)

Substituting (1) and (2) into the above equation, it can be obtained that

$${\dot{\delta }}_i=A{\delta }_i+(d_i+g_i)B_i{u}_i-{\displaystyle \sum _{j\in N_i}{a}_{ij}{B}_j{u}_j}$$

(9)

It can be seen that the dynamics of the local neighborhood error of each agent i is affected by multiple control inputs from node i and its adjacent nodes. The whole MAUS with the communication graph $G(V,E,A)$ presents a complex coupling relationship. It is quite intricate to solve the optimal control problem of dynamic (9) affected by multi-coupling.

3. Multi-Player Zero-Sum Differential Game for Decoupled Multi-Agent System

3.1. The Bounded L₂-Gain Problem for Coupling Attenuation of Multi-Agent System

For decoupling, the inputs from adjacent nodes in the dynamics (9) are replaced by the virtual coupling actions $w_i(t)={[{\stackrel{⌢}{u}}_j^T]}_{j\in N_j}^T$ which is regarded as the external disturbances. The performance output is defined as $z_i(t)={[{\delta }_i^T{u}_i^T]}^T$. It is desired to designed the control protocol $u_i$ to achieve synchronization while satisfying the follow bounded L₂-gain condition for the coupling actions with a given ${\gamma }_i>0$

$$\begin{array}{l}{\displaystyle {\int }_0^T {\Vert z_i(t)\Vert }^2 dt}={\displaystyle {\int }_0^T ({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i) dt}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\le {\gamma }_i^2{\displaystyle {\int }_0^T {\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j} dt}+\beta ({\delta }_i(0))\end{array}$$

(10)

where, $\beta (\cdot )$ is a bounded function such that $\beta (0)=0$, $Q_i>0,R_{ii}>0,R_{ij}>0$. ${\gamma }_i^{*}$ is defined as the minimum value of ${\gamma }_i$ while the bounded L₂-gain condition (10) is satisfied.

3.2. Multi-Player Zero-Sum Differential Game

The following equation is used to define the following performance index function for each agent.

$$J_i({\delta }_i(0) ,u_i , {\stackrel{⌢}{u}}_{-i})=\frac{1}{2}{\displaystyle {\int }_0^{\infty } {\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i- {\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}dt}$$

(11)

where ${\stackrel{⌢}{u}}_{-i}$ denotes the virtual coupling control inputs from neighboring nodes, i.e., ${\stackrel{⌢}{u}}_{-i}=\left\{ {\stackrel{⌢}{u}}_j| j\in N_i\right\}$. It should be noted that the main difference from [17] is that the coupling control inputs from neighboring nodes are regarded as the virtual external disturbances directly, which greatly simplifies the design of the control protocol $u_i$.

The solution for the bounded L₂-gain problem for coupling attenuation depicted in Section 3.1 can be equivalent to the Nash equilibrium solution of the multi-player zero-sum game-base on the performance index function (11). That is

$$V_i({\delta }_i(0))=\underset{u_i}{\min }\underset{{\stackrel{⌢}{u}}_{-i}}{\max }{J}_i({\delta }_i(0) ,u_i , {\stackrel{⌢}{u}}_{-i})$$

(12)

In this multi-player zero-sum game, the goal of $u_i$ is to minimize the value $V_i({\delta }_i(0))$. On the contrary, the virtual coupling inputs ${\stackrel{⌢}{u}}_{-i}$ are assumed to maximize the value. This game has a unique solution if a game-theoretic saddle point $(u_i^{*} , {\stackrel{⌢}{u}}_{-i}^{*})$ exists, i.e.,

$$V_i^{*}({\delta }_i(0))=\underset{u_i}{\min }\underset{{\stackrel{⌢}{u}}_{-i}}{\max }{J}_i({\delta }_i(0) ,u_i , {\stackrel{⌢}{u}}_{-i})=\underset{{\stackrel{⌢}{u}}_{-i}}{\max }\underset{u_i}{\min }{J}_i({\delta }_i(0) ,u_i , {\stackrel{⌢}{u}}_{-i})$$

(13)

Accordingly, the value $V_i^{*}({\delta }_i(0))$ in the above equation is the value of the zero-sum game and satisfies the following Nash equilibrium condition for all policies $u_i^{} , {\stackrel{⌢}{u}}_{-i}^{}$

$$J_i({\delta }_i(0) ,u_i^{*} , {\stackrel{⌢}{u}}_{-i})\le J_i({\delta }_i(0) ,u_i , {\stackrel{⌢}{u}}_{-i})\le J_i({\delta }_i(0) ,u_i , {\stackrel{⌢}{u}}_{-i}^{*})$$

(14)

When the policies $u_i^{} , {\stackrel{⌢}{u}}_{-i}^{}$ are selected, the value function of node i can yield

$$V_i({\delta }_i(t) ,u_i , {\stackrel{⌢}{u}}_{-i})=\frac{1}{2}{\displaystyle {\int }_t^{\infty } {\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i- {\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}dt}$$

(15)

Differential equivalents to each value function are given as

$$\begin{array}{l}0=\frac{1}{2}\left({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i- {\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}\right)\\ \hspace{1em} +\nabla V_i^T(A{\delta }_i+(d_i+g_i)B_i{u}_i-{\displaystyle \sum _{j\in N_i}{a}_{ij}{B}_j{\stackrel{⌢}{u}}_j}) ,\hspace{1em}{V}_i(0)=0 ,\hspace{1em}i\in N\end{array}$$

(16)

where $\nabla V_i=\frac{\partial V_i}{\partial {\delta }_i}\in R^n$ denotes the gradient vector. The Hamiltonian functions are defined as follows,

$$\begin{array}{l}{H}_i({\delta }_i,\nabla V_i,u_i,{\stackrel{⌢}{u}}_{-i})\equiv \frac{1}{2}\left({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i- {\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}\right)\\ \hspace{1em} +\nabla V_i^T(A{\delta }_i+(d_i+g_i)B_i{u}_i-{\displaystyle \sum _{j\in N_i}{a}_{ij}{B}_j{\stackrel{⌢}{u}}_j}) \end{array}$$

(17)

Under certain policies $u_i^{} , {\stackrel{⌢}{u}}_{-i}^{}$, the partial differential equation $H_i({\delta }_i,\nabla V_i,u_i,{\stackrel{⌢}{u}}_{-i})=0$ has a unique solution $V_i({\delta }_i)$. The principle of optimality gives

$$\begin{array}{l}\frac{\partial H_i({\delta }_i,\nabla V_i,u_i,{\stackrel{⌢}{u}}_{-i})}{\partial u_i}=0\Rightarrow u_i=-(d_i+g_i)R_{{}_{ii}}^{-1}{B}_i^T\nabla V_i\\ \frac{\partial H_i({\delta }_i,\nabla V_i,u_i,{\stackrel{⌢}{u}}_{-i})}{\partial {\stackrel{⌢}{u}}_j}=0\Rightarrow {\stackrel{⌢}{u}}_j=-\frac{1}{{\gamma }_{{}_i}^2}{a}_{ij}{R}_{{}_{ij}}^{-1}{B}_j^T\nabla V_i,\hspace{1em}j\in N_i\end{array}$$

(18)

If the $V_i^{*}$ is the Nash equilibrium solution of the multi-player zero-sum game, that is

$$V_i^{*}({\delta }_i)=\underset{u_i}{\min }\underset{{\stackrel{⌢}{u}}_{-i}}{\max }{J}_i({\delta }_i ,u_i , {\stackrel{⌢}{u}}_{-i})$$

(19)

we can obtain

$$\underset{u_i}{\min }\underset{{\stackrel{⌢}{u}}_{-i}}{\max }{H}_i({\delta }_i,\nabla V_i^{*},u_i,{\stackrel{⌢}{u}}_{-i})=0$$

(20)

Substituting the optimal strategy determined by (18) into (20), the coupled Hamilton–Jacobi (HJ) equations yield

$$\begin{array}{l}0=\frac{1}{2}{\delta }_i^T{Q}_i{\delta }_i+\frac{1}{2}{(d_i+g_i)}^2\nabla V_i^T{B}_i{R}_{ii}^{-1}{B}_i^T\nabla V_i-\frac{1}{2{\gamma }_i^2}{\displaystyle \sum _{j\in N_i}{a}_{ij}^2\nabla V_i^T{B}_j{R}_{ij}^{-1}{B}_j^T\nabla V_i}\\ \hspace{1em} +\nabla V_{{}_i}^T(A{\delta }_i-{(d_i+g_i)}^2{B}_i{R}_{ii}^{-1}{B}_i^T\nabla V_i+\frac{1}{{\gamma }_{{}_i}^2}{\displaystyle \sum _{j\in N_i}{a}_{ij}^2{B}_j{R}_{ij}^{-1}{B}_j^T\nabla V_i}) , V_i(0)=0 , i\in N\end{array}$$

(21)

For a given solution $V_i^{*}$, in order to define $u_i^{*}=u_i(V_i^{*})$ and ${\stackrel{⌢}{u}}_j^{*}={\stackrel{⌢}{u}}_j(V_i^{*})$ in the same way as (18), (21) can be written as

$$H_i({\delta }_i,\nabla V_i^{*},u_i^{*},{\stackrel{⌢}{u}}_{-i}^{*})=0, V_i^{*}(0)=0$$

(22)

Lemma 2.

For any policies$u_i,{\stackrel{⌢}{u}}_{-i}$, the following equation holds

$$H_i({\delta }_i,\nabla V_i^{*},u_i,{\stackrel{⌢}{u}}_{-i})=\frac{1}{2}{(u_i-u_i^{*})}^T{R}_{ii}(u_i-u_i^{*})-{\displaystyle \sum _{j\in N_i}\frac{{\gamma }_i}{2}{({\stackrel{⌢}{u}}_j-{\stackrel{⌢}{u}}_j^{*})}^T{R}_{ij}({\stackrel{⌢}{u}}_j-{\stackrel{⌢}{u}}_j^{*})}$$

(23)

Proof of Lemma 2.

Substituting $u_i,{\stackrel{⌢}{u}}_{-i}$ for $u_i^{*},{\stackrel{⌢}{u}}_{-i}^{*}$ in (22)

$$\begin{array}{l}{H}_i({\delta }_i,\nabla V_i^{*},u_i,{\stackrel{⌢}{u}}_{-i})=\nabla V_i^{*T}(A{\delta }_i+(d_i+g_i)B_i{u}_i-{\displaystyle \sum _{j\in N_i}{a}_{ij}{B}_j{\stackrel{⌢}{u}}_j}\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +(d_i+g_i)B_i{u}_i^{*}-(d_i+g_i)B_i{u}_i^{*}-{\displaystyle \sum _{j\in N_i}{a}_{ij}{B}_j{\stackrel{⌢}{u}}_j^{*}}+{\displaystyle \sum _{j\in N_i}{a}_{ij}{B}_j{\stackrel{⌢}{u}}_j^{*}})\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +\frac{1}{2}\left({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i-{\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +\frac{1}{2}\left(u_i^{*T}{R}_{ii}{u}_i^{*}-u_i^{*T}{R}_{ii}{u}_i^{*}-{\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^{*T}{R}_{ij}{\stackrel{⌢}{u}}_j^{*}}+{\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^{*T}{R}_{ij}{\stackrel{⌢}{u}}_j^{*}}\right)\end{array}$$

(24)

Substituting $H_i({\delta }_i,\nabla V_i^{*},u_i^{*},{\stackrel{⌢}{u}}_{-i}^{*})=0$ into (24), we can obtain

$$\begin{array}{l}{H}_i({\delta }_i,\nabla V_{{}_i}^{*},u_i,{\stackrel{⌢}{u}}_{-i})=\nabla V_i^{*T}((d_i+g_i)B_i(u_i-u_i^{*})-{\displaystyle \sum _{j\in N_i}{a}_{ij}{B}_j({\stackrel{⌢}{u}}_j-{\stackrel{⌢}{u}}_j^{*})})\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +\frac{1}{2}\left(u_i^T{R}_{ii}{u}_i-u_i^{*T}{R}_{ii}{u}_i^{*}\right)-\frac{1}{2}\left({\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}-{\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^{*T}{R}_{ij}{\stackrel{⌢}{u}}_j^{*}}\right)\end{array}$$

(25)

Completing the squares in (25) gives (23) upon the relationship between $u_i^{*},{\stackrel{⌢}{u}}_{-i}^{*}$ and $\nabla V_i^{*}$. □

Remark 1.

The polices${\stackrel{⌢}{u}}_{-i}^{}$ in Section 3 are not the real policies of neighboring nodes. These are only defined as the virtual coupling input from neighboring nodes, which are regarded as the external disturbances and have the same channels as the control inputs of neighboring nodes. In this way, the bounded L₂-gain attenuation for the real coupling inputs from neighboring nodes can be realized. In addition, the complex relationships among agents are decoupled virtually during the control protocol design process, and the solving process of zero-sum game is effectively simplified. The coupled HJ equation of each agent is independent of each other.

4. Solution to Bounded L₂-Gain Problem for the Coupling Attenuation and the Equivalent Algebraic Riccati Equation

4.1. Solution to Bounded L₂-Gain Problem for Coupling Attenuation

In this subsection, the control policy $u_i^{}$ is found to guarantee the condition (10) holds for a prescribed ${\gamma }_i>0$ and ${\stackrel{⌢}{u}}_{-i}^{}\in L_2[0,\infty )$. The following Theorem 1 shows that the solution of a coupled HJ equation (22) is actually the solution to the bounded L₂-gain problem for coupling attenuation.

Theorem 1.

Let${\gamma }_i\ge {\gamma }_i^{*}$. Suppose the coupled HJ equation (22) has a smooth positive definite solution $V_i^{*}>0,i\in N$. The control policy is selected as $u_i^{*}=u_i(V_i^{*})$, given by (18) in terms of$V_i^{*}$. The bounded L₂-gain condition (10) holds for all ${\stackrel{⌢}{u}}_{-i}^{}\in L_2[0,\infty )$.

Proof of Theorem 1.

According to Lemma 2

$$\begin{array}{l}{H}_i({\delta }_i,\nabla V_i^{*},u_i,{\stackrel{⌢}{u}}_{-i})\\ =\frac{1}{2}\left({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i-{\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}\right)+\frac{d{V}_i^{*}}{dt}\\ =\frac{1}{2}{(u_i-u_i^{*})}^T{R}_{ii}(u_i-u_i^{*})-{\displaystyle \sum _{j\in N_i}\frac{{\gamma }_i}{2}{({\stackrel{⌢}{u}}_j-{\stackrel{⌢}{u}}_j^{*})}^T{R}_{ij}({\stackrel{⌢}{u}}_j-{\stackrel{⌢}{u}}_j^{*})}\end{array}$$

(26)

Selecting $u_i=u_i^{*}$, we can obtain that

$$\begin{array}{l}\frac{1}{2}\left({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i-{\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}\right)+\frac{d{V}_i^{*}}{dt}\\ \hspace{1em}\hspace{1em}\hspace{1em}=-{\displaystyle \sum _{j\in N_i}\frac{{\gamma }_i}{2}{({\stackrel{⌢}{u}}_j-{\stackrel{⌢}{u}}_j^{*})}^T{R}_{ii}({\stackrel{⌢}{u}}_j-{\stackrel{⌢}{u}}_j^{*})}\le 0\end{array}$$

(27)

Integrating (27) yields,

$$\begin{array}{l}\frac{1}{2}{\displaystyle {\int }_0^T\left({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i-{\gamma }_i^2{\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}\right)} dt\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +V_i^{*}({\delta }_i(T))-V_i^{*}({\delta }_i(0))\le 0\end{array}$$

(28)

$V_i^{*}$ is a smooth positive definite solution, i.e., $V_i^{*}({\delta }_i(T))\ge 0$, one has

$${\displaystyle {\int }_0^T \left({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i\right)} dt\le {\gamma }_i^2{\displaystyle {\int }_0^T {\displaystyle \sum _{j\in N_i}{\stackrel{⌢}{u}}_j^T{R}_{ij}{\stackrel{⌢}{u}}_j}} dt+V_i^{*}({\delta }_i(0))$$

(29)

Hence, the bounded L₂-gain condition (10) for coupling attenuation is satisfied. □

4.2. The Equivalent Algebraic Riccati Equation

It can be seen from the above results that the Nash equilibrium solution can be obtained by solving the coupled HJ equation (22). In this subsection, it will be shown that the coupled HJ equation (22) can be equivalent to an Algebraic Riccati equation (ARE).

Defining the optimal value function $V_i^{*}={\delta }_i^T{P}_i{\delta }_i$, the corresponding optimal $u_i^{*}$ and ${\stackrel{⌢}{u}}_{-i}^{*}$ can be obtained as

$$\begin{array}{l}{u}_i^{*}=-2(d_i+g_i)R_{ii}^{-1}{B}_i^T{P}_i{\delta }_i\\ {\stackrel{⌢}{u}}_j^{*}=-\frac{2}{{\gamma }_{{}_i}^2}{a}_{ij}{R}_{ij}^{-1}{B}_j^T{P}_i{\delta }_i , j\in N_i\end{array}$$

(30)

Substituting (30) and $V_i^{*}={\delta }_i^{T}P{\delta }_i$ into (21) yields

$$\begin{array}{ll}{\delta }_i^T(P_{i}A+A^T{P}_i){\delta }_i+\frac{1}{2}{\delta }_i^T{Q}_i{\delta }_i& -2{(d_i+g_i)}^2{\delta }_i^T{P}_i{B}_i{R}_{ii}^{-1}{B}_i^T{P}_i{\delta }_i\\ & +\frac{2}{{\gamma }_{ij}^2}{\delta }_i^T({\displaystyle \sum _{j\in N_i}{a}_{ij}^2{P}_i{B}_j{R}_{ij}^{-1}{B}_j^T{P}_i}){\delta }_i=0\end{array}$$

(31)

The above equation can be equivalent to

$$\begin{array}{l}{P}_{i}A+A^T{P}_i+\frac{1}{2}{Q}_i-2{(d_i+g_i)}^2{P}_i{B}_i{R}_{ii}^{-1}{B}_i^T{P}_i\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} +\frac{2}{{\gamma }_{ij}^2}{\displaystyle \sum _{j\in N_i}{a}_{ij}^2{P}_i{B}_j{R}_{ij}^{-1}{B}_j^T{P}_i}=0\end{array}$$

(32)

Defining the integrated matrix as

$$\begin{array}{l}R=diag(\frac{R_{ii}}{2{(d_i+g_i)}^2},-\frac{{\gamma }_i^2{R}_{i{j}_1}}{2a_{i{j}_1}},-\frac{{\gamma }_i^2{R}_{i{j}_2}}{2a_{i{j}_2}},\cdots ,-\frac{{\gamma }_i^2{R}_{i{j}_{d_i}}}{2a_{i{j}_{d_i}}})\\ B=[B_i{B}_{j_1}{B}_{j_2}\cdots B_{j_{d_i}}], j_1, j_2, \cdots ,j_{d_i}\in N_i\end{array}$$

Then, (32) can be rewritten as the ARE

$$P_{i}A+A^T{P}_i+\frac{1}{2}{Q}_i-P_{i}B{R}^{-1}{B}^T{P}_i=0$$

(33)

Theorem 2.

Assume that all the real control policies$u_{-i}=\left\{ u_j| j\in N_i\right\}$ of neighboring agents satisfy the bounded condition as follows,

$${\Vert u_j\Vert }_2=u_j^T{u}_j\le \xi , j\in N_i$$

(34)

Selecting the first equation of (30) as the control policies$u_i^{}$, based on the solution of ARE (34). The local neighborhood error vector will ultimately and uniformly enter the following bounded invariant set

$${\mathsf{\Omega }}_i=\{ {\delta }_i|{\Vert {\delta }_i\Vert }_2\le \frac{{\gamma }_i^2\xi {\displaystyle \sum _{j\in N_i}\overline{\lambda }(R_{ij})}}{2\underset{\_}{\lambda }({\overline{Q}}_i)} \}$$

(35)

where${\Vert {\delta }_i\Vert }_2$denotes the Euclidean norm of${\delta }_i$and${\overline{Q}}_i$is the positive definite matrix as${\overline{Q}}_i=\frac{1}{2}{Q}_i+2{(d_i+g_i)}^2{P}_i{B}_i{R}_{{}_{ii}}^{-1}{B}_i^T{P}_i$.

Proof of Theorem 2.

Selecting the optimal positive value function $V_i^{*}={\delta }_i^T{P}_i{\delta }_i\ge 0$, in terms of the solution of the coupled HJ equation (21), as the Lyapunov function. According to Lemma 2, the derivative of $V_i^{*}$ is

$$\begin{array}{l}\frac{d{V}_i^{*}}{dt}=-\frac{1}{2}\left({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i-{\gamma }_i^2{\displaystyle \sum _{j\in N_i}{u}_j^T{R}_{ij}{u}_j}\right)\\ \hspace{1em}\hspace{1em} +\frac{1}{2}{(u_i-u_i^{*})}^T{R}_{ii}(u_i-u_i^{*})-{\displaystyle \sum _{j\in N_i}\frac{{\gamma }_i}{2}{(u_j-{\stackrel{⌢}{u}}_j^{*})}^T{R}_{ij}(u_j-{\stackrel{⌢}{u}}_j^{*})}\end{array}$$

(36)

Selecting $u_i^{}=u_i^{*}$ as the first equation of (30) yields

$$\begin{array}{l}\frac{d{V}_i^{*}}{dt}=-\frac{1}{2}{\delta }_i^T{Q}_i{\delta }_i-2{(d_i+g_i)}^2{\delta }_i^T{P}_i{B}_i{R}_{ii}^{-1}{B}_i^T{P}_i{\delta }_i\\ \hspace{1em}\hspace{1em} +\frac{{\gamma }_i^2}{2}{\displaystyle \sum _{j\in N_i}{u}_j^T{R}_{ij}{u}_j}-{\displaystyle \sum _{j\in N_i}\frac{{\gamma }_i}{2}{(u_j-{\stackrel{⌢}{u}}_j^{*})}^T{R}_{ij}(u_j-{\stackrel{⌢}{u}}_j^{*})}\\ \hspace{1em} \le -{\delta }_i^T(\frac{1}{2}{Q}_i+2{(d_i+g_i)}^2{P}_i{B}_i{R}_{ii}^{-1}{B}_i^T{P}_i){\delta }_i+\frac{{\gamma }_i^2\xi {\displaystyle \sum _{j\in N_i}\overline{\lambda }(R_{ij})}}{2} \end{array}$$

(37)

Let $\frac{d{V}_i^{*}}{dt}\le 0$, we can realize that

$${\delta }_i^T{\delta }_i={\Vert {\delta }_i\Vert }_2\ge \frac{{\gamma }_i^2\xi {\displaystyle \sum _{j\in N_i}\overline{\lambda }(R_{ij})}}{2\underset{\_}{\lambda }(\frac{1}{2}{Q}_i+2{(d_i+g_i)}^2{P}_i{B}_i{R}_{ii}^{-1}{B}_i^T{P}_i)}$$

(38)

Defining ${\overline{Q}}_i=\frac{1}{2}{Q}_i+2{(d_i+g_i)}^2{P}_i{B}_i{R}_{ii}^{-1}{B}_i^T{P}_i$ yields the bounded invariant set (35). □

Remark 2.

Theorem 2 shows the ultimately uniform boundedness (UUB) of the local neighborhood error vector${\delta }_i$. According to the bounded invariant sets ${\mathsf{\Omega }}_i$, the bound of${\delta }_i$can be arbitrarily small by presetting the matrix$Q_i$,$R_{ii}$and$R_{ij}$in the performance index function (11). In fact, this result is conservative because the term$-{\displaystyle \sum _{j\in N_i}\frac{{\gamma }_i}{2}{(u_j-{\stackrel{⌢}{u}}_j^{*})}^T{R}_{ij}(u_j-{\stackrel{⌢}{u}}_j^{*})}$in$\frac{d{V}_i^{*}}{dt}$is omitted. The real control inputs$u_j^{}$ of adjacent agents differ greatly from${\stackrel{⌢}{u}}_j^{*}$in fact, which guarantees the negative characterization of$\frac{d{V}_i^{*}}{dt}$. Therefore, the simulation results in the next section show that the local neighborhood error vector can converge asymptotically and uniformly to the origin.

Remark 3.

In a practical application, matrix$Q_i$,$R_{ii}$and$R_{ij}$and parameter${\gamma }_i$can be selected according to engineering performance requirements. If a high convergence speed and synchronization accuracy are required,$Q_i$can be selected to make its eigenvalues large; if a low control energy consumption is required,$R_{ii}$can be selected to make its eigenvalues large; the coupling attenuation level can be adjusted by adjusting matrix$R_{ij}$and${\gamma }_i$. It should be noted that,$Q_i$,$R_{ii}$,$R_{ij}$and${\gamma }_i$must satisfy the condition in Theorem 1, so that the coupled HJ equation (22) has a smooth positive definite solution$V_i^{*}>0$. That is, (33) has a positive definite solution$P_i$.

5. Simulation Results

This section shows the effectiveness of the equivalent ARE approach described in Section 4 and Theorem 2. The simulation is realized in MATLAB/Simulink. Consider a class of Multiple Autonomous Unmanned homogeneous linear systems referring to [5] which is shown as follows

$${\dot{x}}_i=A{x}_i+B_i{u}_i$$

where $A=\left[\begin{array}{cc}0& 1\\ -1& 0\end{array}\right]$, $B_1=\left[\begin{array}{c}2\\ 1\end{array}\right]$, $B_2=\left[\begin{array}{c}2\\ 3\end{array}\right]$, $B_3=\left[\begin{array}{c}2\\ 2\end{array}\right]$ and $B_3=\left[\begin{array}{c}1\\ 1\end{array}\right]$, with the Leader dynamics ${\dot{x}}_0=A{x}_0$. The communication digraph structure is shown in Figure 1. The edge weights and the pinning gains are taken equal to 1.

Figure 1. The communication digraph structure.

The selected the weight matrices in (11) are $R_{11}=9$, $R_{14}=1$, $R_{22}=9$, $R_{21}=1$, $R_{33}=9$, $R_{32}=1$, $R_{44}=9$, $R_{43}=1$ and $Q_1=Q_2=Q_3=Q_4=\left[\begin{array}{cc}8& 0\\ 0& 8\end{array}\right]$. The bounded L₂-gain coefficient in (11) for each agent are preset as ${\gamma }_1=1.75$, ${\gamma }_2=3.75$, ${\gamma }_3=4.5$ and ${\gamma }_4=6.25$. The cooperative control protocol of each agent is implemented, as in Section 4.2, where the solution of ARE (33) is

$$\begin{gathered} P_1=\left[\begin{array}{cc}4.8954& -5.6084\\ -5.6084& 9.4934\end{array}\right],P_2=\left[\begin{array}{cc}41.9228& -7.3129\\ -7.3129& 2.6742\end{array}\right],P_3=\left[\begin{array}{cc}14.4257& -9.363\\ -9.363& 10.4565\end{array}\right], \\ {P}_4=\left[\begin{array}{cc}17.4696& -0.5066\\ -0.5066& 13.6\end{array}\right]. \end{gathered}$$

Remark 4.

In the process of a simulation design, the${\gamma }_i$should be gradually reduced to search for a feasible and high coupling attenuation level under the premise that (33) has a positive definite solution $P_i$. Using the ARE solver in MATLAB, it is very convenient to solve (33) and obtain$P_i$. Then, the design of the coupling attenuation controller can be completed according to (30).

To elevate the Bounded L₂-gain problem for the coupling attenuation, the following variable $C_{\gamma }$ is introduced based on (29)

$$C_{\gamma }={\displaystyle {\int }_0^T \left({\delta }_i^T{Q}_i{\delta }_i+u_i^T{R}_{ii}{u}_i\right)} dt-{\gamma }_i^2{\displaystyle {\int }_0^T {\displaystyle \sum _{j\in N_i}{u}_j^T{R}_{ij}{u}_j}} dt-V_i^{*}({\delta }_i(0))$$

(39)

That is, $C_{\gamma }\le 0$, which means that the Bounded L₂-gain condition (10) is satisfied.

The local neighborhood error vector of each agent is shown in Figure 2. Figure 3 is the 3-D phase plane plot of the system’s evolution for agents 1, 2, 3, 4 and leader 0. The $C_{\gamma }$ of the node 1 agent is shown in Figure 4. As can be seen from Figure 2 and Figure 3, the neighborhood error vector can converge asymptotically and uniformly to the origin and all agents in the communication digraph are eventually synchronized, which is also consistent with Remark 2. Figure 4 shows that $C_{\gamma }$ is always negative, which is equivalent to that the node 1 agent satisfies the bounded L₂-gain condition with ${\gamma }_1=1.75$ for the coupling attenuation. The effectiveness of the proposed method is thus verified.

Figure 2. The local neighborhood error vector of each agent: (a) The node 1 agent; (b) The node 2 agent; (c) The node 3 agent; (d) The node 4 agent.

Figure 3. The 3-D phase plane plot of the system’s evolution for agents 1, 2, 3, 4 and leader 0.

Figure 4. The $C_{\gamma }$ of node 1 agent.

6. Conclusions

This paper provides a novel idea for the synchronization control of Multiple Autonomous Unmanned linear systems, in which the local neighborhood error dynamic’s coupling part is considered as the virtual external disturbance, so as to decouple the multi-agent cooperative control problem into a relatively independent bounded L₂-gain problem for coupling attenuation. The optimal control theory and differential game theory is utilized to formulate the bounded L₂-gain problem into a centralized multi-player zero-sum game. It is shown that the solution to the multi-player zero-sum game is equivalent to the solution of a coupled HJ equation. It is also shown that the coupled HJ equation can be transformed into an algebraic Riccati equation (ARE) and the solution guarantees the ultimately uniform boundedness (UUB) of the local neighborhood error vector under conservative conditions. The law of parameters selection is summarized. The simulation results show that the proposed method can ensure that the local neighborhood error vectors converge asymptotically to the origin, that is, the multiple autonomous unmanned linear systems can achieve final synchronization, which demonstrates that the UUB of errors is conservative. Meanwhile, the bounded L₂-gain condition for the coupling attenuation can be guaranteed.

This proposed method is suitable for the cooperative consensus control of various homogeneous multiple autonomous unmanned linear systems, such as an underwater robot swarm and aerial UAV swarm. Future work will focus on extending this method to nonlinear multi-agent systems and more serious models with uncertainties.