A Dropout Compensation ILC Method for Formation Tracking of Heterogeneous Multi-Agent Systems with Loss of Multiple Communication Packets

: In this paper, the formation tracking problem for heterogeneous multi-agent systems with loss of multiple communication packets is considered using the iterative learning control (ILC) method. A dropout compensation ILC method is presented to construct e ﬀ ective distributed iterative learning protocols. The convergence conditions are given based on the frequency-domain analysis by using the general Nyquist stability criterion and Greshgorin’s disk theorem. The results show that the multi-agent system with di ﬀ erent packet loss rate can achieve formation tracking without reducing the convergence speed. Numerical simulation results show the e ﬀ ectiveness of the proposed dropout compensation ILC method.


Introduction
In recent years, with the rapid development of communication and computing technology, people pay more and more attention to the control of multi-agent systems [1][2][3]. "Agent" can be broadly defined as a computing entity that is adaptive, autonomous, and has a certain life cycle in a particular environment. Drones, driverless cars, robots, etc., can all be called "Agent". Apart from these, single-agent has played an important role in aerospace, military, and real life. Furthermore, the development of intelligent control, modern communication technologies, artificial intelligence concepts, etc., as well as people inspired by the phenomenon of animal cluster activity in nature, have pushed agent control research into a new stage of development.
By working together, multi-agent can accomplish some parallel, complex tasks, more efficiently than single-agent. For instance, multiple satellites orbiting the Earth to complete designated tasks and multiple drones can work together to transport goods in disaster relief.
In recent years, many researchers have considered applying iterative learning control (ILC) to multi-agent systems with repetitive work. ILC is a tracking control method for systems that require high precision and repeat the same actions. In a study by Ahn and Chen [4], ILC was applied to the formation control of a multi-agent system for the first time. Since then, more and more researchers have applied ILC to the practical application of multi-agent system. Ahn, Moore, and Chen [5] proposed an ILC law to maintain the flight path of satellite formation. Since the satellite rotates the earth periodically, the position related interference can be considered as the time periodic interference. According to Chen and Jia [6], multiple robots can guarantee formation using iterative learning. Furthermore, in order to

Preliminaries
The interaction topologies in a multi-agent system are modeled by an undirected graph G = (V, E, A), where V = {v 1 , . . . , v n } is the set of vertices, E ⊆ V × V is the set of the edge, and A = [a ij ], a ij ≥ 0 is the weighted adjacency matrix of the graph G. If (v i , v j ) ∈ E, a ij = a ji > 0, otherwise, a ij = a ji = 0. Each edge (v i , v j ) represents information that can be transmitted between the ith agent and the jth agent. In addition, we assume that a ii = 0. The neighboring set of agent, v i , is The Laplacian matrix L = [l ij ] ∈ R n×n of graph G is denoted as L = D − A, where D diag{d 1 , · · · , d n } with d i = n j=1 a ij . A path in the undirected graph G is denoted as a finite sequence v i 1 , v i 2 , · · · , v i j and (v i l , v i l+1 ) ∈ E, l = 1, 2, · · · , j − 1.

Control Objective
Consider the heterogeneous linear SISO (Single Input and Single Output) discrete-time multi-agent system of n agents with topology graph G. The dynamic of the ith agent is modeled by the following dynamic system.
x i,k (t + 1) = A i x i,k (t) + B i u i,k (t) y i,k (t) = C i x i,k (t) (1) where i = 1, 2, · · · , n is the index of the ith agent, t = 0, 1, 2, · · · , T is the discrete-time index, and k = 1, 2, · · · is the index of iteration. x i,k (t) ∈ R is the state of the ith agent in time t and kth iteration. Similarly, u i,k (t) ∈ R is the control input signal and y i,k (t) ∈ R is the output signal. Multiple agents have different dynamics which are denoted as A i , B i , and C i . Taking the Z-transformation of Equation (1), we get where z = e jω , ω ∈ [−π, π) with j = √ −1 and ω is frequency, and x i,k (0) is the initial state of ith agent in the kth iteration; let G i (z) (2) can be rewritten as The desired formation is defined as the relative state values between agents. The relative state values could be position, velocity, and so on. The control objective for the multi-agent systems in this paper is to find an appropriate control input sequence u i,k (t), i = 1, 2, · · · , n, t = 0, 1, 2, · · · , T such that every agent can move in the desired formation tracking trajectory accurately in the sense of expectation as the iterative number k tends to infinity. That is, lim k→∞ y i,k (t) = y d i (t). y d i (t) is the desired output of the ith agent and denoted as y d i (t) = r(t) + d i (t) (4) where r(t), t = 0, 1, 2, · · · , T is the desired reference trajectory of multi-agent systems. r(t) can also be called the desired trajectory of a virtual leader. In practice, not all agents can obtain r(t), only some of them can. Let Φ = diag ϕ 1 , ϕ 2 , · · · , ϕ n denote the reference-accessibility matrix, which is a diagonal, non-negative, real matrix. If the ith agent can obtain direct information about the r(t), then ϕ i > 0, i = 1, 2, · · · , n; otherwise, ϕ i = 0, i = 1, 2, · · · , n. In (4), d i (t) is the desired output deviation from desired reference trajectory of the ith agent. d ij (t) = d i (t) − d j (t) represents the desired relative formation between the ith agent and the jth agent. The formation tracking error δ i,k (t) is denoted as δ i,k (t) = d i (t) − y i,k (t) and the output error of the ith agent in kth iteration can be denoted as e i,k (t) = r(t) + δ i,k (t) (5) By applying the Z-transform to Equation (5), we get E i,k (z) = R(z) + ∆ i,k (z) (6) Appl. Sci. 2020, 10, 4752 4 of 15 If the following condition (7) is met lim k→∞ E i,k (z) = 0 (7) that is, lim k→∞ Y i,k (z) = Y d i (z), the multi-agent systems realize formation tracking control.

Structure of ILC for Networked Multi-Agent Systems
The following ILC law is used to achieve control objectives (7) where Γ i ∈ R is the learning gain for ith agent. The formulation of control input of the ith agent requires the control input of the last iteration, error, and information of adjacent agents. Due to the limited bandwidth of communication channel, noise interference, and signal fading, when a multi-agent transmits information through the wireless network, random packet loss is introduced into the multi-agent. For the communication graph of multi-agent systems shown in Figure 1, a block diagram of ILC for multi-agent systems with loss of multiple communication packets is illustrated in Figure 2. When the ith agent and the jth agent exchange information with each other through the wireless network, the ith agent receives a signal from the jth agent, which is denoted as where ξ k,ij obeys Bernoulli distribution and δ j,k (t) is the formation tracking error of the ith agent. When the jth agent sends a signal to the ith agent and there is no packet loss in the kth iteration, let ξ k,ij = 1. Otherwise, ξ k,ij = 0. Since the ith agent can obtain its own output y i,k (t) and the desired output deviation d i (t) without transmission network, there is no packet loss for the ith agent itself. That is, δ i,k (t + 1) does not have packet loss for the ith agent. It is worth noting that, unlike a previous study [18], the scenario discussed in this paper is closer to the engineering practice, assuming that the packet loss rates of communication between agents are different.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 4 of 15 The following ILC law is used to achieve control objectives (7) , ( ) = , ( ) + , , ( + 1) − , ( + 1) + , ( + 1) where ∈ ℝ is the learning gain for ith agent. The formulation of control input of the ith agent requires the control input of the last iteration, error, and information of adjacent agents. Due to the limited bandwidth of communication channel, noise interference, and signal fading, when a multiagent transmits information through the wireless network, random packet loss is introduced into the multi-agent. For the communication graph of multi-agent systems shown in Figure 1, a block diagram of ILC for multi-agent systems with loss of multiple communication packets is illustrated in Figure  2. When the th agent and the th agent exchange information with each other through the wireless network, the th agent receives a signal from the th agent, which is denoted as where ξ , obeys Bernoulli distribution and , ( ) is the formation tracking error of the th agent. When the th agent sends a signal to the th agent and there is no packet loss in the th iteration, let ξ , = 1. Otherwise, ξ , = 0. Since the th agent can obtain its own output , ( ) and the desired output deviation ( ) without transmission network, there is no packet loss for the th agent itself. That is, , ( + 1) does not have packet loss for the th agent. It is worth noting that, unlike a previous study [18], the scenario discussed in this paper is closer to the engineering practice, assuming that the packet loss rates of communication between agents are different.

Dropout Compensation ILC Method for Multi-Agent Systems with Loss of Multiple Communication Packets
In this section, firstly, because the iterative data are changing in the iterative domain, the previous iterative data are used to compensate for the missing data in the iterative process. Secondly, considering that ILC is a two-dimensional system, the convergence of any frequency in the iterative dimension is analyzed by variable iteration and parameter iteration. We make the following assumptions: Assumption 1. The dynamics G i for each agent is stable, that is, |A i | < 1, i = 1, 2, · · · , n. Without loss of generality, the system relative degree is one, that is, G i 0.

Remark 1.
If an agent's dynamics G i is not stable, we can use the local feedback of its output y i to make it stable. Assumption 2. The initial reset condition satisfies all agents and the desired input for each iteration, that is, x i,k (0) can be abbreviated to x i (0) and C i x i,k (0) = y d i (0), k ∈ 1, 2, 3 · · · , i = 1, 2, · · · , n. Remark 2. Assumption 2 is common in ILC of multi-agent systems (see [13,16,19]). If the initial reset condition is not met, we can regard it as the problem of robustness to the initial shifts. In our future work, the ILC with initial-state learning for formation tracking control of multi-agent systems will be explored.

Assumption 3.
The graph G of multi-agent system (1) is an undirected graph. In addition, its graph is connected, that is, there is a path between any two agents so that they can exchange information. Lemma 1. [20]: If an irreducible matrix M = (a i,j ) ∈ C n×n , n ≥ 1 is weakly generalized diagonally dominant and at least one of the rows is strictly diagonally dominant, M is nonsingular.
Remark 3. Assumption 3 implies that the (L + Φ) ∈ C n×n . In L + Φ, the magnitude of the diagonal entry is ϕ i + j N i a i,j , i = 1, 2, · · · , n and the sum of the magnitudes of all non-diagonal entries is j N i a i,j , i = 1, 2, · · · , n.
Since ϕ i + j N i a i,j ≥ j N i a i,j , i = 1, 2, · · · , n and at least one of the rows is satisfied matrix L + Φ is nonsingular by Lemma.

Dropout Compensate ILC Method
To achieve the control objective (7), we propose the ILC law to compensate the loss of packets.
When there are packet losses during multiple agents' communication, we utilize the previous iterative data to compensate this iterative data. In (10), δ j,k (t + 1) is data which the ith agent received from the jth agent where ξ k,ij is mentioned above. In practice, we can utilize a register to store each iterative data. If there is no packet loss, the next iterative data replace the old data. Otherwise, the old data are retained.

Remark 4.
In order to make the multi-agent system closer to the practical engineering applications, ξ k,ij could be different for different agents. That means the multi-agent system is heterogeneous in this paper. (6) can be rewritten as By combining the Z-transform of (10) with (12), we get where E j,k (z) = ξ k,ij E j,k (z) − (1 − ξ k,ij )E j,k−1 (z). According to Equations (3), (11)- (13) and Assumption 2, we have (14) can be rewritten as where L ξ = [l ij,ξ ] ∈ R n×n and A ξ = [a ij,ξ ] ∈ R n×n , Appl. Sci. 2020, 10, 4752 The relationship among the (k − 1)th, the kth, and the (k + 1)th system error is shown in (15). Our goal is to converge the system error when the number of iterations tends to infinity. Thus, (15) will be further analyzed in the iterative domain.

Analysis in Iterative Domain
ILC is a method to control systems operating in a repetitive mode over a finite interval with the requirement that the system accurately follows the desired output trajectory. The same actions are executed again and again. By using the previously executed information, the tracking accuracy is improved from repetition to repetition. It has long been recognized that ILC is a two-dimensional (2D) system in which information travels on two independent axes: the time axis and the iteration axis. Chow and Fang [21,22] proposed ILC 2D analysis methods for discrete-time and continue-time systems. After Z-transformation of system dynamics, ILC is considered as a 2D system in frequency domain and iteration domain in this paper. The following analysis is based on the decomposition of 2D dynamics into two separate 1D problems. E k (z) is considered as a 2D function E(k, z). Exchanging variable frequency and parameter iteration, the convergence of the system is considered as a 1D problem in the iteration domain. Thus, we exchange variable z and parameter k. E k (z) can be rewritten as E z (k), that is, E z (k) = [E 1,z (k), E 2,z (k), · · · , E n,z (k)] T , ξ k,ij can be rewritten as ξ ij (k). Then, (15) can be rewritten as where Take Z-transformation for (17) or (17) and ( ) = ( ) ∈ ℝ × , , can obtain characteristic equation of the system in iterative domain. In gence of the learning system is discussed based on the characteristic say that an iterative learning algorithm converges, it means that the the number of iterations approaches infinity. For the proposed dropout e will analyze the convergence of formation tracking for the multi-agent ion for (17) , we can obtain characteristic equation of the system in iterative domain. In nvergence of the learning system is discussed based on the characteristic s we say that an iterative learning algorithm converges, it means that the ro as the number of iterations approaches infinity. For the proposed dropout d, we will analyze the convergence of formation tracking for the multi-agent sformation for (17) tion (21), we can obtain characteristic equation of the system in iterative domain. In , the convergence of the learning system is discussed based on the characteristic Analysis w, when we say that an iterative learning algorithm converges, it means that the ds to zero as the number of iterations approaches infinity. For the proposed dropout C method, we will analyze the convergence of formation tracking for the multi-agent Take Z-transformation for (17) where ℒ( ) = ( ) ∈ ℝ × and ( ) = ( ) ∈ ℝ × , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent Take Z-transformation for (17) where ℒ( ) = ( ) ∈ ℝ × and ( ) = ( ) ∈ ℝ × , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent Take Z-transformation for (17) where ℒ( ) = ( ) ∈ ℝ × and ( ) = ( ) ∈ ℝ × , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent Take Z-transformation for (17) where ℒ( ) = ( ) ∈ ℝ × and ( ) = ( ) ∈ ℝ × , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent Take Z-transformation for (17) where ℒ( ) = ( ) ∈ ℝ × and ( ) = ( ) ∈ ℝ × , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent )] (19) where L( characteristic equation of the system in iterative domain. In he learning system is discussed based on the characteristic n iterative learning algorithm converges, it means that the and ( ) = ( ) ∈ ℝ × , , an obtain characteristic equation of the system in iterative domain. In ence of the learning system is discussed based on the characteristic ay that an iterative learning algorithm converges, it means that the )] ∈ R n×n and A( -transformation for (17) ds to quation (21), we can obtain characteristic equation of the system in iterative domain. In tion, the convergence of the learning system is discussed based on the characteristic nce Analysis know, when we say that an iterative learning algorithm converges, it means that the Take Z-transformation for (17) ere ℒ( ) = ( ) ∈ ℝ × and ( ) = ( ) ∈ ℝ × , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In next section, the convergence of the learning system is discussed based on the characteristic uation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the eads to Equation (21), we can obtain characteristic equation of the system in iterative domain. In ection, the convergence of the learning system is discussed based on the characteristic Take Z-transformation for (17) where where ℒ ( ) = ( ) ∈ ℝ × , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
-transformation for (17) ds to Take Z-transformation for (17) where where ℒ ( ) = ( ) ∈ ℝ × , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In Take Z-transformation for (17) where Take Z-transformation for (17) where where L ( , )] ∈ R n×n , l ij ( 9) leads to From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent systems in this section. Theorem 1. Consider the multi-agent system (1), let Assumptions 1-3 hold and the learning law (10) be applied. Given that any one of the following conditions is satisfied for all i = 1, 2, · · · , n, where then the formation tracking objective (7) is achieved.

Proof. Let S(
btain characteristic equation of the system in iterative domain. In of the learning system is discussed based on the characteristic that an iterative learning algorithm converges, it means that the umber of iterations approaches infinity. For the proposed dropout l analyze the convergence of formation tracking for the multi-agent 1), we can obtain characteristic equation of the system in iterative domain. In onvergence of the learning system is discussed based on the characteristic sis en we say that an iterative learning algorithm converges, it means that the zero as the number of iterations approaches infinity. For the proposed dropout hod, we will analyze the convergence of formation tracking for the multi-agent . der the multi-agent system (1), let Assumptions 1-3 hold and the learning law (10) be one of the following conditions is satisfied for all = 1,2, ⋯ , , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In next section, the convergence of the learning system is discussed based on the characteristic ation.

onvergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the tem error trends to zero as the number of iterations approaches infinity. For the proposed dropout pensation ILC method, we will analyze the convergence of formation tracking for the multi-agent tems in this section.  (21), we can obtain characteristic equation of the system in iterative dom the next section, the convergence of the learning system is discussed based on the charac equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means t system error trends to zero as the number of iterations approaches infinity. For the proposed d compensation ILC method, we will analyze the convergence of formation tracking for the mult systems in this section. Theorem 1. Consider the multi-agent system (1), let Assumptions 1-3 hold and the learning law applied. Given that any one of the following conditions is satisfied for all = 1,2, ⋯ , , in characteristic equation of the system in iterative domain. In the learning system is discussed based on the characteristic t an iterative learning algorithm converges, it means that the ber of iterations approaches infinity. For the proposed dropout alyze the convergence of formation tracking for the multi-agent 1, we need to prove that the modules of all roots of (25) are smaller than units.
From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
in characteristic equation of the system in iterative domain. In the learning system is discussed based on the characteristic ), we can obtain characteristic equation of the system in iterative domain. In onvergence of the learning system is discussed based on the characteristic we can obtain characteristic equation of the system in iterative domain. In vergence of the learning system is discussed based on the characteristic does not enclose the point (−1, j0) for β ∈ [−π, π), ω ∈ [−π, π), and ω 0. Based on Greshgorin's disk theorem, we have λ[F(ω, β)] ∈ U n i=1 F i for all β ∈ [−π, π), ω ∈ [−π, π), and ω 0, where Since ξ ij (e jω ) can only be 0 or 1, e jω ξ ij (e jω ) + 1 − ξ ij (e jω ) is 1 or e jω , respectively. Taking H i (e jβ ) as
btain characteristic equation of the system in iterative domain. In of the learning system is discussed based on the characteristic hat an iterative learning algorithm converges, it means that the umber of iterations approaches infinity. For the proposed dropout l analyze the convergence of formation tracking for the multi-agent agent system (1), let Assumptions 1-3 hold and the learning law (10) be owing conditions is satisfied for all = 1,2, ⋯ , , uation (21), we can obtain characteristic equation of the system in iterative domain. In on, the convergence of the learning system is discussed based on the characteristic ce Analysis now, when we say that an iterative learning algorithm converges, it means that the rends to zero as the number of iterations approaches infinity. For the proposed dropout ILC method, we will analyze the convergence of formation tracking for the multi-agent s section.

1.
Consider the multi-agent system (1), let Assumptions 1-3 hold and the learning law (10) be that any one of the following conditions is satisfied for all = 1,2, ⋯ , , (21), we can obtain characteristic equation of the system in iterative domain. In e convergence of the learning system is discussed based on the characteristic alysis when we say that an iterative learning algorithm converges, it means that the to zero as the number of iterations approaches infinity. For the proposed dropout ethod, we will analyze the convergence of formation tracking for the multi-agent on. nsider the multi-agent system (1), let Assumptions 1-3 hold and the learning law (10) be y one of the following conditions is satisfied for all = 1,2, ⋯ , , on (21), we can obtain characteristic equation of the system in iterative domain. In the convergence of the learning system is discussed based on the characteristic nalysis , when we say that an iterative learning algorithm converges, it means that the ds to zero as the number of iterations approaches infinity. For the proposed dropout method, we will analyze the convergence of formation tracking for the multi-agent ction.
onsider the multi-agent system (1), let Assumptions 1-3 hold and the learning law (10) be any one of the following conditions is satisfied for all = 1,2, ⋯ , , From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent systems in this section.
From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent systems in this section.
From Equation (21), we can obtain characteristic equation of the system in iterative domain. In the next section, the convergence of the learning system is discussed based on the characteristic equation.

Convergence Analysis
As we know, when we say that an iterative learning algorithm converges, it means that the system error trends to zero as the number of iterations approaches infinity. For the proposed dropout compensation ILC method, we will analyze the convergence of formation tracking for the multi-agent systems in this section.
When considering random communication packet dropout, the ILC algorithm can converge using the compensation method we proposed. When considering the random packet loss, the ILC algorithm can use this proposed compensation method to converge. Theorem 1 gives the convergence condition of learning gain design and proves that the convergence condition is independent of communication packet loss. According to the final value theorem, the error of formation tracking converges to zero as the number of iterations goes to infinity.

Simulation
Firstly, consider the multi-agent system in Section 2.2.1. and let Assumptions 1-3 hold. Next, the learning law (10) is applied and the learning gains satisfy the condition (23). The results of numerical simulations are discussed in this section.
In order to illustrate the effectiveness of the proposed dropout compensation ILC method, we considered a heterogeneous multi-agent system consisting of four agents in the undirected graph as shown in Figure 3. It can be seen that the graph is a connected graph and satisfies Assumption 3. Only the second agent and the third agent can obtain the desired reference trajectory.

Simulation
Firstly, consider the multi-agent system in Section 2.2.1. and let Assumptions 1-3 hold. Next, the learning law (10) is applied and the learning gains satisfy the condition (23). The results of numerical simulations are discussed in this section.
In order to illustrate the effectiveness of the proposed dropout compensation ILC method, we considered a heterogeneous multi-agent system consisting of four agents in the undirected graph as shown in Figure 3. It can be seen that the graph is a connected graph and satisfies Assumption 3. Only the second agent and the third agent can obtain the desired reference trajectory. Then it was obtained that the information interaction weight between agents can be obtained, as shown in Table 1. For every agent in Figure 3, the system parameters were defined in Table 2.  The Laplacian matrix L of graph G and the reference-accessibility matrix Φ were Then it was obtained that the information interaction weight between agents can be obtained, as shown in Table 1. Table 1. Interaction weight of agent.
For every agent in Figure 3, the system parameters were defined in Table 2. Table 2. System parameters.
Let Γ 1 = −0.813, Γ 2 = 1.31, Γ 3 = 0.57, and Γ 4 = 0.92. Then, we can obtain the parameters in Table 3 for Theorem 1. Therefore, the learning gains satisfied the second condition in Theorem 1. To perform our simulation tests, we considered the desired reference trajectory as r(t) = 1.5 + sin πt 25 (33) and the desired output deviation of each agent as Set the initial state of four agents as: x 1 (0) = −4.277, x 2 (0) = 0, x 3 (0) = 6.1333, and x 4 (0) = 2.806, which satisfies Assumption 2. When the iteration step is 500 and t = 1, 2, · · · , 150, the trajectory of the multi-agent system without packet loss is shown as Figure 4. It can be seen that the desired formation is well achieved to track the desired reference trajectory by the desired output deviation. Therefore, the learning gains satisfied the second condition in Theorem 1. To perform our simulation tests, we considered the desired reference trajectory as Set the initial state of four agents as: (0) = −4.277, (0) = 0, (0) = 6.1333, and (0) = 2.806, which satisfies Assumption 2. When the iteration step is 500 and = 1,2, ⋯ ,150, the trajectory of the multi-agent system without packet loss is shown as Figure 4. It can be seen that the desired formation is well achieved to track the desired reference trajectory by the desired output deviation. In order to measure the formation accuracy quantitatively, the disagreement among all agents on their output errors was defined as: ( ) = ∑ ∑ , ( ) . When lim → ( ) = 0, multi-agent system achieved formation tacking. Figure 5 shows that formation performance of the system at the first 500 iterations. Obviously, when the iteration is around 350 the system error can converge to zero and achieve formation tracking. In order to measure the formation accuracy quantitatively, the disagreement among all agents on their output errors was defined as: error(k) = 1 nT n i=1 T t=1 e i,k (t) . When lim k→∞ error(k) = 0, multi-agent system achieved formation tacking. Figure 5 shows that formation performance of the system at the first 500 iterations. Obviously, when the iteration is around 350 the system error can converge to zero and achieve formation tracking.
Appl. Sci. 2020, 10, x FOR PEER REVIEW 11 of 15 here ξ , = 1 when there is no packet loss, otherwise, ξ , = 0. In Figure 6, it can be seen that the system error cannot converge to zero in different packet loss rates as iteration increases. When It is assumed that there is no compensation when considering communication packet loss, that is where ξ k,ij = 1 when there is no packet loss, otherwise, ξ k,ij = 0. In Figure 6, it can be seen that the system error cannot converge to zero in different packet loss rates as iteration increases. When iteration is 500 and packet dropout rate is 10%, the trajectory of the multi-agent system without packet loss compensation is shown as Figure 7. here ξ , = 1 when there is no packet loss, otherwise, ξ , = 0. In Figure 6, it can be seen that the system error cannot converge to zero in different packet loss rates as iteration increases. When iteration is 500 and packet dropout rate is 10%, the trajectory of the multi-agent system without packet loss compensation is shown as Figure 7.
Status value of agents When there are different packet loss rates in a multi-agent system (Figure 8), it can be seen that system errors can converge to zero using (10) to compensate packet loss. The convergence speed hardly gets slower as the packet loss rates increase, as shown in Figure 8b. In addition, the multi-agent system error still can converge to zero around the 350th iteration. When iteration is 350 and packet loss rate is 10%, the trajectory of the multi-agent system with packet loss compensation is shown as in Figure 9.
(c) (d) Figure 6. System error of multi-agent system with different packet loss rates: (a) 1% packet dropout, (b) 2% packet dropout, (c) 5% packet dropout, (d) 10% packet dropout. When there are different packet loss rates in a multi-agent system (Figure 8), it can be seen that system errors can converge to zero using (10) to compensate packet loss. The convergence speed hardly gets slower as the packet loss rates increase, as shown in Figure 8b. In addition, the multiagent system error still can converge to zero around the 350th iteration. When iteration is 350 and packet loss rate is 10%, the trajectory of the multi-agent system with packet loss compensation is shown as in Figure 9.  In order to demonstrate the effectiveness of proposed method, the comparison results with the method of Zhang and Li [19] are shown in Figure 10. If there is data loss, the information for the current iteration will be discarded without any compensation. Therefore, although the error curve   In order to demonstrate the effectiveness of proposed method, the comparison results with the method of Zhang and Li [19] are shown in Figure 10. If there is data loss, the information for the current iteration will be discarded without any compensation. Therefore, although the error curve can converge to 0, the convergence speed gets slower as the packet loss rate increases [19], while the convergence speed hardly gets slower, as shown in Figure 8b, by the proposed method.  In order to demonstrate the effectiveness of proposed method, the comparison results with the method of Zhang and Li [19] are shown in Figure 10. If there is data loss, the information for the current iteration will be discarded without any compensation. Therefore, although the error curve can converge to 0, the convergence speed gets slower as the packet loss rate increases [19], while the convergence speed hardly gets slower, as shown in Figure 8b, by the proposed method.

Conclusions
ILC-based formation tracking for heterogeneous multi-agent systems is a challenging problem since different communication packets are lost between agents. When packet loss occurs at different rates, the tracking error of ILC does not converge. This paper explores the ILC method of packet loss compensation for a heterogeneous multi-agent system. Since the iterative data are constantly changing in the iterative domain, it is effective to use previous data to compensate for the lost data in this iterative process. Based on the ILC model for multi-agent systems in frequency domain, the convergence of the system is considered as a 1D problem in the iterative domain, and the convergence of the system is analyzed. By using the generalized Nyquist criterion and Gershgorin's disk theorem, the eigenvalue range of the system characteristic equation is analyzed, and the convergence condition of the system error is obtained. The dropout compensation ILC method proposed in this paper can not only make the error converge to zero with the increase of iteration times, but also the maintain the convergence speed under the packet loss scenario. The numerical simulation results demonstrate the effectiveness of the proposed method.

Conclusions
ILC-based formation tracking for heterogeneous multi-agent systems is a challenging problem since different communication packets are lost between agents. When packet loss occurs at different rates, the tracking error of ILC does not converge. This paper explores the ILC method of packet loss compensation for a heterogeneous multi-agent system. Since the iterative data are constantly changing in the iterative domain, it is effective to use previous data to compensate for the lost data in this iterative process. Based on the ILC model for multi-agent systems in frequency domain, the convergence of the system is considered as a 1D problem in the iterative domain, and the convergence of the system is analyzed. By using the generalized Nyquist criterion and Gershgorin's disk theorem, the eigenvalue range of the system characteristic equation is analyzed, and the convergence condition of the system error is obtained. The dropout compensation ILC method proposed in this paper can not only make the error converge to zero with the increase of iteration times, but also the maintain the convergence speed under the packet loss scenario. The numerical simulation results demonstrate the effectiveness of the proposed method.