Model-Free Adaptive Iterative Learning Bipartite Containment Control for Multi-Agent Systems

This paper studies the bipartite containment tracking problem for a class of nonlinear multi-agent systems (MASs), where the interactions among agents can be both cooperative or antagonistic. Firstly, by the dynamic linearization method, we propose a novel model-free adaptive iterative learning control (MFAILC) to solve the bipartite containment problem of MASs. The designed controller only relies on the input and output data of the agent without requiring the model information of MASs. Secondly, we give the convergence condition that the containment error asymptotically converges to zero. The result shows that the output states of all followers will converge to the convex hull formed by the output states of leaders and the symmetric output states of leaders. Finally, the simulation verifies the effectiveness of the proposed method.


Introduction
Cooperative control problems of MASs have been widely studied [1][2][3]. As a fundamental topic of cooperative control of MASs, the containment control problem has been investigated in recent years. The containment control problem considers that there are multiple leaders in the network and all followers can converge to the convex hull formed by the leaders. Containment control algorithms for MASs with different dynamics have been proposed [4][5][6][7]. For example, for both continuous-time and discrete-time MASs, Liu et al. [4] presented the necessary and sufficient conditions which guarantee the achievement of containment control. Considering the heterogeneous agents, Zheng et al. [5] further studied the containment problem of heterogeneous MASs composed of first-order and second-order integrator agents. Moreover, the containment control problem for MASs with general linear dynamics has also been studied [6,7].
In the practical application, many industrial processes achieve the tasks in a repetitive environment, such as robot manipulators and injection molding process [8]. For systems that repeat the operation process over a finite time interval, the iterative learning control (ILC) was studied [9][10][11][12][13]. The control method uses the error between the current trajectory and the desired trajectory to improve the control performance. Then, it can achieve the desired trajectory tracking through learning from the repetitive tracking task. The ILC has been used for solving various cooperative problems of MASs [14][15][16][17][18][19]. Yang et al. [14] solved the consensus tracking problem of MASs with time-varying dynamics by the ILC method. Meng et al. [15] further studied the consensus tracking problem of MASs with initial state shifts and disturbances by the ILC method. In addition to the consensus tracking problem, the ILC has also been used to solve the formation control problem of MASs [17][18][19]. For example, Li et al. [18] proposed the distributed ILC protocols for solving the consensus and formation problem of second-order MASs.
Most of the above mentioned papers mainly used model-based control methods to design the controller. However, modeling a practical plant is an approximation of the real plant. As the complexity of system increases, it becomes more difficult to establish the

Preliminaries
denotes the node set, the edge set, the adjacency matrix, respectively. a ij = 0 if (j, i) ∈ E . (j, i) denotes the edge from j to i. a ij > 0 denotes the cooperative interaction between i and j, a ij < 0 denotes the antagonistic interaction, and a ii = 0 for all i ∈ V. The neighbor set of node i is depicted by N i = {j|(j, i) ∈ E }. Let V 1 be a subset of V and N V 1 = {j ∈ V \ V 1 |∃i ∈ V 1 : (j, i) ∈ E }, that is, N V 1 represents the set of neighbors of the agent in V 1 such that these neighbors belong to the complement set of V 1 .
If node i is in a strongly connected subgraph G s of G(V, E , A) and N G s = ∅, node i can be called a leader, and i is called a follower otherwise. The Laplacian matrix L = [l ij ] = D − A, 2}), and a ij ≤ 0, ∀i ∈ V p , j ∈ V q , p = q(p, q ∈ {1, 2}). It is said structurally unbalanced otherwise. Next, we will follow the flowchart in Figure 1 to solve the bipartite containment problem of MASs.

Problem Statement
We will discuss the bipartite containment control of the nonlinear discrete-time MASs by the MFAILC method. We consider m leaders and n followers in a signed network, where the m leaders are divided into groups V 1 and V 2 , respectively. Assume that V 1 has m 1 leaders and V 2 has m 2 leaders, where m = m 1 + m 2 . The graph consisting of m 1 leaders is strongly connected and structurally balanced, that is, it admits a bipartition of the nodes V 1 , V 1 , V 1 = V 1 V 1 , and V 1 ∩ V 1 = ∅. The graph consisting of m 2 leaders is also strongly connected and structurally balanced, that is, it also admits a bipartition of the nodes V 2 , V 2 , V 2 = V 2 V 2 , and V 2 ∩ V 2 = ∅. The interactions among leaders include the antagonistic interaction. Moreover, the n followers are included in group V 3 . The interactions among leaders and followers are cooperative, and the interactions among followers are also cooperative.
The dynamic of the i-th agent is written as where i ∈ {1, 2, . . . , m + n} represents the i-th agent, N ∈ {0, 1, . . . , T} represents time, and h represents the iteration number. u i (N ,h) ∈ R and y i (N ,h) ∈ R, respectively, represents the I/O of the i-th agent. n y and n u , respectively, represents the unknown orders of y and u. f i (·) represents the nonlinear function, and it only consists of the I/O of the i-th agent.

Assumption 2.
The partial derivative of f i (·) with respect to u i (N ,h) is continuous, and f i (·) satisfies the Lipschitz condition |∆y , and b is a positive constant.

Lemma 1 ([34]).
If system (1) satisfies Assumption 2 and ∆u i (N ,h) = 0, then we have where φ i (N ,h) is a time-varying parameter associated with iteration, which is called pseudo partial derivative. φ i (N ,h) ≤ b and b is a positive constant for any N andh.
Equation (2) can transform the nonlinear system of each agent into a linear dynamic system with a time-varying parameter and does not require the model information of MASs. The time-varying parameter φ i (N ,h) can also be estimated only by the input and output data of MASs. Assumption 3. For any time instant N andh, the pseudo partial derivative satisfies where σ is an arbitrarily positive constant. Without loss of generality, we assume φ i (N ,h) > σ > 0.
Assumption 2 sets a bound on the change rate of the output caused by the change of input, which is a common situation in some real systems. Assumption 3 ensures that the increment of output and the increment of input have the same sign, which means that when the control input increases, the output must increase.

Main Result
In this part, we will show the controller and the condition in which the bipartite containment control is achieved.
We assume that agent i, i = 1, 2, . . . , m, is leader, and agent i, i = m + 1, m + 2, . . . , m + n, is follower. The subgraph composed of m leaders is G L and the subgraph composed of n followers is G F . Then, the adjacency matrix of G(V, E , A) is written as where A L denotes the adjacency matrix of G L , A F denotes the adjacency matrix of G F , and A FL denotes the interactions among leaders and followers. The Laplacian matrix of where L L and L F are Laplacian matrices associated with A L and A F , respectively. L F =

Definition 1. The containment error of the i-th agent at theh-th iteration is denoted by
where y 0 is a fixed constant, y 0 = y 0 for i ∈ V 1 , and y 0 = y 0 for i ∈ V 2 . c i ∈ {0, 1}, and c i = 1 denotes that agent i can receive the information y 0 , otherwise c i = 0. We consider that there is at least one c i = 1 for i ∈ V 1 , and there is at least one Containment errors of all agents in the form of vectors are written as In order to facilitate the proof, we let ξ( By Equation (4), we can get Our goal is to design a controller for MASs, so that the bipartite containment can be achieved, that is, the output states y i (N ,h) of followers converge to the convex hull formed by the output states y i (N ,h) of leaders and the reverse output states −y i (N ,h) of leaders. If we can prove that Since the graph consisting of n followers is strongly connected and D FL is a nonzero matrix, by the Gersgorin disc theorem and Theorem 6.2.26 in [35], we know that zero is not an eigenvalue of Then, the output states of followers converge to the convex hull formed by the output states and the reverse output states of leaders. Thus, we just need to prove that the leaders in V 1 and the leaders in V 2 achieve a bipartite consensus, and lim¯h →∞ ξ F (N ,h) = 0.
The controller for every agent is designed as where λ > 0 represents the weighting factor that will effect the stability of MASs, and ρ ∈ (0, 1) represents the parameter of controller (10) that will affect the convergence properties.
where 0 < η < 1 and µ > 0. The following equation is the reset condition which can ensure the robustness of controller:φ which holds if any of the following three equations is satisfied |φ By the optimization condition (∂J(φ i (N ,h)))/(∂φ i (N ,h)) = 0, we can get (11). Similarly, we design the objective function J(u i (N ,h) By the optimization condition (∂J(u i (N ,h)))/(∂u i (N ,h)) = 0, we can get (10).
represent the parameter estimation error. By Equations (2) and (11), By 0 < η < 1 and µ > 0, we have that there is a constant q satisfying By Equations (14) and (15), Equation (13) can be written asφ In order to facilitate the following proof, we consider ∆u i (N ,h). By Equation (10), we have Then, we have where From Assumption 3, we know that φ i (N ,h) is non-negative. By Equation (12), we can obtain that the sign ofφ i (N ,h) is the same as the sign ofφ i (N , 1). As the estimated value of φ i (N ,h), the nonnegativity ofφ i (N ,h) can be guaranteed by choosing the initial valuê φ i (N , 1) ofφ i (N ,h).

|aij|+ci)
, and G(V 1 , E 1 , A 1 ) is strongly connected, then matrix I − ρH 1 (N ,h)L M is nonnegative and irreducible. Since there is at least one c i = 1, then at least one row sum of I − ρH 1 (N ,h)L M is strictly less than one. Thus, I − ρH 1 (N ,h)L M is an irreducible substochastic matrix. By Equation (25), we have (26), then by Lemma 1 in [37], we have where h−1 ω denotes the integer which is smaller than¯h −1 ω and closest to¯h −1 ω , and 0 < δ < 1. By 0 < δ < 1, we have lim¯h →∞ Z(N + 1,h) = 0. By e(N + 1,h) = MZ(N + 1,h), we have lim¯h →∞ e(N + 1,h) = 0, which means that leaders in group V 1 achieve bipartite consensus. That is, lim¯h →∞ y i (N ,h) = y 0 for i ∈ V 1 and lim¯h →∞ y i (N ,h) = −y 0 for i ∈ V 1 . Similarly, leaders in group V 2 can also achieve bipartite consensus.

Simulation
We consider a MAS which consists of 6 leaders and 3 followers. The signed graph is shown in Figure 2 1-6 are leaders, 7-9 are followers. Moreover, the subgraph consisting of leaders 1, 2, 3 and the subgraph consisting of leaders 4, 5, 6 are structurally balanced. By the definition of adjacency matrix, we have the adjacency matrix of the signed graph shown in Figure 2 as follows By the definition of Laplacian matrix, we have the Laplacian matrix of signed graph shown in Figure 2 as follows y 6 (N + 1,h + 1) = y 6 (N ,h + 1) + u 6 (N ,h + 1) 1 + y 6 (N ,h + 1) + 6 · u 6 (N ,h + 1), We choose ρ = 0.1, λ = 10, η = 0.5, µ = 1, σ = 10 −5 . It is worth noting that σ is the parameter used to set the reset condition of parameterφ i (N ,h). The initial state of each agent can be selected arbitrarily. Figure 3 shows that the bipartite containment task has been achieved. We can find that the output states of agents 1, 3 converge to the same value 1 2 , and output state of agent 2 reaches a value with the opposite sign of agents 1, 3. Similarly, the output states of agents 4, 6 converge to the same value 1, and output state of agent 5 reaches a value with the opposite sign of agents 4, 6. The output states of agents 7-9 asymptotically converge to the convex hull formed by agents 1-6. Figure 4 shows that the containment error of agents 7-9 asymptotically converges to zero as the number of iterations increases.

Discussion
The MFAILC is a control design method for nonlinear systems. Its basic idea is to establish an equivalent dynamic linear data model of each multi-agent system near each working point, and use the I/O data of the controlled system to estimate the partial derivatives of the system online. Then, the weighted one-step forward controller is designed by using the relationship between the MASs, and the MFAILC of nonlinear system is realized. Compared with the traditional adaptive ILC method, the model and algorithm proposed in this paper have some remarkable characteristics, as follows. First, the controller design only needs the I/O measurement data of the controlled system, without any model information. Therefore, traditional unmodeled dynamic problems do not exist under the MFAILC framework. Second, the MFAILC method has a simple structure and a small amount of computation. It does not require the construction of an accurate mathematical model of a multi-agent system, and any test signal and training process, thus it is a lowcost controller.
In the process of solving the bipartite containment problem of MASs, the existence of Assumption 1 has certain restrictions on the application of the MFAILC method. However, due to the limitation of communication bandwidth and storage space of MASs, the agent only transmits partial information. Therefore, Assumption 1 holds for some MASs. Moreover, the effect of unknown disturbances and time-varying network on the bipartite containment problem is not considered. We will solve this problem in the future. The simulations presented in this paper demonstrate the effectiveness of our proposed MFAILC method. In Figure 4, due to the selection of the initial value, the containment error of the followers is relatively large in the first few iterations. However, with the increase of the number of iterations, the containment error asymptotically converges to zero, that is, the bipartite containment control of the MASs is achieved by using the MFAILC method.

Conclusions
In this paper, the bipartite containment tracking problem for nonlinear MASs has been studied, where cooperative and antagonistic interactions between agents are considered. To solve this problem, we first show that, if the containment error converges to zero, then the bipartite containment can be achieved. Then, a MFAILC based on the dynamic linearization method is proposed. The designed controller only depends on the input and output data of MASs and does not need the model information of MASs. Furthermore, the condition that the containment error converges to zero is given, that is, all output states of followers asymptotically converge to the convex hull formed by the output states of leaders as well as leaders' symmetric output states. The simulation verifies the effectiveness of the proposed method. Future studies will focus on the bipartite containment problem of MASs with unknown disturbances and time-varying network.

Conflicts of Interest:
The authors declare no conflict of interest.