Multi-Agent Cooperative Control Consensus: A Comparative Review

Abstract

Cooperative control consensus is one of the most actively studied topics within the realm of multi-agent systems. It generally aims to drive multi-agent systems to achieve a common group objective. The core aim of this paper is to promote research in cooperative control community by presenting the latest trends in this field. A summary of theoretical results regarding consensus for agreement analysis for complex dynamic systems and time-invariant information exchange topologies is briefly described in a unified way. The application under both non-formation and formation cooperative control consensus for multi-agent system also investigated. In addition, future recommendations and some open problems are also proposed.

1. Introduction

In comparison to an autonomous single agent/mobile robot, which only executes solo missions, better operational capability and efficiency can be achieved from multi-agent systems that are operating in a coordinated fashion. Owing to the ability to handle abundant computational resources that are embedded in an autonomous agent enables it to enhance operational effective capabilities through a cooperative teamwork of multi-agent in military and civilian applications [1]. So by using multi-agent systems, certain global objectives can be achieved through sensing, exchange of information using communication, computation and their control [2,3].

A significant amount of research effort has been put into the cooperative control of multi-agent systems in the last decade. In the cooperative control consensus, agents share their information with each other. This information may lead to achieve common group objectives, relative position information or common control algorithms.

The behavior-based approach has been used for researchers to examine the social characteristics of animals and insects to apply coordination control findings to the design of multi-agent systems. In Reference [4] Reynolds presented three basic rules of cohesion, separation and alignment. Where cohesion means staying close to all nearby neighbors, separation means avoiding collision and alignment means to match velocities with the remaining agents. By introducing the aggregate motions of a multi-agent system, the author generated the first animation on the computer. Also, for dynamic topologies, Viscek’s model is very important, which can be classified as a special type of a distributed behavioral model with Reynolds’ rules [5].

Cooperative control for multiple agents can be classified as non-formation cooperative control problems such as role assignment, automated parallel delivery of payload transport, foraging, task handling, air traffic control, cooperative search and timing, or as formation control problems such as mobile agents involved in surveillance and reconnaissance operations, flying or unmanned aerial vehicles (UAVs), self-assembly of connected mobile networks, autonomous underwater vehicles, spacecraft, aircraft, satellites and automated highway systems. To enable these applications in multi-agent systems, diverse cooperative control techniques need to be built up, including rendezvous [6,7] flocking [8,9] and swarming [10,11].

The main issues for the multi-agent consensus are coverage problems [12], network consensus [13], multi-agent navigation [14] and formation control [15]. The main approaches found in recent literature to solve these problems include algebraic graph theory-based approaches [16], geometric constraint techniques [17] and the artificial potential field method [18]. In all these approaches, the agents are assumed to remain entirely connected with each other for communication links and particularly for formation control they should further form a predefined shape as well.

The ultimate focus of this paper is to present a review of consensus problems in multi-agent coordination systems with the aim of elevating more and more research in this field by emphasizing highly cited papers. Finally, applications of multi-agent systems including consensus, flocking and swarming are also presented.

The paper outline is as follows: A brief background on algebraic graphs and matrix theory is given in Section 2. Theoretical progress in consensus that includes convergence analysis for the time-invariant and dynamic state, consensus speed and heterogeneous agents is presented in Section 3. Section 4 demonstrates the convergence constraint due to practical limitations. The application of multi-agent systems can be seen in Section 5. Finally, conclusions and some open questions are discussed in Section 6.

2. Preliminaries

In order to exchange information among multi-agent systems, it is natural to model this by way of an undirected or directed graph, where vertices represent agents and edges are the information exchange links among agents. A pair $\left(V,E\right)$ is called a directed graph where $V=\left\{1,\dots ,n\right\}$ is a nonempty finite node set and $E\subseteq V\times V$ is called an edge set. The neighbor of $i^{th}$ agent is denoted by $N_i=\left\{j\in V:\left(i,j\right)\in E\right\}$. For the edge $\left(i,j\right)$, $i$ is known as parent node whereas j is mentioned as the child node. The edge $\left(i,j\right)\in E$ means that agent j can get updates from agent $i$ but for agent $i$ it is not permissible. Contrary to a directed graph, the undirected graph can be seen as a special type of directed graph where un-ordered pairs of nodes are allowed. The edge $\left(i,j\right)\in E$ is referred to as agent $i$ and $j$ and can receive updates from each other so edges $\left(i,j\right)$ and $\left(j,i\right)$ in the directed graph equate to an edge $\left(i,j\right)$ in the undirected graph [19].

In a directed graph, if there is a directed path from every node to every other node it is known as “strongly connected”. Similarly, in an undirected graph, if there is an undirected path between every pair of distinct nodes then it is called “connected graph”. A node is called a root if it has a directed path to the remaining nodes without having a parent itself. A rooted directed tree is a directed graph where except for one node remaining nodes should have exactly one parent node [20].

For a directed graph with a node set $V=\left\{1,\dots ,n\right\}$ the adjacency matrix $A=\left[a_{ij}\right]\in R^{n\times n}$ is defined as a “positive weight” where $a_{ij}=1$ if $\left(j,i\right)\in E$ and $a_{ij}=0$ for $\left(j,i\right)\notin E$. As all the graphs have some weights so if weights are not significant in a particular situation, then $a_{ij}$ is assumed to be equal to one for all $\left(j,i\right)\in E$. Self-edges with positive weight are also allowed. For some $a_{ij}$ graph is referred to as balanced if ${\sum }_{i=1}^n{a}_{ij}={\sum }_{j=1}^n{a}_{ji}$ for all i. As adjacency matrix is symmetric for undirected graph thus every undirected graph will be automatically balanced. Thus, the adjacency matrix can be written as

$$A=[a_{ij}]=\{\begin{array}{cc}1& if\Vert q_j-q_i\Vert <r\\ 0& otherwise\end{array}$$

The directed graph Laplacian matrix $L=\left[l_{ij}\right]\in R^{n\times n}$ can be defined as $l_{ii}{\sum }_{j\ne i}{a}_{ij}$ and $l_{ij}=-a_{ij}$ for all $i\ne j$. Similarly, if $\left(j,i\right)\notin E$, then $l_{ij}=-a_{ij}=0$. The graph Laplacian elements are defined as

$$L=[l_{ij}]=\{\begin{array}{cc}-a_{ij},& j\in N_i\\ \left|N_i\right|,& j=i\end{array}$$

Here, $\left|N_i\right|$ is the number of neighbors of node $i$ or simply its in-degree, where the degree matrix is defined as $D=diag\left({deg}_{in}\left(1\right),\dots ,{deg}_{in}\left(N\right)\right)$. For a directed graph, L is not necessarily symmetric however for an undirected graph, L is always symmetric. For a balanced graph, the degree-in and degree-out should be same.

$$de{g}_{in}\left(i\right)={\displaystyle \sum }_{j=1}^N{a}_{ij}={\displaystyle \sum }_{j=1}^N{a}_{ji}=de{g}_{out}\left(i\right)$$

Similarly, the Laplacian matrix in terms of degree matrix can be defined as “$L=D-A$”. For an undirected graph, if $i^{\mathrm{th}}$ smallest eigenvalue of Laplacian matrix is ${\lambda }_i\left(L\right)$ with ${\lambda }_1\left(L\right)\le {\lambda }_2\left(L\right)\le \dots \le {\lambda }_n\left(L\right)$ such that first eigenvalue is zero then convergence rate of consensus algorithms is quantified by ${\lambda }_2\left(L\right)$ which is known as algebraic connectivity [21]. If the undirected graph is connected then ${\lambda }_2\left(L\right)$ is always greater than zero [22]. Also, zero is the simple eigenvalue of Laplacian matrix, if the directed graph is strongly connected or the undirected graph is connected, while the reverse does not hold.

The eigenvalue for Laplacian matrix having undirected topology are ${\lambda }_1\left(L\right)=0$ and ${\lambda }_i\left(L\right)>0$, for a graph having spanning tree are ${\lambda }_1\left(L\right)=0$ and $\mathrm{Re}({\lambda }_i\left(L\right))>0$ and for a complete graph are ${\lambda }_1\left(L\right)=0$ and ${\lambda }_i\left(L\right)=N-1$ where $i=2,3,\dots ,N$. For more details on graph theory, the reader should refer to [19].

3. Theoretical Progress in Consensus

A consensus protocol is a communication rule that specifies the exchange of information within a network, between an agent and all its nearby neighbors. If the information exchange among agents allows continuous communication or the bandwidth of communication is significantly large, then differential equation is used to model the information state. Similarly, if the information exchange data arrive in discrete packets, then difference equation is used to model information state of each agent [1]. The three major areas that have been covered under consensus network are consensus algorithm, network topology and convergence rate. So, in this section, there is a review of the recent theoretical aspects of multi-agent consensus problems.

3.1. Convergence Analysis for Time-Invariant Topology

Time-invariant communication topology means that if at any time instant an agent can access information of another agent, then it is assumed that it can get updates from those agents all the time. The comparison between continuous and discrete consensus is presented in

Table 1. Continuous vs. Discrete: Consensus Comparison.

	Continuous	Discrete
Dynamics	${\dot{x}}_i\left(t\right)={\displaystyle {\displaystyle \sum }_{j=1}^n}{a}_{ij}\left(t\right)\left(x_j\left(t\right)-x_i\left(t\right)\right)$	$x_i\left[k+1\right]={\displaystyle {\displaystyle \sum }_{j=1}^n}{b}_{ij}\left[k\right]x_j\left[k\right]$
	$\dot{x}\left(t\right)=-Lx\left(t\right)$	$x\left[k+1\right]=D\left[k\right]x\left[k\right]$
Key Matrix	${\log }_{t\to \infty }{e}^{-Lt}\to \mathbf{1}{\mathit{\nu }}^T$	${\log }_{k\to \infty }{D}^{-k}\to \mathbf{1}{\mu }^T$
	${\lambda }_1\left(L\right)=0$	${\lambda }_1\left(D\right)=1$
Eigen Value	$Re({\lambda }_i\left(L\right))=0,i=2,3,\dots ,N$	${\lambda }_i\left(L\right)<\left|1\right|,i=2,3,\dots ,N$

. From here, it can be seen that the final equilibrium state for continuous and discrete case is the weighted average of the initial condition of each agent. Still, it is not obvious that whether every agent will have some effect on the ultimate equilibrium state or not. As $v_j$ and ${\mu }_j$ are positive for strongly connected topology, thus in this situation each agent will have some effect on the final equilibrium state [23]. Besides this, for continuous and discrete protocol if $v_i=v_j=1/n$ and ${\mu }_i={\mu }_j=1/n$ where $i\ne j$ then the average consensus can be achieved which is the average of the initial condition of each agent [20].

For a directed topology, consensus can be achieved when the topology is strongly connected because all the agents can pass information to each other as depicted in Figure 1 but to achieve average consensus topology should be strongly connected and balanced as can be seen in Figure 2. For an undirected graph, the agreement can be achieved only if the graph is connected [20]. It also claims in [19] that if a graph is balanced and weakly connected then automatically it is strongly connected hence it contains rooted out branching thus average consensus can be achieved.

Moreover, if the number of edges increases then algebraic connection will also increase causing settling time to reduce but overall cost will be increased, as shown in Figure 3.

3.2. Convergence Analysis for Complex Dynamic Systems

The network can be classified as static, dynamic and random network. Where static depends on linear-time invariant systems, hybrid systems are dynamic, which leads to the Lyapunov function and at last, random networks rely on both Lyapunov and stochastic stability [19]. Here, the complex dynamic system is classified as switching network and synchronization network.

3.2.1. Switching Network

Recently, much research efforts for switching information exchange topologies of coordination of multi-agent system has been performed. In communication, there are constraints in terms of bandwidth and energy while in sensing the limitations are in term of range and resolution [19]. There is also the possibility that the neighbor of an autonomous agent will not remain its neighbor for all the time, it may change according to the situation. For example, if the communication is being done using the direct sensor, then there is a possibility that the visible neighbors may change with respect to time. Similarly, the information exchange links may be changeable due to communication range limitations or disturbances.

To achieve consensus for a time-invariant, communication topology is notably simple. But, this is the not case with the communication topology to be dynamic. In switching for the single integral kinematics, the final value is constant while for the double integral kinematics the final value is dynamic. Here, the dynamic topology can be achieved using algebraic graph. Transition matrix $\mathrm{\Phi }\left(t,0\right)$ corresponding to $-L\left(t\right)$ in continuous-time then the solution of the continuous-time and discrete-time agreement protocols can be modified as $x\left(t\right)=\mathrm{\Phi }\left(t,0\right)x\left(0\right)$ and $x\left[k\right]=D\left[k\right]\dots D\left[1\right]D\left[0\right]x\left[0\right]$ respectively.

Therefore, consensus can be guaranteed if $li{m}_{t\to \infty }\mathrm{\Phi }\left(t,0\right)\to \mathbf{1}{v}^T$ and $li{m}_{k\to \infty }D\left[k\right]\dots D\left[1\right]D\left[0\right]\to \mathbf{1}{\mu }^T$, where $v={\left[v_1,\dots ,v_n\right]}^T$ and $\mu ={\left[{\mu }_1,\dots ,{\mu }_n\right]}^T$ correspond for continuous and discrete-time respectively. There is another possibility when $L\left(t\right)$ is constant with some dwell time ${\tau }_j=t_{j+1}-t_j$, where “dwell time” is defined as the time when there is no change in information exchange topology. The connectivity of 15-nodes realization of a random geometric network with different radius is illustrated in Figure 4, where it can be seen that for increasing radius (r) the average node degree is also increasing. For $r=0.48$ the average node degree is $5.87$, whereas for $r=0.60$ and $r=0.80$ the average node degree is $9.60$ and $12.80$ respectively.

3.2.2. Synchronization Network

Synchronization in the complex network has also been extensively studied in [24,25] where the agents should synchronize their state with the remaining agents while achieving the target. The equation can be modeled as shown in Equation (1).

$$\left|x_i\left(t-{\tau }_i\right)-x_j\left(t-{\tau }_j\right)\right|\to \infty \forall i,j$$

(1)

Different approaches are utilized for both consensus and synchronization because consensus for a team of agent concerns for distributed cooperative control while in complex network synchronization concerns for non-linear dynamics. Similarly, in multi-agent consensus the self-dynamic is linear or zero thus leading to a constant final state [20] and can be studied through algebraic graph theory, stochastic matrix theory [26] and convexity analysis [27]. While synchronization in the complex network the self-dynamic is non-linear thus leading to a time-varying final state where the connectivity is known prior and can be studied through algebraic graph theory, matrix theory [28] and Lyapunov function [29].

3.3. Convergence Speed in Finite Time

An important performance measure in consensus protocol is the convergence speed in finite time. The main objective is to design a control law in such a way that states should reach a common point in $t\ge T$, where T is the constant consensus time. The finite time consensus has two significant properties as compared to others in the sense of robust against uncertainties and disturbance rejection.

As discussed earlier that except the smallest eigenvalue of Laplacian matrix which is always zero, all the remaining eigenvalues should be positive for a connected undirected graph, where ${\lambda }_2$ is the smallest positive eigenvalue. There are two measures to know about the convergence speed, one is by using ${\lambda }_2$ value which is expected to be large for a random network and the third smallest eigenvalue should be far away from ${\lambda }_2$ for faster convergence. Another measure is by using the ratio in Equation (2) [30].

$$\rho =\underset{t\to \infty ,X\left(t\right)\ne X^{*}}{\lim }{\left(\frac{\Vert X\left(t\right)-X^{*}\Vert }{\Vert X\left(0\right)-X^{*}\Vert }\right)}^{1/t}$$

(2)

where, $X^{*}$ represents as the final equilibrium state. Reference [20] mentions the worst-convergence speed for an undirected connected graph is Equation (3).

$$\underset{X\ne 0,{\mathbf{1}}^{T}X=0}{\min }\frac{X^{T}LX}{{\Vert X\Vert }^2}={\lambda }_2$$

(3)

Using optimal weight, consensus maximum fastest speed is discussed in [31]. Also, per-step convergence factor for convergence speed is presented in [32] using stochastic theory. Continuous finite time consensus with single integral dynamic [33] and double integral kinematic [34] are demonstrated using a signum function [35]. Recently discrete finite time consensus is investigated in [36].

3.4. Heterogeneous Agents

In comparison to homogeneous agents, where all the agents have the same dynamics, which is not practical in many real-life applications. So an extensive study has been done recently on heterogeneous agents group where each agent has different objectives, role, preference and capabilities [37,38]. For example, in the multi-agent system case when there are some agents on the ground and others are in space, having different dynamics with each other. This motivates a hot research topic for researchers in multi-agent system. Similarly, it is also not practical that all agents may updates their states synchronously with each other at the same time, because some agents may not get any update from any other agent will stay to its previous status as compared to dominant agents who require to update their state continuously. Thus, in reality, it is more suitable that agent should update their states regardless of other agents known as an asynchronous effect.

This effect is also investigated using matrix theory [39] and linear matrix inequality [40]. Non-linear multi-agent consensus protocol for heterogeneous agents using fuzzy disturbance observer [38], distributed $H_{\infty }$ controller [41], internal model principle [42] and switching topology [43] are also studied recently.

4. Convergence Constraints due to Practical Limitations

4.1. Computation, Execution, Control and Communication Delays

In practical, there is always some time delay while sharing information among all the agents. It may affect system stability or the performance of the system by degrading it. The time delay can be categorized as communication, control, computation and actuation delay. Due to the limited communication speed, the delay is called “communication delay”, the time required for the sensor to get information is called “control delay”, the computation time required to generate control input is called “computation delay” and time required for input to be actuated is called “actuation delay”.

If we consider the time delay for the transmitted and received information states which are called input delay that contains both the computation and execution delays, then the equation of continuous time consensus protocol can be modified as shown in (4)

$${\dot{x}}_i\left(t\right)={\displaystyle \sum }_{j=1}^n{a}_{ij}\left(t\right)\left(x_j\left(t-{\tau }_{ij}\right)-x_i\left(t-{\tau }_{ij}\right)\right)$$

(4)

where ${\tau }_{ij}$ is the time delay between agent $i$ and $j$. If the time delay degrades the system performance but not leads to instability, then there is a threshold that must be obeyed. Reference [20], shown that for a graph Laplacian matrix, where for all $i$ and $j$, ${\tau }_{ij}=\tau$ and the communication topology is connected, undirected and fixed then average consensus can be achieved if and only if it obeys (5)

$${\tau }_{ij}\in [0,\frac{\pi }{2{\lambda }_{max}\left(L\right)})$$

(5)

There is another possibility when only the transmitted information state is affected by the time delay, which is known as “communication delay” then the equation of continuous time consensus protocol will be modified as (6)

$${\dot{x}}_i\left(t\right)={\displaystyle \sum }_{j=1}^n{a}_{ij}\left(t\right)\left(x_j\left(t-{\tau }_{ij}\right)-x_i\left(t\right)\right)$$

(6)

where, the collective dynamic can be expressed as ${\dot{x}}_i\left(t\right)=-Lx\left(t-\tau \right)$. Fortunately, packet loss can be treated as a special case of communication delay, where packet loss can be considered as retransmission of that packet after it is loss in exchanging information. It is mentioned in Reference [44] that input delay does affect the consensus ability but the communication delay does not. Reference [45], describes that for all $i$ and $j$, ${\tau }_{ij}=\tau$ and the communication topology is switching and directed then consensus results presented previously is still applicable for an arbitrary time delay.

For a regular or scale-free network, there is more delay as compared to the random or small world [46], because in the regular network there is a high degree of nodes that is not good for consensus speed thus leading it to a trade-off between time delay and large maximum degree. For a regular network with $N=20$ and $K=80$, ${\lambda }_2$ is small as shown in Figure 5a, in contrast to random network with $N=20$ and $K=80$ as shown in Figure 5b.

As the second smallest eigenvalue is relatively large for the dense graph as compared to sparse one and the third smallest eigenvalue should be far away from ${\lambda }_2$ for fastest convergence. The author in [20], claims that the average consensus can be achieved if for all initial conditions one hop time delay is within $0\le \tau <\pi /2{\lambda }_n$. The speed of convergence for 3-different networks with $100$ nodes is shown in Figure 6.

Time delay consensus with complex dynamics are also studied for double-integrator dynamics [48], general nonlinear dynamics [49] and rigid bodies [50]. Similarly, main tools of stability analysis for both linear and nonlinear dynamics are addressed through matrix theory [51], Lyapunov approach [52], contraction principle [53] and frequency domain function [44].

4.2. Quantization & Sample-Data Consensus

Quantization and sample data are significant practical constraints. As states are real values but only finite bits of information are transmitted at each time step. In reality, to implement the consensus algorithm the measurements should be digital rather than analog is known as quantization consensus. The main research on quantization consensus is that maximum state difference should not be lager enough then the associated system accuracy level which is motivated by digital signal processing. For a quantize signal $Q\left(s\right)$, if $s$ is the analog signal and $\delta$ is the accuracy parameter or quantization step size then the typical quantizer can be written as $Q\left(s\right)=q\left(s,\delta \right)$. Reference [54], indicates that for an integer set $Z$ the rounding signal to its closest neighbor can be modeled as (7)

$$s\in \left[\left(n-\frac{1}{2}\right)\delta ,\left(n+\frac{1}{2}\right)\delta \right]$$

(7)

There are some notable features for consensus with quantization such that in a finite time the system should converge to an accuracy level, where the convergence time depends on both network topology and quantization level. Also, the author in [55] claims that convergence rate does not depend on the coding or decoding, rather that it only depends on system accuracy and for distributed consensus, the convergence rate depends on the connectivity and synchronizability [56]. It is also the reality that by any method someone will choose the control parameters the quantization level will surely increase as the number of agent increases.

As the graph-theoretic method has some limitation for environment and feedback over a delayed and loss-less line where there are some issues of packet loss and even for a pair of agents are dominant [19]. So for stabilizing the linear time-invariant system, the minimum channel capacity in term of bit rate is proposed in [57,58] and with logarithm quantizer, it is shown in [59]. An algorithm related to quantization consensus which is based on sector bound is proposed in [60]. Similarly to ensure stabilization using stochastic [61], sample encoded measurement [62] and saturating quantized measurements [63] are also investigated. For average consensus, the dynamic quantizer and dithered quantizer can be seen in [64,65].

There are also some limitations in measurements and control because it is almost impractical to acquire measurement information with no time delay and then take a control action instantaneously. Thus, in reality, to tackle this issue a hybrid model is proposed where the system plant is continuous and the control/measurements inputs are addressed as a piecewise constant. By doing so the obvious advantage is that it requires much less computational power and information exchange as compared to continuous time consensus. So, the equation for the sample data consensus can be modified as (8)

$$\dot{x_i}\left(t\right)=\dot{x_i}\left(kT\right)={\displaystyle \sum }_{j=1}^n{a}_{ij}\left(kT\right)\left(x_j\left(kT\right)-x_i\left(kT\right)\right),ForkT\le t<\left(k+1\right)T$$

(8)

where $T$ is the sampling period and $k$ is described as a discrete-time index.

The main research concentrates to find different conditions on sampling period. Sample date consensus with a fix and switching topology using single integrator [66] and double integrator kinematics [67] are also investigated. Similarly approaches with different conditions for sample data consensus are address using stochastic matrices [67], Lyapunov function [68], matrix theory [48] and linear matrix inequality [69].

5. Application of Consensus in Multi-Agent Network

General multi-agent consensus applications that are being extensively pursued in this field, are briefly mentioned ahead

• Multi-agent flocking means to achieve some common group objectives by interacting with each other.
• Swarm is an approach of multi-agent system that takes inspiration from social animals that exhibit a self-organized behavior. Through local interactions and simple rules, swarm agents focus flexible, scalable and robust collective behaviors for the coordination of multi-agent system.

The difference in flocking and swarming is that in flocking there is a homogeneous and coherent directed motion whereas swarming is the condensation into a compact group propagating in a coherent fashion. Consensus-based different protocols can be seen in

Table 2. Different Coordination Protocols.

	Protocol	Source
Consensus	$1/2{\Vert x_j\left(t\right)-x_i\left(t\right)\Vert }^2=0$	[20,26,70,71]
Flocking	$1/2{\Vert x_j\left(t\right)-x_i\left(t\right)-d\Vert }^2=d$	[5,8,9,72,73]
Swarming	$1/2{\Vert x_j\left(t\right)-x_i\left(t\right)\Vert }^2<d$	[74,75,76,77]

.

5.1. Rendezvous

Each agent should reach a common point at a common time is called “rendezvous” problem, where time can be dynamic or fixed depending on the application. Rendezvous can be classified as the cooperative tracking problems that require multi-agent system to reach a pre-specified point [78].

Rendezvous has been mentioned in many literatures in a variety of settings. Basically, there are two main types which are proposed for rendezvous known as, synchronous and asynchronous rendezvous problems where in synchronous all agents share the same clock and in asynchronous they did not [79,80]. In [81] for both synchronized and non-synchronized multi-agent, consensus-based rendezvous problem is applied to guaranteed a common location. The simple rendezvous algorithm of three agents achieving a common point is shown in Figure 7.

5.2. Formation Control

As compared to multi-agent consensus where all the agents need to reach a common point, in formation control a particular geometric shape of all the agents using consensus scheme is required and it is extensively studied in many literatures. Formation control of multi-agent system can be found in [19,82], while consensus bases formation can be seen in [47]. Moreover, formation control in term of sensing capabilities is subdivided as position, displacement and distance-based control.

• In position-based control, agents sense their own position with respect to the global coordinate system.
• In displacement-based control, agents sense the relative position of its neighbor with respect to the global coordinate system.
• In distance-based control, agents sense the relative position of its neighbor with respect to own local coordinate system.

The difference between displacement and distance based control is depicted in Figure 8.

Distance-based control is a bit complicated because now the system becomes nonlinear. The important factor in distance-based control is the graph rigidity. According to [83] for $n$ agents, the minimum number of edges for rigid graph should be $2n-3$. The example of an undirected framework for rigid can be seen in Figure 9. In Figure 9a the structure is not unique and can easily be deformed as depicted in dotted line so it is not rigid. By joining node 2 and 3 it becomes locally unique to lie in the category of the rigid graph as shown in Figure 9b. Finally, joining node 1 and 4 it becomes globally rigid as it is globally unique as shown in Figure 9c.

The example of a directed framework for rigid and persistent can be seen in Figure 10. As Figure 10a can easily be deformed so it is not rigid. Although Figure 10b is rigid but the node 2 independently cannot control the lengths of three edges so it is not persistent. Finally, in Figure 10c, the responsibility to control the length of edges is well-distributed so it is persistent [84].

The distinction between position, displacement and distance-based formation control in term of sensed variables, controlled variables, coordinate system and interaction topology is presented in

Table 3. Distinctions among position, displacement and distance-based formation control [85].

	Position-Based	Displacement-Based	Distance-Based
Sensed Variables	Positions of agents	Relative positions of neighbors	Relative positions of neighbors
Controller Variables	Positions of agents	Relative positions of neighbors	Inter-agent distance
Coordinate System	A global coordinate system	Orientation aligned local coordinate systems	Local Coordinate systems
Interaction Topology	Usually not required	Connectedness or existence of spanning tree	Rigidity or persistence

.

To form a formation control, first the system has to execute the rendezvous control strategy until it achieves a complete graph then it will change the desired formation control using only local interaction as shown in Figure 11, where from (a–c) rendezvous is achieved and from (d–f) formation control is formed.

6. Discussion

Based on the presented literature review, the authors conclude this article on the multi-agent cooperative control consensus as well as highlight potential research challenges in this field.

6.1. Consensus

A lot of work has been done regarding continuous and discrete time consensus. For a directed topology, consensus can be achieved when the topology is strongly connected but to achieve average consensus topology should be strongly connected and balanced. Whereas for undirected graph, the agreement can be achieved only if the graph is connected [20]. For this control, most researchers focus on the homogenous agent using simple dynamic. For effective control of multi-agent system, further research is essential to harness the nonlinear dynamics of these dynamical systems. Future directions and some open problems that can be pursued in this area are as follows.

• Existing control strategies for multi-agent consensus and formation focus on simple system dynamics using basic connectivity assumption and Laplacian matrix, so higher order dynamics or nonlinear dynamics is still needed to be investigated.
• Using heterogeneous agents, more work is required in the field of consensus for more complicated nonlinear dynamic so that each agent can choose the best responses based on its own objectives.

6.2. Formation

As in formation control, a particular geometric shape of all the agents is required based on some consensus scheme.

In ideal condition for position-based control there is no interaction among all the agents to form a formation but practically there is some issue like time delay, actuator saturation and disturbances causing inter-agent interaction. This control is a practical solution for formation but the disadvantage is the requirement of very high sensing equipment like GPS, thus owing to this requirement they are very costly. The author in [86] proposed a position-based control law for double integrator while for a non-holonomic agent similar idea is presented in [87]. Using relative position feedback variable size formation is addressed in [88], where desired scaling size information is known to only a few agents whereas shape information is known to all the agents.

In displacement-based control, for the undirected graph the only requirement is that the graph should be connected and for the undirected one, the requirement is that the topology should be spanning tree. The consensus dynamic for an undirected graph is discussed in [20] and the extension of this for directed one is discussed in [23]. Reference [89] described the intermittent interaction for general linear agents. Under displacement-based problem, a significant topic that has been studied in the recent years is the formation scaling issue which is discussed in [88,90]. The reason for this scaling is the adjustment of all the agents depending on the situation.

In distance-based control, depending on the desired value among the agents the formation is formed by the inter-agent interaction. Thus, more interaction is required among the agents but the advantage is in term of less sensing capabilities because less global information is needed. The author in [91] argued that by reaching the consensus on the center position of the formation can lead to agent formation. The paper also discusses different communication topologies to improve the performance of formation and stability margins. Formation Control in term of sensing capabilities is categorized in

Table 4. Formation Control with respect to Sensing Capabilities.

Sensing Capabilities Formation Control	Strengths	Bottlenecks	References
Position-based	• Easy to implement • Simple mechanism	Costly because GPS system is required	[86,87,88]
Displacement-based	• Spanning tree required for directed network • Connectedness required for undirected network	Orientation aligned local coordinate system	[20,89,90,92]
Distance-based	• Less sensing capabilities required • Less global information required	Complicated because system is non-linear	[93,94]

.

Formation control in term of time-varying can be categories as formation producing and formation tracking.

In formation producing the control objective is to form a pre-specified pattern without a group reference. Ref. [95], in order to offset the control input by some angle a coupling matrix $C$ with compatible size is introduced in the consensus equation. Thus, the modified equation can be described in (9)

$${\dot{x}}_i\left(t\right)={\displaystyle \sum }_{j=1}^n{a}_{ij}\left(t\right)C\left(x_j\left(t\right)-x_i\left(t\right)\right)$$

(9)

In linear closed loop system, the formation production has two significant aspects in the fixed network topology. The first aspect is that at least one eigenvalue should be zero and other is the presence of at least one imaginary eigenvalues pair.

In formation tracking the control objective is to form a pre-specified pattern with a particular group reference making it more complex as compared to formation producing. In such control, the main task is to design an algorithm for distributed control in order to derive multi-agent system to follow some pre-specified state so that by keeping the geometric formation agents also track the group reference. Formation Control in term of time-varying is categories in

Table 5. Formation Control with respect to Time-varying.

Time-Varying Formation Control	Control Objective	Methodologies	References
Formation Producing	Pre-specified pattern without a group reference	Matrix theory, Lyapunov, Graph rigidity, Receding horizon approach, Leaderless flocking, Inverse agreement problem, Circulation formation	[96,97,98,99]
Formation Tracking	Pre-specified pattern with a particular group reference	Matrix theory, Lyapunov, Gradient-based function, Variable structural based law	[100,101,102,103]

.

In real life, there are many applications where the network topology is directed but this literature review reveals that mostly the researchers discuss the undirected network for formation rather than directed one. Also, the stability analysis is stick to only three agents. Future directions and some open problems that can be pursued in this area are as follows.

• Global stability properties for general rigid and persistent formation are yet to be investigated (Only triangle formation is done so far).
• For formation producing the network topology is assumed to be undirected which is not applicable to many practical applications.
• More research efforts on distance-based formation with moving leader is needed.

6.3. Obstacle Avoidance

As revealed by reviewed studies, the obstacle avoidance phenomena in multi-agent system have been extensively analyzed. But, in multi-agent system obstacle avoidance mostly the researchers focus on the point or circular obstacles which are not practical in real life. Certainly, there is an imperative need for improvised mathematical methods so that dynamical systems can be studied analytically in place of computer simulations and numerical analysis. The dynamic of agents to achieve all the three tasks; achieve target using consensus, formation formed and avoid obstacles is shown in (10), whereas the simulation can be seen in Figure 12.

$${\dot{x}}_i=F_{form}+F_{goal}+F_{obstacle}$$

(10)

There is a dearth of literature on complex obstacles in this area. Future directions and some open problems that can be pursued in this area are as follows.

• Multi-agent consensus for complex obstacles is still to be investigated, where the task connectivity preservation and collision avoidance issues are important.

7. Conclusions

From this review article, it can be appreciated that research on the multi-agent formation has progressed through a number of stages. During the first stage, the consensus techniques for directed and undirected dynamics in multi-agent system were uncovered. It was followed by formation algorithms. This endeavor has entered the last phase, where researchers are applying cooperative control techniques to resolve practical problems for unseen obstacles.

This paper describes the current status and future of cooperative control consensus in multi-agent system. The main focus of this work was to give a brief overview of cooperative control consensus, which has significant importance in multi-agent system. Summary of theoretical result including dynamically changing and time-invariant communication topologies, convergence speed in finite time and heterogeneous agents are presented. Furthermore, the convergence constraint due to practical limitations and general application are also discussed. To promote more research in this filed, useful future recommendation with some open problems are also proposed.