Hidden Markov Model-Based Control for Cooperative Output Regulation of Heterogeneous Multi-Agent Systems under Switching Network Topology

Abstract

This paper investigates the problem of stochastically cooperative output regulation of heterogeneous multi-agent systems (MASs) subject to hidden Markov jumps using observer-based distributed control. In order to address a more realistic situation than prior studies, this paper focuses on the following issues: (1) asynchronous phenomena in the system mode’s transmission to the controller; (2) the impact of system mode switching on network topology; and (3) the emergence of coupled terms between the mode-dependent Lyapunov matrix and the control gain in control design conditions. Specifically, to reduce the complexity arising from the asynchronous controller-side mode, the leader–state observer is developed so that the solution pair of regulator equations can be integrated into the observer. Furthermore, a linear decoupling method is proposed to handle the emergence of the aforementioned coupled terms; this provides sufficient LMI conditions to achieve stochastically cooperative output regulation for heterogeneous MASs. Finally, the validity of the proposed method is shown through two illustrative examples.

1. Introduction

Multi-agent systems (MASs) refer to complex systems composed of multiple autonomous agents that interact with each other and their environment, they have been used in various research fields, including robotics [1,2], automated vehicles [3,4], unmanned autonomous vehicles [5,6,7,8], and urban networks [9,10]. Recently, with the advent of these systems, effective techniques for cooperating MASs with different structures and parameters (referred to as heterogeneous MASs) have been swiftly developed for various purposes, including leader following and formation.

Over the past few years, the cooperative output regulation problem has also been regarded as one of the most fundamental consensus problems for MASs. In this problem, one essential requirement is to develop a control strategy that ensures stable and efficient cooperation between agents while achieving the desired overall performance. Additionally, the control strategy should be able to achieve appropriate behaviors with MASs by explicitly considering the interconnections and interactions between multiple agents. Following this, various methods have been proposed to deal with the cooperative output regulation problem of heterogeneous MASs on the premise of a fixed network topology and system parameters (see [11,12,13,14,15] and references therein). However, the network topology and system parameters can randomly change due to obstacles posed by network sizes, functional connectivity disturbances, limited communication ranges, and random packet losses.

As a mathematical model for handling the aforementioned random changes, Markov jump multi-agent systems have been widely utilized in many control problems, such as the leader–following consensus control [16,17], scaled consensus control [18], the formation control [19], and the cooperative output regulation [20,21,22,23]. The above studies mainly focused on control problems for MASs with deterministic dynamics (with no sudden changes), and the Markov process was only used to model sudden changes in the network topology, or vice versa. However, little effort has been devoted to research on realistic cases in which rapid changes in the system modes of MASs affect the network topology. A more serious problem is that due to network issues, such as packet dropout and data transmission delay, the controller modes cannot be designed in accurate synchronization with the system or network topology modes. Thus, it is necessary to carefully consider the impact of this asynchronous problem when designing a controller that operates in such an environment. According to this need, [24] used a hidden Markov model (HMM) to deal with the problem of the leader–following consensus for MASs with asynchronous control modes. However, in [24], random changes in the network topology are modeled as a Markov process, but the system parameters of MASs are assumed to be deterministic. Hence, to overcome these weaknesses, more progress needs to be made toward addressing the impact of changes in both system parameters and network topology while achieving the HMM-based cooperative output regulation for continuous-time heterogeneous MASs.

Based on the above discussion, the main goal of this paper is to address the problem of stochastically cooperative output regulation for continuous-time heterogeneous MASs with hidden Markov jumps in the system mode and network topology. First, a mode-dependent leader–state observer is designed to transmit an estimated leader–state to each follower agent. After that, an asynchronous mode-dependent distributed controller is designed so that it can ensure the stochastically cooperative output regulation of MASs. To be specific, the main contributions of this paper can be summarized as follows.

• This paper makes a first attempt to reflect the influence of the asynchronous mode between heterogeneous MASs and observer-based distributed controllers while achieving stochastically cooperative output regulation subject to Markov jumps. Different from [22,23,25,26], the realistic case where rapid changes in the system modes of MASs affect the network topology is considered in the control design processes.
• This paper proposes a method to design a continuous-time leader–state observer capable of estimating the leader–state value for each agent under abrupt changes in both systems and network topology. Also, it introduces an alternative mechanism by integrating system-mode-dependent solutions of regulator equations into the output of the leader–state observer to reduce the complexity arising from the asynchronous controller-side mode.
• In the control design process, the asynchronous mode-dependent control gain is coupled with the system-mode-dependent Lyapunov matrix, which makes it difficult to directly use the well-known variable replacement technique [27]. For this reason, this paper suggests a suitable linear decoupling method that is capable of handling the aforementioned coupling problem.

The rest of the paper is organized as follows. Section 2 presents a class of heterogeneous MASs with hidden Markov jumps under our consideration. Next, Section 3 presents methods for designing a mode-dependent leader–state observer and asynchronous mode-dependent distributed controllers for MASs. In Section 4, two illustrative examples are provided to demonstrate the validity of the proposed method. Finally, the concluding remarks are given in Section 5.

Notations: $\mathrm{spec}\left(A\right)$ denotes the eigenvalue set of matrix A. In symmetric block matrices, $(\ast )$ is used as an ellipsis for terms induced by symmetry. $\mathbf{diag}(·)$ stands for a block-diagonal matrix; $\mathbf{col}(v_1,v_2,\cdots ,v_n)={[v_1^T{v}_2^T\cdots v_n^T]}^T$ denotes the scalars or vectors, $v_i$; $\mathbf{He}\{\mathcal{Q}\}=\mathcal{Q}+{\mathcal{Q}}^T$ denotes any square matrix $\mathcal{Q}$; ⊗ denotes the Kronecker product; $I_n$ denotes the n-dimension identity matrix; ${\left|\right|x\left|\right|}_2$ denotes the Euclidean norm of vector x; ${\lambda }_{max}\left(A\right)$ denotes the maximum eigenvalue of matrix A; ${\left|\right|A\left|\right|}_2^2={\lambda }_{max}\left(A^{T}A\right)$ denotes matrix A; $Re\left(\lambda \right)$ denotes the real part of $\lambda$; ${\mathbb{N}}_n$ denotes the set $\{1,2,\cdots ,n\}$; $\mathbf{E}\{·\}$ denote the mathematical expectation; “T” and “$-1$” represent matrix transposition and inverse, respectively. The triplet notation $\left(\Omega ,\mathcal{F},Pr\right)$ denotes a probability space, where $\Omega$, and Pr represent the sample space, the algebra of events, and the probability measure defined on $\mathcal{F}$, respectively.

2. Preliminaries and Problem Statement

2.1. Heterogeneous Multi-Agent System Description

Let us consider the following continuous-time Markov jump dynamics of the ith follower and the leader (or exogenous system), defined on a complete probability space $(\Omega ,\mathcal{F},Pr)$:

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}\hfill & {\dot{x}}_i\left(t\right)=A_i\left(\varphi \left(t\right)\right)x_i\left(t\right)+B_i\left(\varphi \left(t\right)\right)u_i\left(t\right)+D_i\left(\varphi \left(t\right)\right)x_0\left(t\right),\hfill \\ \hfill & y_i\left(t\right)=C_i\left(\varphi \left(t\right)\right)x_i\left(t\right),for\forall i\in {\mathbb{N}}_N\hfill \\ \hfill & {\dot{x}}_0\left(t\right)=A_0\left(\varphi \left(t\right)\right)x_0\left(t\right),\hfill \\ \hfill & y_0\left(t\right)=C_0\left(\varphi \left(t\right)\right)x_0\left(t\right),\hfill \end{array}\right.\end{array}$$

(1)

where $x_i\left(t\right)\in {\mathbb{R}}^n$, $u_i\left(t\right)\in {\mathbb{R}}^m$, and $y_i\left(t\right)\in {\mathbb{R}}^p$ denote the state, the control input, and the output of the ith follower, respectively; $x_0\left(t\right)\in {\mathbb{R}}^l$ and $y_0\left(t\right)\in {\mathbb{R}}^p$ are the state and the output of the leader, respectively; ${\mathbb{N}}_N=1,2,\dots ,N$ is the set of agents; $\varphi \left(t\right)\in {\mathbb{N}}_{\alpha }$ denotes the time-varying switching system mode with ${\mathbb{N}}_{\alpha }$ denoting the set of systems modes; and N is the number of follower agents. In (1), $A_i(\varphi \left(t\right)=g)=A_{ig}$, $B_i(\varphi \left(t\right)=g)=B_{ig}$, $C_i(\varphi \left(t\right)=g)=C_{ig}$, $D_i(\varphi \left(t\right)=g)=D_{ig}$, $A_0(\varphi \left(t\right)=g)=A_{0g}$, and $C_0(\varphi \left(t\right)=g)=C_{0g}$ are known system matrices with appropriate dimensions; and $\left(A_{ig},B_{ig}\right)$ and $\left(C_{ig},A_{ig}\right)$ are controllable and detectable, respectively. In addition, the process $\{\varphi \left(t\right)\in {\mathbb{N}}_{\alpha },t\ge 0\}$ is characterized by a continuous-time homogeneous Markov process with the following transition probabilities (TPs):

$$\begin{array}{c}\hfill Pr\left(\varphi (t+\delta )=h\left|\varphi \right(t)=g\right)=\left\{\begin{array}{c}{\pi }_{gh}\delta +o\left(\delta \right)\hfill \\ 1+{\pi }_{gg}\delta +o\left(\delta \right)\hfill \end{array}\right.,\end{array}$$

(2)

where $o\left(\delta \right)$ is the little-o notation defined as ${lim}_{\delta \to 0}\left(o\left(\delta \right)/\delta \right)=0$; and ${\pi }_{gh}$ indicates the transition rate (TR) from mode g to mode h at time $t+\delta$ and satisfies that ${\sum }_{h\in {\mathbb{N}}_{\alpha }}{\pi }_{gh}=0$, and ${\pi }_{gh}\ge 0$ for $h\in {\mathbb{N}}_{\alpha }\setminus \left\{g\right\}$.

Remark 1. 

Different from other studies [22,23,25,26,28], this paper deals with the case where changes in system parameters (affected by the life span of system components, the increase in heat of motors, wear and tear, etc.) directly lead to signal loss or degradation in the transmission quality between agents, inducing changes in the network topology. In addition, external influences could also impact the system parameters and communication network of agents at the same time. Therefore, in this paper, the network topology mode is set to be the same as the system mode $\varphi \left(t\right)$.

To be specific, Figure 1 shows a block diagram of the cooperative output regulation of multi-agent systems under our consideration, which contains four main parts: the multi-agent system ($v_i$), network analyzer, leader–state observer, and distributed controller ($c_i$). Functionally, the network analyzer finds the current network topology mode using data transmitted over a wireless local area network (black dash line), and the mode information is transmitted to the observer directly and to the distributed controller over a wireless wide area network (blue dash line). As in [23,29], the leader–state observer provides the estimated leader–state ${\widehat{x}}_{0i}$ for the ith agent to overcome the difficulty that some followers cannot receive information directly from the leader. Next, based on $x_i$, ${\Pi }_{ig}{\widehat{x}}_{0i}$, and ${\Lambda }_{ig}{\widehat{x}}_{0i}$, the controller $c_i$ generates the control input $u_i$ and transmits it to the ith agent so that the output $y_i$ approaches $y_0$ as time increases. In a real environment, since the transmission of the network topology mode can be affected by various network issues from a wireless wide area network, such as network-induced delay, packet dropout, network congestion, and interference, this paper employs another asynchronous observation mode, $\rho \left(t\right)\in {\mathbb{N}}_{\beta }$. In other words, this paper considers a hidden Markov model (HMM) concept with $\varphi \left(t\right)$ and $\rho \left(t\right)$ when designing the distributed controller, where the asynchronism between the two is described as the following conditional probability: $Pr(\rho \left(t\right)=s|\varphi \left(t\right)=g)={\varpi }_{gs}$, which satisfies ${\sum }_{s\in {\mathbb{N}}_{\beta }}{\varpi }_{gs}=1$ and ${\varpi }_{gs}\ge 0$ for $g\in {\mathbb{N}}_{\alpha }$ and $s\in {\mathbb{N}}_{\beta }$.

Remark 2. 

If the network topology mode observed in the network analyzer is not affected by network issues in the transmission process, there will be no asynchronous phenomenon between $\varphi \left(t\right)$ and $\rho \left(t\right)$, i.e., $\rho \left(t\right)=\varphi \left(t\right)$ and ${\mathbb{N}}_{\beta }={\mathbb{N}}_{\alpha }$. To cover this special case, the conditional probability matrix ${\left[{\varpi }_{gs}\right]}_{g\in {\mathbb{N}}_{\alpha },s\in {\mathbb{N}}_{\alpha }}\in {\mathbb{R}}^{\alpha \times \alpha }$ can be set to the identity matrix $I\in {\mathbb{R}}^{\alpha \times \alpha }$.

Assumption A1. 

For $\varphi \left(t\right)=g$, $Re\left({\lambda }_{0g}\right)\ge 0,\forall {\lambda }_{0g}\in spec\left(A_{0g}\right)$.

Assumption A2. 

For $\varphi \left(t\right)=g$, there are pairs of solutions $({\Lambda }_{ig},{\Pi }_{ig})$ for the following regulator equations:

$$\begin{array}{cc}\hfill & 0=A_{ig}{\Lambda }_{ig}+B_{ig}{\Pi }_{ig}+D_{ig}-{\Lambda }_{ig}{A}_{0g},\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & 0=C_{ig}{\Lambda }_{ig}-C_{0g},\forall i\in {\mathbb{N}}_N.\hfill \end{array}$$

(4)

where ${\Lambda }_{ig}\in {\mathbb{R}}^{n\times l}$ and ${\Pi }_{ig}\in {\mathbb{R}}^{m\times l}$.

Remark 3. 

Assumption 1 is made only for convenience and loses no generality. In fact, if the linear output regulation problem is solvable by any controller under Assumption 1, it is also solvable by the same controller even if Assumption 1 is violated. More explanations can be found in [30]. Assumption 2 provides the well-known regulator equations [23,31] whose solutions impose a necessary condition for the control design process. Furthermore, the feasibility of Assumption 2 is guaranteed according to Remark 4.

Remark 4 

([30]). For $\varphi \left(t\right)=g$, the solution pair $\left({\Lambda }_{ig},{\Pi }_{ig}\right)$ of Assumption 2 exists if and only if all eigenvalues $\lambda \in spec\left(A_{0g}\right)$ satisfy

$$rank\left[\begin{array}{cc}{A}_{ig}-\lambda I& B_{ig}\\ C_{ig}& 0\end{array}\right]=n+p,\forall i\in {\mathbb{N}}_N.$$

(5)

2.2. Communication Topology

As mentioned in Remark 1 and Figure 1, the network topology of (1) with $\varphi \left(t\right)=g\in {\mathbb{N}}_{\alpha }$ is represented as a mode-dependent weighted and directed graph (digraph) ${\mathcal{G}}_g=\left(\mathcal{V},{\epsilon }_g,{\mathcal{A}}_g\right)$ formed by N follower agents. Here, $\mathcal{V}=\{v_1,v_2,\cdots ,v_N\}$ denotes the node set; ${\epsilon }_g⫅\left\{(v_j,v_i)|v_i,v_j\in \mathcal{V},j\ne i\right\}$ denotes the edge set with the ordered pair $(v_j,v_i)$ that has information flow leaving from agent $v_j$ to agent $v_i$ at time t; and ${\mathcal{A}}_g={\left[a_{ij,g}\right]}_{i,j\in \{1,2,\cdots ,N\}}$ denotes the adjacency matrix with $a_{ij,g}>0$ if and only if $(v_j,v_i)\in {\epsilon }_g$, and $a_{ij,g}=0$ otherwise. In addition, the neighbor set and degree of $v_i\in \mathcal{V}$ are defined as ${\mathcal{N}}_{i,g}=\left\{v_j\right|(v_j,v_i){\in }_g\}$ and $d_{i,g}={\sum }_{j=1}^N{a}_{ij,g}$, respectively, and the Laplacian matrix of ${\mathcal{G}}_g$ is given by ${\mathcal{L}}_g={\mathcal{D}}_g-{\mathcal{A}}_g\in {\mathbb{R}}^{N\times N}$, where ${\mathcal{D}}_g=\mathbf{diag}\left(d_{1,g},\cdots ,d_{N,g}\right)$. Furthermore, a directed path leaving from node $v_j$ to node $v_i$ is a sequence of ordered edges [24]. In particular, the multi-agent system under our consideration consists of N follower agents and one leader $v_0$ with a spanning tree from the leader to each agent in the network topology. Thus, to represent such a system, this paper considers an extended graph ${\mathcal{G}}_{0,g}=({\mathcal{V}}_0,{\epsilon }_{0,g},{\mathcal{M}}_g)$ with ${\mathcal{V}}_0=\mathcal{V}\bigcup \left\{v_0\right\}$, ${\epsilon }_{0,g}\subseteq \left\{(v_0,v_i)\right|i\in \mathcal{V}\}$, and ${\mathcal{M}}_g=\mathbf{diag}(m_{1,g},\cdots ,m_{N,g})\in {\mathbb{R}}^{N\times N}$, where ${\mathcal{M}}_g$ denotes the leader adjacency matrix with $m_{i,g}>0$ if and only if the leader $v_0$ transmits information to $v_i$, and $m_{i,g}=0$ otherwise. That is, the union of graphs is given by ${\mathcal{G}}_0:={\bigcup }_{g=1}^{\alpha }{\mathcal{G}}_{0,g}$, and its node set is equal to ${\mathcal{V}}_0$.

Remark 5. 

By considering the bidirectional information link between agents, this paper ensures that all agents have the opportunity to communicate with each other, depending on the network topology modes.

The following definitions and lemma will be adopted in this paper.

Definition 1 

([32,33]). Let us consider a Markov jump linear system with state $\chi \left(t\right)$. For any initial conditions, $\chi \left(0\right)$, $\varphi \left(0\right)$, and $\rho \left(0\right)$, if the state $\chi \left(t\right)$ satisfies

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}\mathbf{E}\left\{{\int }_0^t{\left|\right|\chi \left(\tau \right)\left|\right|}_2^{2}d\tau |\chi \left(0\right),\varphi \left(0\right),\rho \left(0\right)\right\}<\infty ,\end{array}$$

(6)

then the Markov jump linear system is stochastically stable.

Definition 2 

([20,23,34]). The heterogeneous MAS system (1) is said to achieve the stochastically cooperative output regulation if it holds that

• System (1) is stochastically stable when $x_0\left(t\right)=0$,
• For any initial conditions, $x_i\left(0\right)$and $x_0\left(0\right)$,

  $$\begin{array}{cc}\hfill & \underset{t\to \infty }{lim}\mathbf{E}\left\{{\int }_0^t\left|\right|e_i\left(\tau \right){\left|\right|}_2^{2}d\tau |e_i\left(0\right),\varphi \left(0\right),\rho \left(0\right)\right\}<\infty \hfill \end{array}$$

  (7)

   where$e_i\left(t\right)=y_i\left(t\right)-y_0\left(t\right)$ represents the error between the output of the ith agent and the output of the leader.

Lemma 1 

([35]). For any matrix $U>0$, and matrices X and Y of compatible dimensions, the following inequality holds:

$$\begin{array}{c}\hfill \mathbf{He}\left\{X^{T}Y\right\}\le X^{T}UX+Y^T{U}^{-1}Y.\end{array}$$

3. Main Results

As in [29], this paper first designs a mode-dependent leader–state observer that provides an estimated leader–state for each agent. Following that, this paper designs a distributed controller that achieves cooperative output regulation for heterogeneous multi-agent systems (1).

3.1. Leader–State Observer Design

As depicted in Figure 1, the leader–state observer directly receives g and $x_0$ from the network analyzer and the leader. Thus, the mode-dependent observer for the ith agent can be established as follows: for $\varphi \left(t\right)=g\in {\mathbb{N}}_{\alpha }$,

$$\begin{array}{c}\hfill {\dot{\widehat{x}}}_{0i}\left(t\right)=A_{0g}{\widehat{x}}_{0i}\left(t\right)+F_g\left(\sum _{j=1}^N{a}_{ij,g}\left({\widehat{x}}_{0i}\left(t\right)-{\widehat{x}}_{0j}\left(t\right)\right)+m_{i,g}\left({\widehat{x}}_{0i}\left(t\right)-x_0\left(t\right)\right)\right),\end{array}$$

(8)

where ${\widehat{x}}_{0i}\left(t\right)\in {\mathbb{R}}^l$ is the estimated leader–state for the ith agent; and $F_g\in {\mathbb{R}}^{l\times l}$ is the observer gain. Thus, based on the ith observation error state ${\widehat{e}}_{0i}\left(t\right)={\widehat{x}}_{0i}\left(t\right)-x_0\left(t\right)$, it is obtained that

$$\begin{array}{c}\hfill {\dot{\widehat{e}}}_{0i}\left(t\right)=A_{0g}{\widehat{e}}_{0i}\left(t\right)+F_g\left(\sum _{j=1}^N{a}_{ij,g}\left({\widehat{e}}_{0i}\left(t\right)-{\widehat{e}}_{0j}\left(t\right)\right)+m_{i,g}{\widehat{e}}_{0i}\left(t\right)\right).\end{array}$$

(9)

That is, the resultant error system is represented as follows:

$$\begin{array}{ll}{\dot{\widehat{e}}}_0\left(t\right)& =\left(I_N\otimes A_{0g}\right){\widehat{e}}_0\left(t\right)+\left({\mathcal{L}}_g+{\mathcal{M}}_g\right)\otimes F_g{\widehat{e}}_0\left(t\right)\\ & =\left(\left(I_N\otimes A_{0g}\right)+\left({\mathcal{L}}_g+{\mathcal{M}}_g\right)\otimes F_g\right){\widehat{e}}_0\left(t\right),\end{array}$$

(10)

where ${\widehat{e}}_0\left(t\right)=\mathbf{col}\left({\widehat{e}}_{01}\left(t\right),{\widehat{e}}_{02}\left(t\right),\cdots ,{\widehat{e}}_{0N}\left(t\right)\right)$.

The following theorem provides the stabilization condition for system (10), formulated in terms of LMIs.

Theorem 1. 

Suppose there exists $0<Q_g=Q_g^T\in {\mathbb{R}}^{l\times l}$ and ${\overline{F}}_g\in {\mathbb{R}}^{l\times l}$ such that the following conditions hold: for $g\in {\mathbb{N}}_{\alpha }$,

$$\begin{array}{c}\hfill 0>I_N\otimes \left(\mathbf{He}\left\{Q_g{A}_{0g}\right\}+\sum _{h\in {\mathbb{N}}_{\alpha }}{\pi }_{gh}{Q}_h\right)+\mathbf{He}\left\{\left({\mathcal{L}}_g+{\mathcal{M}}_g\right)\otimes {\overline{F}}_g\right\}.\end{array}$$

(11)

Then, system (10) is stochastically stable under the Markov network topology, and the observer gain is obtained by $F_g=Q_g^{-1}{\overline{F}}_g$.

Proof. 

Let us choose a mode-dependent Lyapunov function of the following form:

$$\begin{array}{c}\hfill V(t,\varphi \left(t\right)=g)={\widehat{e}}_0^T\left(t\right)\left(I_N\otimes Q_g\right){\widehat{e}}_0\left(t\right),\end{array}$$

(12)

where $I_N\otimes Q_g=\mathbf{diag}\left(Q_g,\cdots ,Q_g\right)>0$. Then, the weak infinitesimal operator acting on $V(t,\varphi (t\left)\right)$ provides

$$\begin{array}{ll}\nabla V\left(t\right)& =\underset{\delta \to 0}{lim}\frac{1}{\delta }\left(\mathbf{E}\left\{V(t+\delta ,\varphi (t+\delta )=h|\varphi \left(t\right)=g)\right\}-V(t,\varphi \left(t\right)=g)\right)\\ & ={\displaystyle \sum _{h\in {\mathbb{N}}_{\alpha }}}{\pi }_{gh}V(t,h)+\dot{V}(t,g)\\ & ={\displaystyle \sum _{h\in {\mathbb{N}}_{\alpha }}}{\pi }_{gh}{\widehat{e}}_0^T\left(t\right)\left(I_N\otimes Q_h\right){\widehat{e}}_0\left(t\right)+\mathbf{He}\left\{{\widehat{e}}_0^T\left(t\right)\left(I_N\otimes Q_g\right){\dot{\widehat{e}}}_0\left(t\right)\right\}\\ & ={\widehat{e}}_0^T\left(t\right)(\mathbf{He}\left\{\left(I_N\otimes Q_g\right)\left(\left(I_N\otimes A_{0g}\right)+\left({\mathcal{L}}_g+{\mathcal{M}}_g\right)\otimes F_g\right)\right\}\\ & \left.+I_N\otimes {\displaystyle \sum _{h\in {\mathbb{N}}_{\alpha }}}{\pi }_{gh}{Q}_h\right){\widehat{e}}_0\left(t\right).\end{array}$$

(13)

Thus, by (13), it can be seen that $\nabla V\left(t\right)<0$ holds if

$$\begin{array}{c}\hfill 0>\mathbf{He}\left\{\left(I_N\otimes Q_g\right)\left(\left(I_N\otimes A_{0g}\right)+\left({\mathcal{L}}_g+{\mathcal{M}}_g\right)\otimes F_g\right)\right\}+I_N\otimes {\tilde{Q}}_g,\end{array}$$

(14)

where ${\tilde{Q}}_g={\sum }_{h\in {\mathbb{N}}_{\alpha }}{\pi }_{gh}{Q}_h$. That is, (14) implies that there exists a small scalar $\kappa >0$, such that $\nabla V\left(t\right)\le -\kappa \left|\right|{\widehat{e}}_0\left(t\right){\left|\right|}_2^2$, and by a generalization of Dynkin’s formula, it is obtained that

$$\begin{array}{ll}\mathbf{E}\left\{V\right(t\left)\right\}-V\left(0\right)& =\mathbf{E}\left\{{\int }_0^t\nabla V\left(\tau \right)d\tau |{\widehat{e}}_0\left(0\right),\varphi \left(0\right)\right\}\hfill \\ & \le -\kappa \mathbf{E}\left\{-{\int }_0^t\left|\right|{\widehat{e}}_0{\left(\tau \right)\left|\right|}_2^{2}d\tau |{\widehat{e}}_0\left(0\right),\varphi \left(0\right)\right\},\end{array}$$

(15)

which results in

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}\mathbf{E}\left\{{\int }_0^t\left|\right|{\widehat{e}}_0{\left(\tau \right)\left|\right|}_2^{2}d\tau |{\widehat{e}}_0\left(0\right),\varphi \left(0\right)\right\}<{\kappa }^{-1}V\left(0\right).\end{array}$$

(16)

Hence, based on Definition 1, system (10) can be said to be stochastically stable. Moreover, defining ${\overline{F}}_g=Q_g{F}_g$, condition (14) can be converted into

$$\begin{array}{c}\hfill 0>I_N\otimes \mathbf{He}\left\{Q_g{A}_{0g}\right\}+\mathbf{He}\left\{\left({\mathcal{L}}_g+{\mathcal{M}}_g\right)\otimes {\overline{F}}_g\right\}+I_N\otimes {\tilde{Q}}_g,\end{array}$$

which becomes (11). □

Remark 6. 

It is worth noting that not all follower agents can obtain the leader–state values directly under the switching network topology, but only the one who is connected to the leader. For distributed control purposes, the mode-dependent leader–state observer is designed for each agent, such that the estimated states can track the leader–state values by intermittent communication. Indeed, following Theorem 1, system (10) is stochastically stable, which leads to ${lim}_{t\to \infty }{\widehat{e}}_{0i}\left(t\right)={\widehat{x}}_{0i}\left(t\right)-x_0\left(t\right)=0$.

Remark 7. 

As shown in Figure 1, ${\widehat{x}}_{0i}\left(t\right)$ is multiplied by each component in the solution pair $({\Lambda }_{ig},{\Pi }_{ig})$ of regulator equations as the output of the leader–state observer and then sent to the controller to reduce the complexity arising from the asynchronous controller-side mode.

3.2. Distributed Controller Design

Let us define ${\tilde{x}}_i\left(t\right)=x_i\left(t\right)-{\Lambda }_{ig}{x}_0\left(t\right)$. Then, the error system of the ith agent is given as follows:

$$\begin{array}{c}\hfill {\dot{\tilde{x}}}_i\left(t\right)=A_{ig}{x}_i\left(t\right)+B_{ig}{u}_i\left(t\right)+D_{ig}{x}_0\left(t\right)-{\Lambda }_{ig}{A}_{0g}{x}_0\left(t\right).\end{array}$$

(17)

Furthermore, for (17), the following distributed control law is considered:

$$\begin{array}{c}\hfill u_i\left(t\right)=K_{is}\left(x_i\left(t\right)-{\Lambda }_{ig}{\widehat{x}}_{0i}\left(t\right)\right)+{\Pi }_{ig}{\widehat{x}}_{0i}\left(t\right),\end{array}$$

(18)

where $K_{is}=K_i(\rho \left(t\right)=s)$ denotes the asynchronous mode-dependent controller gain with $s\in {\mathbb{N}}_{\beta }$.

Thus, based on Assumption 2, the closed-loop system with (17) and (18) is described as follows:

$$\begin{array}{ll}{\dot{\tilde{x}}}_i\left(t\right)& =A_{ig}{x}_i\left(t\right)+B_{ig}\left(K_{is}\left(x_i\left(t\right)-{\Lambda }_{ig}{\widehat{x}}_{0i}\left(t\right)\right)+{\Pi }_{ig}{\widehat{x}}_{0i}\right)\hfill \\ & +D_{ig}{x}_0\left(t\right)-{\Lambda }_{ig}{A}_{0g}{x}_0\left(t\right)\\ & =A_{ig}\left({\tilde{x}}_i\left(t\right)+{\Lambda }_{ig}{x}_0\left(t\right)\right)+B_{ig}{K}_{is}\left({\tilde{x}}_i\left(t\right)+{\Lambda }_{ig}{x}_0\left(t\right)\right)\\ & -B_{ig}{K}_{is}{\Lambda }_{ig}{\widehat{x}}_{0i}\left(t\right)+B_{ig}{\Pi }_{ig}{\widehat{x}}_{0i}\left(t\right)+D_{ig}{x}_0\left(t\right)-{\Lambda }_{ig}{A}_{0g}{x}_0\left(t\right)\\ & =\left(A_{ig}+B_{ig}{K}_{is}\right){\tilde{x}}_i\left(t\right)-\left(B_{ig}{K}_{is}{\Lambda }_{ig}-B_{ig}{\Pi }_{ig}\right){\widehat{e}}_{0i}\left(t\right).\end{array}$$

(19)

The following theorem presents the LMI-based cooperative output regulation conditions of (19).

Theorem 2. 

For any scalars $\mu >0$, $ϵ>0$, and $\delta >0$, suppose that there exist $0<{\overline{P}}_{ig}={\overline{P}}_{ig}^T\in {\mathbb{R}}^{n\times n}$, $0<Z_{igh}=Z_{igh}^T\in {\mathbb{R}}^{n\times n}$, $W_i=W_i^T\in {\mathbb{R}}^{n\times n}$, and ${\overline{K}}_{is}\in {\mathbb{R}}^{m\times n}$, such that the following conditions hold for $i\in {\mathbb{N}}_N$, $g\in {\mathbb{N}}_{\alpha }$:

$$0>\left[\begin{array}{cccccc}-2W_i& (\ast )& (\ast )& (\ast )& (\ast )& (\ast )\\ (\mathbf{2},\mathbf{1})& (\mathbf{2},\mathbf{2})& (\ast )& (\ast )& (\ast )& (\ast )\\ 0& 0& -I& (\ast )& (\ast )& (\ast )\\ 0& {\displaystyle \sum _{s\in {\mathbb{N}}_{\beta }}}{\varpi }_{gs}{\overline{K}}_{is}^T{B}_{ig}^T& 0& -{\delta }^{-1}{W}_i& (\ast )& (\ast )\\ 0& 0& I& 0& -\delta W_i& (\ast )\\ W_i& 0& 0& 0& 0& -\mu {\overline{P}}_{ig},\end{array}\right],$$

(20)

$$0\le \left[\begin{array}{cc}{Z}_{igh}& {\overline{P}}_{ig}\\ (\ast )& {\overline{P}}_{ih}\end{array}\right],\forall h\in {\mathbb{N}}_{\alpha }\setminus \left\{g\right\},$$

(21)

where

$$\begin{array}{cc}\hfill & (\mathbf{2},\mathbf{1})=A_{ig}{W}_i+\sum _{s\in {\mathbb{N}}_{\beta }}{\varpi }_{gs}{B}_{ig}{\overline{K}}_{is}+{\overline{P}}_{ig},\hfill \\ \hfill & (\mathbf{2},\mathbf{2})=(ϵ+{\pi }_{gg}){\overline{P}}_{ig}+\sum _{h\in {\mathbb{N}}_{\alpha }\setminus \left\{g\right\}}{\pi }_{gh}{Z}_{igh}+B_{ig}{B}_{ig}^T-{\mu }^{-1}{\overline{P}}_{ig}.\hfill \end{array}$$

Then, system (1) achieves the cooperative output regulation, where the controller gain is constructed by $K_{is}={\overline{K}}_{is}{W}_i^{-1}$.

Proof. 

Let us consider the following mode-dependent Lyapunov function:

$${\mathcal{V}}_i(t,\varphi \left(t\right)=g)={\tilde{x}}_i^T\left(t\right)P_{ig}{\tilde{x}}_i\left(t\right),$$

(22)

where $P_{ig}=P_{ig}^T>0$. Then, applying the weak infinitesimal operator to $V_i(t,\varphi \left(t\right))$ leads to

$$\begin{array}{ll}\hfill \nabla {\mathcal{V}}_i\left(t\right)+ϵ{\mathcal{V}}_i\left(t\right)& =\underset{\delta \to 0}{lim}\frac{1}{\delta }\left(\mathbf{E}\left\{{\mathcal{V}}_i(t+\delta ,\varphi (t+\delta )=h|\varphi \left(t\right)=g)\right\}-{\mathcal{V}}_i(t,\varphi \left(t\right)=g)\right)+ϵ{\mathcal{V}}_i\left(t\right)\hfill \\ & ={\displaystyle \sum _{h\in {\mathcal{V}}_{\alpha }}}{\pi }_{gh}\mathcal{V}i(t,h)+\mathbf{E}\left\{{\dot{\mathcal{V}}}_i(t,g)\right\}+ϵ{\mathcal{V}}_i\left(t\right)\\ & =2{\tilde{x}}_i^T\left(t\right)P_{ig}{\displaystyle \sum _{s\in {\mathbb{N}}_{\beta }}}{\varpi }_{gs}\left(\left(A_{ig}+B_{ig}{K}_{is}\right){\tilde{x}}_i\left(t\right)-\left(B_{ig}{K}_{is}{\Lambda }_{ig}-B_{ig}{\Pi }_{ig}\right){\widehat{e}}_{i0}\left(t\right)\right)\\ & +{\tilde{x}}_i^T\left(t\right)\left({\displaystyle \sum _{h\in {\mathbb{N}}_{\alpha }}}{\pi }_{gh}{P}_{ih}\right){\tilde{x}}_i\left(t\right)+ϵ{\mathcal{V}}_i\left(t\right).\end{array}$$

Also, by Lemma 1, it is obtained that

$$\begin{array}{ll}\nabla {\mathcal{V}}_i\left(t\right)+ϵ{\mathcal{V}}_i\left(t\right)& \le {\tilde{x}}_i^T\left(t\right)\mathbf{He}\left\{P_{ig}\left(A_{ig}+B_{ig}{\mathbf{K}}_{ig}\right)\right\}{\tilde{x}}_i\left(t\right)+{\tilde{x}}_i^T\left(t\right)P_{ig}{B}_{ig}{\mathbf{K}}_{ig}{\mathbf{K}}_{ig}^T{B}_{ig}^T{P}_{ig}{\tilde{x}}_i\left(t\right)\hfill \\ & +{\widehat{e}}_{0i}^T\left(t\right){\Lambda }_{ig}^T{\Lambda }_{ig}{\widehat{e}}_{0i}\left(t\right)+{\tilde{x}}_i^T\left(t\right)P_{ig}{B}_{ig}{B}_{ig}^T{P}_{ig}{\tilde{x}}_i\left(t\right)+{\widehat{e}}_{0i}^T\left(t\right){\Pi }_{ig}^T{\Pi }_{ig}{\widehat{e}}_{0i}\left(t\right)\\ & +{\tilde{x}}_i^T\left(t\right)({\displaystyle \sum _{h\in {\mathbb{N}}_{\alpha }}}{\pi }_{gh}{P}_{ih}\mathrm{)}{\tilde{x}}_i\left(t\right)+ϵ{\tilde{x}}_i^T\left(t\right)P_{ig}{\tilde{x}}_i\left(t\right)\hfill \\ & \le {\tilde{x}}_i^T\left(t\right){\mathrm{Y}}_{ig}{\tilde{x}}_i\left(t\right)+\left(\left|\right|{\Lambda }_{ig}|{|}_2^2+|\left|{\Pi }_{ig}\right|{|}_2^2\right)\left|\right|{\widehat{e}}_{0i}\left(t\right){\left|\right|}_2^2,\hfill \\ & \end{array}$$

(23)

where ${\mathbf{K}}_{ig}={\sum }_{s\in {\mathbb{N}}_{\beta }}{\varpi }_{gs}{K}_{is}$, and

$$\begin{array}{cc}\hfill {\mathrm{Y}}_{ig}& =ϵP_{ig}+\sum _{h\in {\mathbb{N}}_{\alpha }}{\pi }_{gh}{P}_{ih}+\mathbf{He}\left\{P_{ig}\left(A_{ig}+B_{ig}{\mathbf{K}}_{ig}\right)\right\}+P_{ig}{B}_{ig}{B}_{ig}^T{P}_{ig}+P_{ig}{B}_{ig}{\mathbf{K}}_{ig}{\mathbf{K}}_{ig}^T{B}_{ig}^T{P}_{ig}.\hfill \end{array}$$

Thus, the condition ${\mathrm{Y}}_{ig}<0$ implies

$$\begin{array}{cc}\hfill & \nabla {\mathcal{V}}_i\left(t\right)<-ϵ{\mathcal{V}}_i\left(t\right)+\gamma \left|\right|{\widehat{e}}_{0i}\left(t\right){\left|\right|}_2^2,\hfill \end{array}$$

(24)

where $\gamma =\left|\right|{\Lambda }_{ig}|{|}_2^2+|\left|{\Pi }_{ig}\right|{|}_2^2$. Furthermore, using a generalization of Dynkin’s formula, it is obtained that

$$\begin{array}{c}\hfill \mathbf{E}\left\{{\mathcal{V}}_i\left(t\right)\right\}-{\mathcal{V}}_i\left(0\right)=\mathbf{E}\left\{{\int }_0^t{\nabla }_i\left(\tau \right)d\tau |{\tilde{x}}_i\left(0\right),\varphi \left(0\right),\rho \left(0\right)\right\}.\end{array}$$

Thus, it is valid that

$$\begin{array}{cc}\hfill \mathbf{E}\left\{{\mathcal{V}}_i\left(t\right)\right\}-{\mathcal{V}}_i\left(0\right)<& -\mathbf{E}\left\{{\int }_0^tϵ{\mathcal{V}}_i\left(\tau \right)d\tau |{\tilde{x}}_i\left(0\right),\varphi \left(0\right),\rho \left(0\right)\right\}\hfill \\ \hfill & +\gamma \mathbf{E}\left\{{\int }_0^t\left|\right|{\widehat{e}}_{0i}{\left(\tau \right)\left|\right|}_2^{2}d\tau |{\widehat{e}}_{0i}\left(0\right),\varphi \left(0\right)\right\}.\hfill \end{array}$$

(25)

Also, Theorem 1 ensures $\mathbf{E}\left\{{\int }_0^t\left|\right|{\widehat{e}}_{0i}{\left(\tau \right)\left|\right|}_2^{2}d\tau |{\widehat{e}}_{0i}\left(0\right),\varphi \left(0\right)\right\}<{\eta }_i<\infty$, where scalar ${\eta }_i$ stands for a finite constant. Accordingly, by considering a sufficiently small scalar $\vartheta >0$, such that ${ϵ}_i\left(\tau \right)\ge \vartheta \left|\right|{\tilde{x}}_i\left(\tau \right){\left|\right|}_2^2$, it follows from (25) that

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}\mathbf{E}\left\{{\int }_0^t\left|\right|{\tilde{x}}_i\left(\tau \right){\left|\right|}_2^{2}d\tau |{\tilde{x}}_i\left(0\right),\varphi \left(0\right),\rho \left(0\right)\right\}<{\vartheta }^{-1}\left({\mathcal{V}}_i\left(0\right)+\gamma {\eta }_i\right)<\infty ,\end{array}$$

(26)

which guarantees that the system (19) is stochastically stable according to Definition 1. As a result, since $e_i\left(t\right)=y_i\left(t\right)-y_0\left(t\right)=C_{ig}{\tilde{x}}_i\left(t\right)+C_{ig}{\Lambda }_{ig}{x}_0\left(t\right)-C_{0g}{x}_0\left(t\right)=C_{ig}{\tilde{x}}_i\left(t\right),$ condition (26) leads to

$$\begin{array}{ll}\hfill & \underset{t\to \infty }{lim}\mathbf{E}\left\{{\int }_0^t\left|\right|e_i\left(\tau \right){\left|\right|}_2^{2}d\tau |e_i\left(0\right),\varphi \left(0\right),\rho \left(0\right)\right\}\hfill \\ \hfill & =\underset{t\to \infty }{lim}\mathbf{E}\left\{{\int }_0^t\left|\right|C_{ig}{\tilde{x}}_i\left(\tau \right){\left|\right|}_2^{2}d\tau |{\tilde{x}}_i\left(0\right),\varphi \left(0\right),\rho \left(0\right)\right\}<\infty .\hfill \end{array}$$

(27)

Hence, based on Definition 2, it can be seen that condition ${\mathrm{Y}}_{ig}<0$ guarantees the stochastically cooperative output regulation of system (1). Meanwhile, condition ${\mathrm{Y}}_{ig}<0$ can be converted into

$$\begin{array}{cc}\hfill 0>(ϵ+{\pi }_{gg}){\overline{P}}_{ig}+{\overline{\mathbf{P}}}_{ig}& +\mathbf{He}\left\{\left(A_{ig}+B_{ig}{\mathbf{K}}_{ig}\right){\overline{P}}_{ig}\right\}+B_{ig}{B}_{ig}^T+B_{ig}{\mathbf{K}}_{ig}{\mathbf{K}}_{ig}^T{B}_{ig}^T,\hfill \end{array}$$

(28)

where ${\overline{P}}_{ig}=P_{ig}^{-1}$, and ${\overline{\mathbf{P}}}_{ig}={\sum }_{h\in {\mathbb{N}}_{\alpha }\setminus \left\{g\right\}}{\pi }_{gh}{\overline{P}}_{ig}{P}_{ih}{\overline{P}}_{ig}$. Furthermore, by the Schur complement, condition (21) ensures ${\overline{P}}_{ig}{P}_{ih}{\overline{P}}_{ig}\le Z_{igh}$ for $h\in {\mathbb{N}}_{\alpha }\setminus \left\{g\right\}$. Thus, based on (21), condition (28) holds if it is satisfied that

$$\begin{array}{c}0>(ϵ+{\pi }_{gg}){\overline{P}}_{ig}+\sum _{h\in {\mathbb{N}}_{\alpha }\setminus \left\{g\right\}}{\pi }_{gh}{Z}_{igh}+\mathbf{He}\left\{\left(A_{ig}+B_{ig}{\mathbf{K}}_{ig}\right){\overline{P}}_{ig}\right\}\\ +B_{ig}{B}_{ig}^T+B_{ig}{\mathbf{K}}_{ig}{\mathbf{K}}_{ig}^T{B}_{ig}^T.\end{array}$$

(29)

Also, applying the Schur complement to (29) yields

$$\begin{array}{c}0>\left[\begin{array}{cc}\mathbf{He}\left\{\right(A_{ig}+B_{ig}{\mathbf{K}}_{ig}\left){\overline{P}}_{ig}\right\}{+}_{ig}& (\ast )\\ 0& -I\end{array}\right]+\left[\begin{array}{c}{B}_{ig}{\mathbf{K}}_{ig}{W}_i\\ 0\end{array}\right][0\hspace{1em}{W}_i^{-1}]\\ \left[\begin{array}{c}0\\ W_i^{-1}\end{array}\right][W_i{\mathbf{K}}_{ig}^T{B}_{ig}^T\hspace{1em}0],\end{array}$$

(30)

where

$$\begin{array}{c}\hfill {\mathcal{X}}_{ig}=(ϵ+{\pi }_{gg}){\overline{P}}_{ig}+\sum _{h\in {\mathbb{N}}_{\alpha }\setminus \left\{g\right\}}{\pi }_{gh}{Z}_{igh}+B_{ig}{B}_{ig}^T.\end{array}$$

Accordingly, by Lemma 1, condition (30) holds if

$$\begin{gathered} 0>\left[\begin{array}{cc}\mathbf{He}\left\{\left(A_{ig}+B_{ig}{\mathbf{K}}_{ig}\right){\overline{P}}_{ig}\right\}+{\mathcal{X}}_{ig}& (\ast )\\ 0& -I\end{array}\right]+{\delta }^{-1}\left[\begin{array}{c}0\\ W_i^{-1}\end{array}\right]W_i[0 W_i^{-1}] \\ +\delta \left[\begin{array}{c}{B}_{ig}{\mathbf{K}}_{ig}{W}_i\\ 0\end{array}\right]W_i^{-1}[W_i{\mathbf{K}}_{ig}^T{B}_{ig}^T 0], \end{gathered}$$

(31)

which is converted by the Schur complement as follows:

$$0>\left[\begin{array}{cccc}\mathbf{He}\left\{\left(A_{ig}+B_{ig}{\mathbf{K}}_{ig}\right){\overline{P}}_{ig}\right\}+{\mathcal{X}}_{ig}& (\ast )& (\ast )& (\ast )\\ 0& -I& (\ast )& (\ast )\\ {\displaystyle \sum _{s\in {\mathbb{N}}_{\beta }}}{\varpi }_{gs}{\overline{K}}_{is}^T{B}_{ig}^T& 0& -{\delta }^{-1}{W}_i& (\ast )\\ 0& I& 0& -\delta W_i.\end{array}\right].$$

(32)

Indeed, performing the congruent transformation to (20) by

$$S=\left[\begin{array}{cccccc}{A}_{ig}+B_{ig}{\mathbf{K}}_{ig}& I& 0& 0& 0& 0\\ 0& 0& I& 0& 0& 0\\ 0& 0& 0& I& 0& 0\\ I& 0& 0& 0& 0& I\end{array}\right],$$

we can obtain

$$0>\left[\begin{array}{ccccc}{\mathcal{Q}}_{ig}& (\ast )& (\ast )& (\ast )& (\ast )\\ 0& -I& (\ast )& (\ast )& (\ast )\\ {\displaystyle \sum _{s\in {\mathbb{N}}_{\beta }}{\varpi }_{gs}{\overline{K}}_{is}^T{B}_{ig}^T}& 0& -{\delta }^{-1}{W}_i& (\ast )& (\ast )\\ 0& I& 0& -\delta W_i& (\ast )\\ {\overline{P}}_{ig}& 0& 0& 0& -\mu {\overline{P}}_{ig},\end{array}\right],$$

(33)

where ${\overline{K}}_{is}=K_{is}{W}_i$, and

$$\begin{array}{cc}\hfill {\mathcal{Q}}_{ig}& =\mathbf{He}\left\{\left(A_{ig}+B_{ig}{\mathbf{K}}_{ig}\right){\overline{P}}_{ig}\right\}+(ϵ+{\pi }_{gg}){\overline{P}}_{ig}+{\displaystyle \sum _{h\in {\mathbb{N}}_{\alpha }\setminus \left\{g\right\}}}{\pi }_{gh}{Z}_{igh}+B_{ig}{B}_{ig}^T-{\mu }^{-1}{\overline{P}}_{ig}.\hfill \end{array}$$

That is, since (33) is equivalent to (32) by the Schur complement, it can be seen that (20) and (21) imply (28), i.e, ${\mathrm{Y}}_{ig}<0$. □

Remark 8. 

In the synchronous case ($\varphi \left(t\right)\equiv \rho \left(t\right)=g$), the bilinear problems (the third and fifth terms on the right-hand side) of condition (29) could be addressed using the conventional variable replacement technique by denoting ${\tilde{K}}_{ig}=K_{ig}{\overline{P}}_{ig}$, and then applying the Schur complement method to derive the final LMI conditions. However, because of the difference between the system and controller modes in the asynchronous case ($\varphi \left(t\right)=g\ne \rho \left(t\right)=s$), it is impossible to use the aforementioned approach. Hence, additional numerical processes and equivalent LMI conditions are introduced here to handle the coupling problems in condition (29).

Remark 9. 

The most recent studies concentrated on addressing the cooperative output regulation problem of heterogeneous Markov jump MASs, where both system and controller modes operate synchronously [23,34], or on deterministic system parameters with a switching network topology [20,21]. Meanwhile, the proposed control strategy in this work can, overall, cover these two problems. Furthermore, we make a first attempt to reflect the influence of the asynchronous mode between the continuous-time heterogeneous MASs and observer-based distributed controllers on achieving stochastically cooperative output regulation subject to Markov jumps. Thus, it is hard to draw a direct comparison since the lack of comparable results developed in a similar framework to this study.

4. Illustrative Examples

To show the validity of the proposed method, this paper provides two examples.

Example 1. 

Let us consider the following heterogeneous multi-agent system with $g\in {\mathbb{N}}_2$ and $i\in {\mathbb{N}}_4$, adopted in [23]:

$$\begin{array}{l}{A}_{11}=\left[\begin{array}{cc}-0.3& 1.4\\ -1.1& -0.3\end{array}\right],A_{21}=\left[\begin{array}{cc}2& 1\\ 1& 0\end{array}\right],A_{31}=\left[\begin{array}{cc}2& 1\\ 1& 0\end{array}\right],\\ A_{41}=\left[\begin{array}{cc}-0.2& 1.5\\ -1.2& -0.3\end{array}\right],A_{12}=\left[\begin{array}{cc}0& 0.9\\ 0.5& 0.3\end{array}\right],A_{22}=\left[\begin{array}{cc}-2& 1\\ 1& -1\end{array}\right],\\ A_{32}=\left[\begin{array}{cc}-2& 1\\ 1& -1\end{array}\right],A_{42}=\left[\begin{array}{cc}0.1& 1\\ 0.6& 0.4\end{array}\right],B_{ig}=\left[\begin{array}{c}1\\ 1\end{array}\right],\\ D_{11}=\left[\begin{array}{cc}0.4& 0.1\\ 1.1& 0.5\end{array}\right],D_{21}=\left[\begin{array}{cc}-1.9& 0.5\\ -1& 0.2\end{array}\right],D_{31}=\left[\begin{array}{cc}-1.9& 0.5\\ -1& 0.2\end{array}\right],\\ D_{41}=\left[\begin{array}{cc}0.3& 0\\ 1.2& 0.5\end{array}\right],D_{12}=\left[\begin{array}{cc}0.1& 0.6\\ -0.5& -0.1\end{array}\right],D_{22}=\left[\begin{array}{cc}2.1& 0.5\\ -1& 1.2\end{array}\right],\\ D_{32}=\left[\begin{array}{cc}2.1& 0.5\\ -1& 1.2\end{array}\right],D_{42}=\left[\begin{array}{cc}0& 0.5\\ -0.6& -0.2\end{array}\right],C_{ig}=\left[\begin{array}{cc}1& 0.1\end{array}\right],\end{array}$$

(34)

$$A_{01}=\left[\begin{array}{cc}0.1& 1.5\\ 0& 0.2\end{array}\right],A_{02}=\left[\begin{array}{cc}0.1& 1.4\\ 0& 0.1\end{array}\right],C_{0g}=\left[\begin{array}{cc}1& 0.1\end{array}\right].$$

(35)

In addition, the used network topology is depicted in Figure 2 (each agent is a numbered circle), which can be characterized as follows:

$${\mathcal{L}}_1=\left[\begin{array}{cccc}3& -3& 0& 0\\ 0& 2& 0& -2\\ -1& 0& 1& 0\\ 0& -3& 0& 3\end{array}\right],{\mathcal{L}}_2=\left[\begin{array}{cccc}1& 0& 0& -1\\ -3& 3& 0& 0\\ 0& -3& 3& 0\\ 0& -2& 0& 2\end{array}\right],$$

$${\mathcal{M}}_1=\mathbf{diag}(0,2,0,0), {\mathcal{M}}_2=\mathbf{diag}(3,0,0,0).$$

Also, based on Assumption 2, ${\Lambda }_{ig}=I_2,{\Pi }_{i1}=0,{\Pi }_{i2}=[0 -0.1].$ Furthermore, to handle two synchronous and asynchronous cases between the system and controller modes, we consider the following transition rate ${\pi }_{gh}$ and conditional probability ${\varpi }_{gs}$:

$${\left[{\pi }_{gh}\right]}_{g,h\in {\mathbb{N}}_2}=\left[\begin{array}{cc}-0.5& 0.5\\ 0.7& 0.7\end{array}\right],$$

(36)

$$\mathbf{Case}\mathbf{1}(\mathrm{Synchronous} \mathrm{case}):{\left[{\varpi }_{gs}\right]}_{g\in {\mathbb{N}}_2,s\in {\mathbb{N}}_2}=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],$$

(37)

$$\mathbf{Case}\mathbf{2}(\mathrm{Aynchronous} \mathrm{case}):{\left[{\varpi }_{gs}\right]}_{g\in {\mathbb{N}}_2,s\in {\mathbb{N}}_2}=\left[\begin{array}{cc}0.7& 0.3\\ 0.5& 0.5\end{array}\right].$$

(38)

Thereupon, for $\mu =0.05$, $\delta =0.05$, and $ϵ=0.1$, Theorems 1 and 2 provide the following leader–state observer gain $F_g$ and controller gain $K_{is}$:

$$F_1=\left[\begin{array}{cc}-1.8266& -0.5765\\ -0.5059& -0.1793\end{array}\right],F_2=\left[\begin{array}{cc}-1.0958& -0.4640\\ -0.3766& -0.2335\end{array}\right],$$

(39)

Case 1 (Synchronous case):

$$\begin{array}{l}{K}_{11}=[-7.1527 -6.1036], K_{21}=[-10.5399 -5.1965], \\ K_{31}=[-10.5399 -5.1965], K_{41}=[-7.3693 -6.0404], \\ K_{12}=[-8.0258 -6.1664], K_{22}=[-8.3219 -4.9543], \\ K_{32}=[-8.3219 -4.9543], K_{42}=[-8.3020 -6.1352],\end{array}$$

(40)

Case 2 (Asynchronous case):

$$\begin{gathered} \begin{array}{l}{K}_{11}=[-5.8430 -6.0093], K_{21}=[-13.8669 -5.5598], \\ {K}_{31}=[-13.8669 -5.5598], K_{41}=[-5.9701 -5.8983], \\ {K}_{12}=[-10.2085 -6.3236], K_{22}=[-2.7768 -4.3487], \\ {K}_{32}=[-2.7768 -4.3487],K_{42}=[-10.6340 -6.3720].\end{array} \end{gathered}$$

(41)

The details of the hardware used in this experiment are provided in

Table 1. System specifications.

Hardware Resources	Information
Operating System	Microsoft Windows 10 Pro
RAM	8 GB
Processor	3.20 GHz
Hard Drive	120 GB SSD

and the experiment is conducted using MATLAB for both cases. Based on the central limit theorem, the expected computation times are nearly 0.0363 s for the synchronous case and 0.0394 s for the asynchronous case (after 10,000 trials). Although there is a slight difference, the computation times to derive the final results are approximately similar for both cases overall.

Let us consider the following initial conditions: $x_0\left(0\right)={[0.1,0.1]}^T=x_i\left(0\right)$, ${\widehat{x}}_{01}\left(0\right)={[0.2,0.2]}^T$, ${\widehat{x}}_{02}\left(0\right)={[0.3,0.2]}^T$, ${\widehat{x}}_{03}\left(0\right)={[0.2,-0.3]}^T$, and ${\widehat{x}}_{04}\left(0\right)={[-0.1,0.2]}^T$. As shown in Figure 3, suppose that the hidden mode (also called the system mode) and the observed mode (also called the control mode) are generated according to (36)–(38). Then, based on (39), Figure 4 shows the leader–state estimation error ${\widehat{e}}_{0i}={\widehat{x}}_{0i}-x_0$, where ${\widehat{e}}_{0i}={\left[{\widehat{e}}_{0i1}{\widehat{e}}_{0i2}\right]}^T$. That is, from Figure 4, it can be seen that ${\widehat{x}}_{0i}$ steadily approaches $x_0$ as time increases, which reveals that the observer (8) with (39) can accurately estimate the leader–state despite abrupt changes in the system mode (related to the network topology mode). Next, based on (40), Figure 5a shows the leader and agent outputs for the synchronous case, which demonstrates the cooperative output regulations to ensure that all agent outputs (see the solid lines) follow the leader’s output (see the green-dotted line). Furthermore, Figure 6a shows the output error $e_i=y_i-y_0$, clearly illustrating how the agent output approaches the leader’s output from the given initial conditions. Meanwhile, based on (41), Figure 5b shows the leader and agent outputs for the asynchronous case, which illustrates that all agent outputs follow the leader’s output as time increases, despite the emergence of Markov switching and the asynchronous phenomenon. Figure 6b shows that the output error $e_i$ steadily converges to zero as time increases; hence, it validates the results presented in Figure 5b. Eventually, from Figure 5a,b, it can be seen that the proposed method can be successfully used to achieve the cooperative output regulation for heterogeneous multi-agent systems, with hidden Markov jumps containing both synchronous and asynchronous cases.

Example 2 

(Practical application). Consider the following double integrator dynamics driven by different types of actuators, adopted in [36], for $g\in {\mathbb{N}}_2$, and $i\in {\mathbb{N}}_4$:

$$A_{ig}=\left[\begin{array}{ccc}0& 1& 0\\ 0& 0& c_i\\ 0& -d_{ig}& -a_i\end{array}\right],B_{ig}=\left[\begin{array}{c}0\\ 0\\ b_i\end{array}\right],C_{ig}=\left[\begin{array}{ccc}1& 0& 0\end{array}\right],D_i=0,$$

(42)

where the system state $x_i={\left[x_{1i},x_{2i},x_{3i}\right]}^T$ consists of the position $x_{1i}$, the velocity $x_{2i}$, and the actuator state $x_{3i}$; $a_i>0$ denotes the speed of the actuator; $b_i$ and $c_i>0$ are gains; and $d_{ig}\ge 0$ represents the influence of velocity on the actuator. Specifically, the value of $d_{ig}$ is changed according to the Markov process of system mode $g\in {\mathbb{N}}_2$, which is affected by the environment on the actuator, or internal factors, such as temperature, humidity, and lifespan.

If $d_{ig}>0$, the actuator is influenced by the velocity, otherwise, $d_{ig}=0$ (refer to [36] for more details). In this paper, the system parameters are set to $a_1=1$, $a_2=10$, $a_3=a_4=2$, $b_1=b_3=b_4=1$, $b_2=2$, $c_i=1$, $d_{11}=d_{21}=d_{42}=0$, $d_{41}=d_{12}=d_{22}=1$, $d_{31}=10$, and $d_{32}=5$. Furthermore, to synchronize the position and velocity of agents, the leader is described according to [36] as follows:

$$A_{0g}=\left[\begin{array}{cc}0& 1\\ 0& 0\end{array}\right],C_{0g}=\left[\begin{array}{cc}1& 0\end{array}\right].$$

(43)

In addition, the used network topology is depicted in Figure 7 (each agent is a numbered circle), which can be characterized as follows:

$${\mathcal{L}}_1=\left[\begin{array}{cccc}1& 0& 0& -1\\ -1& 1& 0& 0\\ 0& -1& 2& -1\\ -1& 0& -1& 2\end{array}\right],{\mathcal{L}}_2=\left[\begin{array}{cccc}2& -1& 0& -1\\ 0& 1& -1& 0\\ 0& -1& 1& 0\\ -1& 0& 0& 1\end{array}\right],$$

$${\mathcal{M}}_1=\mathbf{diag}(1,0,0,0), {\mathcal{M}}_2=\mathbf{diag}(0,1,0,0).$$

Also, based on Assumption 2, we can see that

$${\Lambda }_{ig}=\left[\begin{array}{cc}1& 0\\ 0& 1\\ 0& 0\end{array}\right], {\Pi }_{ig}=[0 \frac{d_{ig}}{b_i}].$$

Furthermore, to handle the two synchronous and asynchronous cases between the system and controller modes, we consider the following transition rate ${\pi }_{gh}$ and conditional probability ${\varpi }_{gs}$:

$${\left[{\pi }_{gh}\right]}_{g,h\in {\mathbb{N}}_2}=\left[\begin{array}{cc}-0.6& 0.6\\ 0.4& -0.4\end{array}\right],$$

(44)

$$\mathbf{Case}\mathbf{1}(\mathrm{Synchronous} \mathrm{case}):{\left[{\varpi }_{gs}\right]}_{g\in {\mathbb{N}}_2,s\in {\mathbb{N}}_2}=\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right],$$

(45)

$$\mathbf{Case}\mathbf{2}(\mathrm{Asynchronous} \mathrm{case}):{\left[{\varpi }_{gs}\right]}_{g\in {\mathbb{N}}_2,s\in {\mathbb{N}}_2}=\left[\begin{array}{cc}0.2& 0.8\\ 0.6& 0.4\end{array}\right].$$

(46)

Thereupon, for $\mu =0.05$, $\delta =0.05$, and $ϵ=0.1$, Theorems 1 and 2 provide the following observer gain $F_g$ and controller gain $K_{is}$:

$$F_1=\left[\begin{array}{cc}-0.8526& -0.4293\\ -0.3945& -0.4292\end{array}\right],F_2=\left[\begin{array}{cc}-0.9814& -0.5368\\ -0.5160& -0.4497\end{array}\right],$$

(47)

Case 1 (Synchronous case):

$$\begin{array}{l}{K}_{11}=[-6.8769 -13.0047 -14.1589], K_{21}=[-3.4643 -6.5518 -2.6852],\hfill \\ K_{31}=[-6.8790 -3.0301 -13.1795], K_{41}=[-6.8890 -12.0268 -13.1877],\\ K_{12}=[-6.7689 -11.8254 -13.9587], K_{22}=[-3.3989 -5.9427 -2.5656], \\ K_{32}=[-6.7734 -7.8439 -12.9812], K_{42}=[-6.7824 -12.8468 -12.9897],\end{array}$$

(48)

Case 2 (Asynchronous case):

$$\begin{array}{l}{K}_{11}=[-6.6609 -10.6460 -13.7584], K_{21}=[-3.3334 -5.3336 -2.4461],\\ K_{31}=[-6.6678 -12.6577 -12.7828], K_{41}=[-6.6759 -13.6669 -12.7918],\\ K_{12}=[-6.9308 -13.5943 -14.2590], K_{22}=[-3.4971 -6.8563 -2.7450], \\ K_{32}=[-6.9318 -0.6321 -13.2787],K_{42}=[-6.9422 -11.6168 -13.2866].\end{array}$$

(49)

Let us consider the following initial conditions: $x_0\left(0\right)={[0.2,0.1]}^T$, $x_i\left(0\right)={[0.1,0.1,0.2]}^T$ ($\forall i\in {\mathbb{N}}_4$), ${\widehat{x}}_{01}\left(0\right)={[0.2,0.3]}^T$, ${\widehat{x}}_{02}\left(0\right)={[0.3,0.2]}^T$, ${\widehat{x}}_{03}\left(0\right)={[0.1,-0.1]}^T$, and ${\widehat{x}}_{04}\left(0\right)={[-0.3,0.2]}^T$. And, as shown in Figure 8, suppose that the hidden mode (also called the system mode) and the observed mode (also called the control mode) are generated according to (44)–(46). Then, based on (47), Figure 9 shows the leader–state estimation error ${\widehat{e}}_{0i}={\widehat{x}}_{0i}-x_0$, where ${\widehat{e}}_{0i}={\left[{\widehat{e}}_{0i1}{\widehat{e}}_{0i2}\right]}^T$. That is, as shown in Figure 9, ${\widehat{x}}_{0i}$ steadily approach $x_0$ as time increases, which reveals that the observer (8) with (47) can accurately estimate the leader–state regardless of the abrupt changes in the system (42). Subsequently, Figure 10a shows the leader and agent outputs for the synchronous case, which verifies that (48) achieves the cooperative output regulation of (42) since all the agent outputs (see the solid lines) follow the leader’s output (see the green-dotted line). Meanwhile, based on (49), Figure 10b shows the leader and agent outputs for the asynchronous case; this also illustrates that all agent outputs follow the leader’s output as time increases despite the emergence of the Markov switching and the asynchronous phenomenon. Eventually, from Figure 10a,b, it can be seen that the proposed method can be effectively used to realize the cooperative output regulation for (42) with hidden Markov jumps containing both synchronous and asynchronous cases.

5. Concluding Remarks

In this paper, we investigated the stochastic cooperative output regulation problem of heterogeneous multi-agent systems subject to hidden Markov jumps. In particular, when dealing with this problem, we also considered a time-varying network topology that changes according to the system operation mode. First, a leader–state observer was designed using a mode-dependent Lyapunov function to ensure that all agents can accurately estimate the leader–state. Then, an asynchronous mode-dependent distributed controller was designed to ensure the stochastic cooperative output regulation for heterogeneous multi-agent systems with hidden Markov jumps. In addition, recent studies [37,38,39] motivated us to extend the proposed strategy to cover more practical control problems, such as stochastic time delay, input saturation, and unknown system dynamics in the continuous-time (discrete-time) domain for a wider range of applications.