Next Article in Journal
Degeneration-Aware Localization with Arbitrary Global-Local Sensor Fusion
Next Article in Special Issue
A Combined Model and Data-Driven Approach for the Determination of Rotor Temperature in an Induction Machine
Previous Article in Journal
Vision-Based Tunnel Lining Health Monitoring via Bi-Temporal Image Comparison and Decision-Level Fusion of Change Maps
Previous Article in Special Issue
Multivariable Optimisation for Waiting-Time Minimisation at Roundabout Intersections in a Cyber-Physical Framework
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

A Coalitional Distributed Model Predictive Control Perspective for a Cyber-Physical Multi-Agent Application

Anca Maxim
Constantin-Florin Caruntu
Department of Automatic Control and Applied Informatics, “Gheorghe Asachi” Technical University of Iasi, 700050 Iasi, Romania
Author to whom correspondence should be addressed.
Sensors 2021, 21(12), 4041;
Submission received: 6 May 2021 / Revised: 3 June 2021 / Accepted: 9 June 2021 / Published: 11 June 2021
(This article belongs to the Special Issue Cyber-Physical Systems - from Perception to Action)


Following the current technological development and informational advancement, more and more physical systems have become interconnected and linked via communication networks. The objective of this work is the development of a Coalitional Distributed Model Predictive Control (C- DMPC) strategy suitable for controlling cyber-physical, multi-agent systems. The motivation behind this endeavour is to design a novel algorithm with a flexible control architecture by combining the advantages of classical DMPC with Coalitional MPC. The simulation results were achieved using a test scenario composed of four dynamically coupled sub-systems, connected through an unidirectional communication topology. The obtained results illustrate that, when the feasibility of the local optimization problem is lost, forming a coalition between neighbouring agents solves this shortcoming and maintains the functionality of the entire system. These findings successfully prove the efficiency and performance of the proposed coalitional DMPC method.

1. Introduction

Presently, manifold systems are modular, interconnected and have a cyber-physical setup, meaning they can be viewed as coupled physical sub-systems, which are connected via communication networks [1,2,3,4,5]. For such processes, Distributed Model Predictive Control (DMPC) is a reliable control solution that uses local controllers that compute the control action using both (i) the local information derived from specific sensors and (ii) coupling data received/transmitted using the communication network [6].
As recent studies attest, the DMPC strategy was successfully applied on multi-agent systems in varying applications, such as formation control of autonomous surface and aerial vehicles [7], leader–follower platoons [8,9], traffic signal control [10], temperature regulation systems [11], battery energy storage systems [12] and microgrids [13,14]. In [15], a DMPC strategy for multi-agent systems based on error upper bounds is provided. This criterion is used in a min–max optimization of the cost function to minimize the communication between neighbouring agents. An event-triggered synchronous DMPC for multi-agent systems is introduced in [16]. The method is tailored for dynamically decoupled sub-systems, coupled through a cost function. An event-triggered mechanism designed using the forward difference of the cost function is deployed to activate the local optimization problem at each sampling time; otherwise the agents use the solutions computed in the previous sampling period. In [17], a DMPC to reach consensus for time-varying, multi-agent systems is proposed. The consensus DMPC algorithm is designed for heterogeneous, time-varying decoupled sub-systems, connected uni-directionally with a coupled cost function.
In all the research mentioned above, regardless of the application or the methodology details, one key feature is noticeable, namely that the architecture of both sub-systems and agents (i.e., local controllers) is fixed. The latter is predefined in the initialization phase of the control design, based on the sensors placements and interconnection between the local sub-systems. Therefore, the configuration of the DMPC neighbourhoods (i.e., groups of local sub-systems that are interconnected either dynamically or through cost functions or constraints) is established and predefined [18].
To overcome this shortcoming, a new approach emerged from cooperative game theory framework named Coalitional Control was introduced, with the following characteristics [19,20]: (i) the topology of the communication links between agents is flexible (i.e., links can be enabled or disabled when necessary), (ii) the control strategy encourages the agents to group in cooperative clusters called coalitions (to reduce the communication burden), and (iii) the communication links between agents that ensure their cooperation are weighted and introduce supplementary costs in the cost functions when activated.
Using this foundation, a Coalitional Model Predictive Control (C-MPC) strategy was developed and applied on different applications such as cellular networks in [21,22] or an eight-coupled tank process [23]. Thus, the agents charged with controlling the local sub-systems can form coalitions depending on the activation of the communication links between them. Several topologies can be derived, starting from the default one, which is the decentralized MPC strategy (i.e., with no communication between agents), to the most complex one. described as centralized MPC (i.e., in which all the communication links are active). In between, there are the coalitions between several agents in a neighbourhood, while the remaining ones work independently. In [24], details regarding the feasible regions for tube-based MPC controllers are discussed. The coalitions are associated with different partitions of a large-scale system (i.e., several sub-systems can be joined in a single entity), and their feasible region is analysed.
The main contribution of this work is the development of a novel perspective of the DMPC algorithm, called Coalitional DMPC (C-DMPC), which combines both the advantages and features of classical DMPC strategy with the characteristics of Coalitional MPC. Hence, the envisioned solution is to dynamically reshape the controller network by merging some of the agents within a neighbourhood into coalitions when needed. The advantages of this approach are shown when, due to various reasons (e.g., reference changes, unknown disturbances, etc.), one or more local optimal solutions become infeasible. In this case, to maintain the feasibility and functionality of the interconnected cyber-physical multi-agent system, the agents will decide to form a coalition. This means that inside a coalition, the sub-systems become a single entity and the controllers aggregate and solve a cooperative optimization problem (i.e., a global cost function is minimized) [25].
The main difference of our approach with respect to the cited coalitional literature is that the default topology is a non-cooperative DMPC (i.e., each agent minimizes a local cost function, using received information from its neighbours) [26]. This means that when a coalition occurs, the remaining agents outside the coalitions are not independent but retain their previous status and solve a non-cooperative optimization problem. Thus, depending on the topology, it is possible that the coalition must exchange information with its neighbours (if not all the agents inside a neighbourhood are merged into the coalition). Since all the agents start as non-cooperative players, they use the communication network to share relevant data, according to their coupling within neighbourhoods, and all pertinent communication links are activated and not weighted.
Another key difference is our proposed merging procedure, which selects the agents that will form a coalition. This is done at each agent level, without using a hierarchical supervisory layer. Moreover, when the local optimization problem becomes unfeasible (due to the coupling information), the coalition is activated. Furthermore, two simplified versions of this method with different agent merging procedures were published in [27,28]. In [27], each agent considers that the coupling information received from the neighboring agents is an uncertainty in the local nominal model. When a predefined threshold value for the local uncertainty level is crossed, than a coalition between the agents is formed. The further development of this idea is given in [28], in which the coalition between the agents is formed, when the local optimization problems become infeasible due to the received uncertainty level.
With respect to our previous papers, the method proposed in the current work has significant improvements, such as the following: (i) the network topology is tailored for in-chain coupled sub-systems, with unidirectional communication links; (ii) a more realistic academic example is used for simulation tests, with four heterogeneous sub-systems dynamically coupled through the inputs; (iii) each sub-system model is augmented with an additional state defined as the integral of control error to ensure a non-zero reference tracking; and (iv) multiple coalitions between agents can be simultaneously active at each sampling time.
The remaining of this paper is structured as follows: Section 2 presents the problem formulation and details the proposed method, whereas the simulation configuration, results, and discussions are provided in Section 3. The conclusions of this work and future work plans are addressed in Section 4.

2. Problem Formulation

A cyber-physical multi-agent system (CP-MAS), as depicted in Figure 1, is composed of N interconnected cyber-physical sub-systems (CPsS). Each CPsS is defined by the pair ( S i , A i ), i N , where N denotes the set { 1 , , N } N , with N N the number of sub-systems and N the set of natural numbers. The physical part of the CPsS is denoted with S i , whereas the cyber part of the CPsS is denoted with A i and represents the corresponding local controller or agent. All the interconnected sub-systems S i form the physical layer (depicted with grey colour), while the cyber layer (depicted with blue colour) is composed of all the agents and the communication networks.
Let each sub-system S i be defined by the following model:
x p i ( k + 1 ) = A p i , i x p i ( k ) + B p i , i u p i ( k ) + w p i ( k ) w p i ( k ) = B p i , i 1 u p i 1 ( k ) y p i ( k ) = C p i x p i ( k ) , i N
with the notations for the state x p i R n i , input u p i R m i , input uncertainty w p i R p i and output y p i R q i . A p i , i , B p i , i , B p i , i 1 and C p i are matrices with adequate dimensions. n i , m i , p i and q i are the number of states, inputs, input uncertainties and outputs, respectively. R denotes the set of real numbers. Note that, u p i 1 R m i denotes the input signal received from the predecessor sub-system with index i 1 .
Note that (1) defines a model in which the input-coupling information w p i is considered an uncertainty in the nominal model. Moreover, all the sub-systems S i , i N , are in a chain architecture, and for sub-system indexed i, the information is received through an unidirectional link from its predecessor and neighbour, defined as the sub-system with index i 1 .
To ensure that the reference tracking control problem has a zero error in stationary regime, the state vector x p i , i N , from (1) is extended with an additional state x ¯ p i defined as integral of the control error, using the following definition [29,30]:
x ¯ p i ( k + 1 ) = x ¯ p i ( k ) C p i x p i ( k ) + r i ( k ) ,
x p i ( k + 1 ) x ¯ p i ( k + 1 ) x i ( k + 1 ) = A p i , i O C p i I A i , i x p i ( k ) x ¯ p i ( k ) x i ( k ) + B p i , i O B i , i u i ( k ) + B p i , i 1 O B i , i 1 u i 1 ( k ) + O I R sp i r i ( k ) y i ( k ) = C p i O C i x p i ( k ) x ¯ p i ( k ) x i ( k )
where r i ( k ) is the imposed reference value at time k. x i ( k ) , u i ( k ) and y i ( k ) are the extended state, input and output vectors, respectively. Note that the input uncertainty w i ( k ) is defined based on the input vector received from the predecessor u i 1 ( k ) . I and O are the identity and zero matrix, respectively, each with appropriate dimensions.
Hereafter, each sub-system S i , i N , will be represented by the compact extended model:
x i ( k + 1 ) = A i , i x i ( k ) + B i , i u i ( k ) + w i ( k ) + R sp i r i ( k ) w i ( k ) = B i , i 1 u i 1 ( k ) y i ( k ) = C i x i ( k )
where A i , i , B i , i , B i , i 1 , C i and R sp i are matrices with adequate dimensions.
Consider linear inequality constraints for the outputs, inputs and uncertainties defined with:
y i Y i , u i U i , w i W i , i N
where Y i , U i and W i are sets defined by linear inequalities.
At every sampling time, each agent A i , i N , solves a min–max optimization problem, which aims to obtain the minimum optimal input with respect to the maximum level of uncertainty received from its neighbour.
J i ( x i 0 ) = min u i 0 , , u i N p 1 u ^ i max max w i 0 , , w i N p 1 R w i u ^ i max 1 + l = 0 N p 1 r i l y i l 1 + R u i u i l 1 + r i N p y i N p 1 s . t . ( 4 ) y i l Y i , l = 1 , , N p 1 y i N p Ω i u i l u ^ i max u i max , l = 0 , , N p 1 w i l w i max
where y i l = y i ( k + l | k ) denotes the output predictions for sub-system S i at time k + l , computed at time step k; this is calculated recursively starting from the initial state x i 0 = x i ( k ) measured at time k , using the model (4); the input sequence u i l = u i 0 , , u i N p 1 computed over the prediction horizon N p ; and the uncertainty sequence W i = B i , i 1 U i 1 received from the neighbour (where W i is the uncertainty polytope and U i 1 = { u R m i : A u b is a H-polytope); r i l is the value at time k + l for the output reference trajectory; r i N p and y i N p are the values for the reference and the output trajectories, at the end of the prediction horizon k + N p , respectively; u i max , w i max are the maximum limits for the input and the uncertainty sequences, respectively; . 1 denotes the 1-norm; R u i R m i × m i and R w i R m i × m i are the weight matrices for the input and self-imposed input limit u ^ i max . The latter is an additional optimization parameter introduced in the local cost function, and its value is communicated at each sampling period to the neighbour. This will guarantee that the uncertainty level received from the neighbour is smaller than this value, without actually transmitting the entire input sequence. The set Ω i is a robust positive invariant set used to ensure the closed-loop stability of the algorithm by means of the terminal invariant set.
Remark 1.
The uncertainty in each sub-system model refers to the coupling information that must be received from the neighbouring sub-system. Please note that the local optimization problem (6) minimizes the control input, for the worst-case scenario related to uncertainty level received from the predecessor agent. This means that, although unknown, this uncertainty must be bounded to a known value, which is shared between consecutive sub-system. Moreover, this ensures that each local sub-system is prepared for the disturbance signal, which is received via the coupling links.
Next, some details regarding the computation of the invariant set Ω i , followed by the proposed coalitional DMPC method are given.

2.1. Robust Positive Invariant Set Computation

In this sub-section, the details regarding the computation of the robust positive invariant set Ω i , i N , which acts as a constraint region for the terminal state y i N p Ω i are presented. To this end, the procedure firstly introduced in [28] is briefly summarized below, tailored for the extended sub-system model.
For each sub-system S i , i N , with the model defined in (4) and subject to constraints (5), only the nominal model (i.e., w i and r i are zero) is considered. Let us compute a local linear feedback u i = K i x i , which ensures that the closed loop eigenvalues are in the unit circle. One suggestion to compute the state feedback matrix K i is to apply classical state-space feedback control designed for the nominal model using Ackermann’s formula (i.e., solving a pole allocation problem) [31], or to calculate it through the minimization of a linear-quadratic cost function, by solving a discrete-time Riccatti Equation [32].
The set Ω i is robust positive invariant for the nominal model from (4), if the following assumption holds [28,33]:
x i Ω i ( A i , i + B i , i K i ) x i + w i Ω i , K i x i U i , C i Ω i Y i , w i W i
It is worth mentioning the following observations regarding the use of the invariant set in the C-DMPC context:
  • the default working framework is non-cooperative DMPC, which implies that each agent A i , i N , from the multi-agent application communicates with its neighbour, in order to compute the local solution;
  • each sub-system model S i , i N , is subject to input uncertainties received from the sub-system to whom it is connected (in our case its predecessor);
  • to provide a simplified algorithm with minimal communication load in the network, only the self-imposed upper bound for the local input trajectory is broadcast in the network (i.e., the optimization variable u ^ max introduced in (6));
  • a table with different predefined robust positive invariant sets Ω i is computed using the constraints limits from (5), in which each element is a particular combination of the variable bounds (see Algorithm 1);
  • at each sampling period, after the uncertainty upper bound is received from the neighbour, each agent A i uses this information to compute the uncertainty polytope. Next, from the predefined terminal sets table, a set Ω i is searched for, which includes the received uncertainty polytope (i.e., which will ensure a local feasible solution in the terminal state framework).
Further on, the pseudo-code algorithm used to compute the invariant set table is provided (where for simplicity the sub-system indices are omitted):
Thus, each agent A i , i N , uses Algorithm 1 in the initialization phase of the proposed method to compute a table of invariant sets Ω i , for different input and uncertainty parametrizations (i.e., distinct combinations for the two parameters α and β ). Note that the first set Ω i from the table corresponds to the largest value for the input constraint, denoted u m a x , whereas the uncertainty has the smallest value. The latter is gradually increased with a step size denoted s t e p β , until it reaches its maximum admissible value w m a x . In doing so, the size of the invariant set slowly reduces, as the input constraint limit value decreases with a step size denoted s t e p α and the uncertainty level rises.
In practice, a good start for u m a x and w m a x bounds are the values for the imposed constraints (5). The values of the step size s t e p α , s t e p β should be selected such that the table size remains reasonable, with various invariant sets. Moreover, the limits in the state constraints are considered fixed, according to the sub-systems dynamics and used to compute every set Ω i from the table.
Algorithm 1
For α = u m a x : s t e p α : 0.1
For β = 0.1 : s t e p β : w m a x
  1. Compute the inequality constraints:
A u u α b u ; A w w β b w ; A x x b x
  2. Compute the robust positive-invariant set:
Ω ( A , B , K , A u , b u , A w , b w , A x , b x )
  3. Save the information α , β , Ω

2.2. Coalitional Distributed Model Predictive Control (C-Dmpc) Methodology

As previously mentioned, what differentiates our proposed coalitional algorithm from the existing works is the flexible framework set for the cyber-physical multi-agent system with a chain architecture. Hence, at each step time, the agents architecture starts as non-cooperative DMPC and will switch to coalitional DMPC (C-DMPC)—when the local feasibility of the interconnected agents is lost. In the C-DMPC framework, the coalition procedure is initialized without a hierarchical level by the local agents with infeasible problems, because due to the coupling links between sub-systems, if not solved, this problem will propagate among neighbouring sub-systems. Using the communication links, these agents share their optimization status with their neighbour, and after that, one of them is randomly selected to start a coalition. Once the coalition procedure is activated, the agents framework changes.
To simplify the design and computational costs, the size of the coalition is increased gradually, if needed. That is, if a coalition of two agents, coupled with the remaining agents from the network, still does not provide feasible solutions for all involved actors, then more work needs to be done. The idea is to first activate all coalitions of two agents, if needed, then the coalitions of three agents, and so on, until in the end, in the extreme case, all the agents are involved in a single coalition. Note that this last case is equivalent to solving a centralized problem for the multi-agent system and will be used in the last resort, if nothing else solved the infeasibility problems that started the coalitional procedure. The reason for this is related with the coalition dynamics (i.e., when two or more agents form a coalition, their respective sub-system models are aggregated and become a single entity). Thus, the number of the optimization variables in a coalition increases with its size, and the local non-cooperative optimization problem becomes a cooperative one inside the coalition. The extreme case of a ‘grand’ coalition between all agents will aggregate all the sub-systems in a single entity (from the control point of view).

2.2.1. Coalition Dynamics

As described before, our C-DMPC algorithm is tailored specifically for cyber-physical multi-agent systems, linked in a unidirectional communication topology. Thus, the coupling information, which is treated as an uncertainty in the local nominal model of each sub-system S i , is received from its predecessor sub-system S i 1 . To minimize the communication burden between consecutive agents, only the self imposed optimization variable u ^ max introduced in (6) is broadcast. This value is firstly used to search for an invariant set inside the predefined table, and secondly acts as the uncertainty limit constraint in the local optimization problem. Using this information, the local optimization problem is then solved, and if the solution is infeasible, then the coalition procedure must be started.
Inside a coalition between different consecutive agents, the aim is to solve a cooperative optimization problem; thus the uncertainty variable becomes fully known. Each agent A i , i N , can form a coalition only with its predecessor, i.e., agent A i 1 , due to the particular dynamical coupling between their corresponding sub-systems (i.e., linked in a chain). When this occurs, the agents involved will form a compact set denoted generically C . To simplify the notations, the coalition is described without sub-script indices with the following model:
x C ( k + 1 ) = A C x C ( k ) + B C u C ( k ) + w C ( k ) w C ( k ) = j N C B C j u j ( k ) y C ( k ) = C C x C ( k )
where x C is the state vector of the coalition, u C is the coalition’s input vector, w C is the uncertainty vector of the coalition and y C is output vector for the coalition. All these vectors are composed by aggregating the local vectors corresponding to each sub-system involved in the coalition (e.g., x C = [ x i ] i C ). Moreover, the matrices A C , B C , B C j and C C are computed according to the aggregation.
The set N C denotes the coalition’s C neighbour, defined as the predecessor sub-system for the sub-systems inside the coalition (e.g., if Agent 2 and 3 form a coalition, then N C = { 1 } , because sub-system 2 is coupled to sub-system 1; thus the coalition in which Agent 2 is involved must receive relevant information from Agent 1, which is outside the coalition and solves a non-cooperative DMPC problem). Moreover, following this reasoning, a coalition involving Agent 1 does not have neighbours (i.e., N C = , because Agent 1 does not have predecessors).

2.2.2. Coalition Problem Definition

In this section, some details regarding the construction of the constraints sets imposed for the coalition and the optimization problem solved by the coalition are presented.
Hence, the constraint sets for the coalition C are computed as the union of the constraints sets (5) corresponding to each agent A i , i C :
y C Y C = i C Y i , u C U C = i C U i , w C W C = i C W i ,
and the min–max optimization problem solved by the coalition is:
J C ( x C 0 ) = min u C 0 , , u C N 1 u ^ C max max w C 0 , , w C N 1 R w C u ^ C max 1 + l = 0 N p 1 r C l y C l 1 + R u C u C l 1 + r C N p y C N p 1 s . t . ( 8 ) y C l Y C , l = 1 , , N p 1 y C N p Ω C u C l u ^ C max u C max w C l w C max
The weighting matrices R u C and R w C are block diagonal, Ω C is the aggregated terminal set and r C l and x C 0 are aggregated vectors containing the corresponding imposed references and initial state values, respectively. r C N p and y C N p are aggregated vectors containing the corresponding imposed references and output predictions values at time k + N p , respectively. u C l , w C l and y C l are are aggregated vectors containing the corresponding input, uncertainty and output sequences, respectively. u ^ C max is an aggregated vector containing the corresponding self-imposed input limits. u C max and w C max are aggregated vectors containing the corresponding input and uncertainty limits, respectively.

2.2.3. C-Dmpc Algorithm

To summarize the C-DMPC methodology, the following pseudo-code is provided: With regard to Algorithm 2, the following observations are in order:
  • the default uncertainty value used in Step 1 is selected to ensure that optimization problems from Step 3 are feasible, thus ensuring that the proposed methodology is recursively stable (i.e., the terminal set for the coalition is obtained by aggregating the terminal sets of the involved individual agents).
  • if the condition from Step 6 is satisfied, then at that sampling period, the working framework is non-cooperative DMPC; otherwise the framework changes to coalitional DMPC (since at least one coalition is activated).
  • the priority value, which is used as a condition term to initialize a coalition, is defined by each agent as a random sub-unitary number. In this manner, there is no use of a hierarchical control level to assign these priorities.
  • in the extreme, all the agents can be combined in a coalition ( C = N ), which corresponds to a centralized MPC working framework.
  • one or more coalitions can be active simultaneously and are dissolved at the end of each sampling period.
Remark 2
([28]). In Algorithm 2, the coalitional control problem is feasible (i.e., Step 6. (c). ii.), because W C i C W i , U C = i C U i and Ω C = i C Ω i . The stability of the coalition is ensured by the terminal constraint set of the coalition, which is calculated as the Minkowski sum of the terminal sets polytopes defined for each individual agent from the coalition. The coalitional algorithm is recursive-feasible, contingent on Step 3, for which all the optimization problems are feasible, i.e., for which systems can work in a decentralized fashion.
Next, the C-DMPC methodology is validated in simulation, and the results are provided in Section 3.

3. Illustrative Example

In this section, the simulation results and discussion for the C-DMPC method are presented. The proposed simulation scenario for the cyber-physical multi-agent system described in Section 2, Figure 1, has the following characteristics:
  • Four heterogeneous discrete-time sub-systems S i , i { 1 , , 4 } , coupled in a chain architecture were defined using (1), with the following numerical matrices:
    S 1 : A p 1 , 1 = 0.7913 0.2020 0.1010 0.8417 B p 1 , 1 = 0.0271 0.2291 B p 1 , 0 = 0 0 C p 1 = 0 1
    S 2 : A p 2 , 2 = 0.7936 0.1996 0.1198 0.8236 B p 2 , 2 = 0.0269 0.2265 B p 2 , 1 = 0.0004 0.0034 C p 2 = 0 1
    S 3 : A p 3 , 3 = 0.7888 0.2043 0.0817 0.8604 B p 3 , 3 = 0.0273 0.2316 B p 3 , 2 = 0.0004 0.0034 C p 3 = 0 1
    S 4 : A p 4 , 4 = 0.7912 0.1994 0.0997 0.8211 B p 4 , 4 = 0.0269 0.2263 B p 4 , 3 = 0.0004 0.0034 C p 4 = 0 1
  • The limit constraints for the inputs, disturbances and outputs are the following:
    u i min = 5 u i max = 5 w i min = 1 w i max = 1 w i min = 8 w i max = 8 i { 1 , , 4 }
  • For all sub-systems S i , i { 1 , , 4 } , the following optimization parameters are used: the prediction horizon N p = 5 , the input weights R u i = 0.1 and R w i = 0.01 .
Remark 3.
The optimization parameters were carefully selected after a thorough analysis from the point of view of achieved performances. Several tests were performed, with different values for the weights and the prediction horizon. The chosen values ensured the best performances.
  • The feedback laws were computed using classical state-feedback control based on the Ackermann’s formula [31], applied for the extended model (4), obtaining:
    K 1 = [ 0.5494 2.6061 0.7488 ] , K 2 = [ 0.3473 2.5199 0.7047 ] , K 3 = [ 0.6249 2.6565 0.7401 ] , K 4 = [ 0.4355 2.5107 0.7051 ] .
Remark 4.
The Ackermann’s formula [31] was used to achieve specific closed-loop transient performances, chosen as an overshoot value of 5 % and settling time of 5 time units, for sub-systems S 1 and S 3 , and an overshoot value of 4 % , and the same settling time, corresponding to sub-systems S 2 and S 4 . These performance values, were accordingly selected based on each sub-system dynamics.
  • The reference tracking scenario was constructed for 12 time samples, using a sampling period T s = 0.25 s , with the following imposed references:
    r 1 = [ 0.2 0.2 0.2 0.2 0.2 0.2 0.5 0.5 0.5 0.5 0.5 0.5 ] , r 2 = [ 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 ] , r 3 = [ 0.2 0.2 0.2 0.2 0.5 0.5 0.5 0.5 0.2 0.2 0.2 0.2 ] , r 4 = [ 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 ] .
Since our proposed scenario has four sub-systems in a chain architecture with unidirectional communication links between the agents, there are eight possible frameworks including coalitions of two, three or four agents defined as follows:
default case—no coalitions between A 1 , A 2 , A 3 , A 4 ;
coalition C 12 between A 1 and A 2 , while A 3 , A 4 remain outside the coalition but interconnected;
coalition C 123 between A 1 , A 2 , and A 3 , while A 4 remains outside the coalition but interconnected;
twosimultaneous active coalitions C 12 and C 34 between A 1 and A 2 and A 3 and A 4 , respectively, which are interconnected;
coalition C 23 between A 2 and A 3 , while A 1 , A 4 remain outside the coalition but interconnected;
coalition C 234 between A 2 , A 3 and A 4 , while A 1 remains outside the coalition but interconnected;
coalition C 34 between A 3 and A 4 , while A 1 , A 2 remain outside the coalition but interconnected;
extreme case: coalition C 1234 between all agents A 1 , A 2 , A 3 , A 4 .
Algorithm 2
 Initialization: For each agent A i , i N , compute a table T i , with potential terminal sets Ω i .
 At each sampling time k, each agent A i , i N , receives the local state value and performs the following steps:
 1. Computes the uncertainty polytope using default limit values for the constraints:
W i = B i , i 1 U i 1 0 U i 1 = [ u i 1 max , 0 ; u i 1 max , 0 ] .
 2. Searches in the predefined table T i for a terminal set Ω i 0 that includes the default uncertainty W i Ω i 0 .
 3. Solves the local optimization problem (6) and obtains the optimal values U i * , 0 , u ^ i max , 0 using the default values Ω i = Ω i 0 for the terminal set and the uncertainty constraint limit ( w i max = u i 1 max , 0 ).
 4. Broadcasts to its successor the local optimal value u ^ i max , 0 and receives the corresponding value u ^ i 1 max , 0 from its predecessor.
 5. Repeats Steps 1–3 using the uncertainty constraint value received in Step 4.
 6. Checks the feasibility of the local optimization problem:
If the optimization problem from Step 5 is feasible:
then: Coalitions between agents are not necessary.Each local agent A i sends to its sub-system S i , the first value from the optimal trajectory U i * ;
else: Coalitions between agents are necessary. In this case, in order to be included in a coalition, each agent A i , i N , performs the following steps:
   a. Receives, from its predecessor, a coalitional report containing the following information: the feasibility status (for the local optimization problem solved at Step 5) and priority value relating to all the predecessor agents from the chain architecture.
   b. Sends to its successor, the updated coalitional report (i.e., all the relevant information received, together with its own local feasibility and priority data).
   c. Initializes a coalition only if its local priority is the highest from the report. Within a coalition between two agents, the following steps are performed:
      i. the coalition model is defined as (8);
      ii. the optimization problem (10) subject to (9) is solved.
      iii. the relevant information is broadcast to the coalition’s neighbour.
      iv. a feasibility check for all the optimization problems is done.
       If the all the optimization problems are feasible:
       then: The existing coalition was successful and can be dissolved after every sub-system S i receives the first value from the optimal trajectory U i * ;
       else: The existing coalition was not successful. Another agent must be included in the existing coalition (if the coalition’s status is infeasible), or another coalition can be activated (if more agents outside the existing coalition have infeasible problems). At this stage, Step (c) is repeated as necessary.
 7. End algorithm.
Remark 5.
Please note that our proposed Coalitional DMPC algorithm is tailored specifically for cyber-physical multi-agent systems. The key feature is its capability to switch between control architectures, whenever the feasibility of the multi-agent system is lost, due to uncertainties in the local sub-systems. One example of such multi-agent system is a vehicle platoon. In this case, it is clear that classical centralized MPC is not suitable for controlling this application. Moreover, decentralized MPC, in which the couplings between sub-systems are ignored, can render instability within the platoon. One compromise solution is distributed MPC, in which the interactions are taken into account when computing the local solutions. However, if the distributed MPC (i.e., in non-cooperative framework) fails at this task, then our proposed coalitional DMPC provides a backup plan, namely to merge different sub-systems into coalitions. Inside a coalition, all the information is known; thus, only the coupling signals with sub-systems outside the coalition must be accounted for.
The invariant sets obtained for sub-system S 1 using Algorithm 1 presented in Section 2.1 are depicted in Figure 2. For the computation, the following numerical values were used: u m a x = 5 , w m a x = 5 , s t e p α = s t e p β = 0.5 . As expected, the larger invariant set (depicted with red colour) was obtained for α = u m a x = 5 and β = 0.1 . Moreover, as the constraint limits become smaller, the set Ω i decreases in dimension and is included in the larger red set (Figure 3—the sets plotted with green, blue, magenta and black colours, respectively). Since, the state variable for the extended model (4) has three values, the computed invariant sets are three dimensional and can be plotted as convex hulls (ref. Figure 2 and Figure 3). This graphical representation of the invariant sets, which are predefined options for the terminal set constraint (6), are also useful when defining the reference target for the multi-agent system. Thus, one must take into account that the imposed trajectory for each sub-system S i , i { 1 , , 4 } should be placed in the interior of the invariant set.
Remark 6.
It is worth mentioning that the step values s t e p α = s t e p β = 0.5 were selected taking into account the numerical values of the input and uncertainty constraints to ensure a sufficient number of invariant sets computed. If a smaller value, e.g., s t e p α = s t e p β = 0.1 is chosen, the result would be an increased size for the table containing the invariant sets. However, as depicted in Figure 2, these values also parametrize the dimensions of the invariant set polytopes. Thus, although we would have more available sets, their dimensions would be too similar, to justify the involved computational costs.
The reference tracking results and the formation of the coalitions during the simulation are presented in Figure 4.
As depicted in Figure 4, lower subplot, during the first seven time steps, the simulation runs in the default scenario, in which all the agents solve a non-cooperative DMPC algorithm without being involved in a coalition. This is marked with blue circles, at each time step, for each agent A i , i { 1 , , 4 } . At time step 8, due to the setpoint change of 0.3 units in r 3 for sub-system S 3 and the corresponding increase in the control effort u 3 , the local feasibility for sub-system S 4 is lost. Hence, the coalition C 34 is activated, which is plotted with a red star marker for A 3 and A 4 . At the next time step, coalition C 234 between agents A 2 , A 3 and A 4 is active and is coupled with the remaining agent A 1 , because sub-system S 2 is dynamically coupled through the input with S 1 and their corresponding agents share information. At time step 10, coalition C 34 is activated, and for the remaining two time steps of the simulation, coalition C 234 is active. Moreover, the reference tracking results show that all the imposed set-points are successfully reached in one sampling time, with zero offset error. This occurs for the first seven time steps, in which all the agents work outside a coalition, and also for the remaining simulation time, when coalitions of two or three agents are necessary to maintain the feasibility of the CP-MAS. The results clearly prove the efficiency of our proposed C-DMPC method in a reference tracking scenario.

4. Conclusions

In this work, a coalitional distributed model predictive (C-DMPC) methodology suitable for input coupled cyber-physical multi-agent systems was proposed. The algorithm was tailored for an in-chain system architecture with unidirectional communication links (i.e., the coupling information viewed as an uncertainty in the local nominal model was broadcast from a predecessor sub-system to a successor). The methodology was validated in simulation, using an academic cyber-physical multi-agent system as a proof of concept for the proposed algorithm. The simulation results show that if the uncertainty level received by the local agent is manageable, a non-cooperative DMPC algorithm could be locally solved. However, when the local feasibility of the optimization problem was lost, then forming coalitions between agents showed satisfactory performance and the usefulness of the C-DMPC algorithm was proven.
Future work will test the efficiency of the proposed algorithm on a vehicle platooning application.

5. Materials and Methods

The simulations from this work were performed using MATLAB R2020b on Windows 10, 64-bit Operating System with a laptop Intel Core i7-9850H CPU @ 2.60 GHz and 16 GB RAM.
The optimizations were implemented using the YALMIP toolbox [34].

Author Contributions

Conceptualization, A.M. and C.-F.C.; methodology, A.M.; software, A.M.; validation, A.M. and C.-F.C.; formal analysis, A.M.; investigation, A.M.; resources, A.M.; data curation, A.M.; writing—original draft preparation, A.M.; writing—review and editing, A.M. and C.-F.C.; visualization, A.M.; supervision, C.-F.C.; project administration, A.M. and C.-F.C.; funding acquisition, A.M. and C.-F.C. All authors have read and agreed to the published version of the manuscript.


This work was supported by two grants of the Ministry of Research, Innovation and Digitization, CNCS/CCCDI – UEFISCDI, project numbers PN-III-P1-1.1-PD-2019-0757 and PN-III-P1-1.1-TE-373 2019-1123, within PNCDI III.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.


The following abbreviations are used in this manuscript:
MPCModel Predictive Control
DMPCDistributed Model Predictive Control
C-MPCCoalitional Model Predictive Control
C-DMPCCoalitional Distributed Model Predictive Control
CP-MASCyber-Physical Multi-Agent System
CPsSCyber-Physical sub-system


  1. Gomes, L.; Almeida, C.; Vale, Z. Recommendation of Workplaces in a Coworking Building: A Cyber-Physical Approach Supported by a Context-Aware Multi-Agent System. Sensors 2020, 20, 3597. [Google Scholar] [CrossRef] [PubMed]
  2. Dumitrache, I.; Caramihai, S.I.; Moinescu, M.A.; Sacala, I.S.; Vladareanu, L.; Repta, D. A Perceptive Interface for Intelligent Cyber Enterprises. Sensors 2019, 19, 4422. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Ierardi, C.; Orihuela, L.; Jurado, I. Distributed Estimation Techniques for Cyber-Physical Systems: A Systematic Review. Sensors 2019, 19, 4720. [Google Scholar] [CrossRef] [Green Version]
  4. Dumitrache, I.; Sacala, I.S.; Moinescu, M.A.; Caramihai, S.I. A Conceptual Framework for Modeling and Design of Cyber-Physical Systems. Stud. Inform. Control 2017, 26, 325–334. [Google Scholar] [CrossRef] [Green Version]
  5. Lee, E.A. The Past, Present and Future of Cyber-Physical Systems: A Focus on Models. Sensors 2015, 15, 4837–4869. [Google Scholar] [CrossRef]
  6. Christofides, P.D.; Scattolini, R.; Muñoz de la Peña, D.; Liu, J. Distributed model predictive control: A tutorial review and future research directions. Comput. Chem. Eng. 2013, 51, 21–41. [Google Scholar] [CrossRef]
  7. Huang, D.; Li, H.; Li, X. Formation of Generic UAVs-USVs System Under Distributed Model Predictive Control Scheme. IEEE Trans. Circ. Syst. Express Briefs 2020, 67, 3123–3127. [Google Scholar] [CrossRef]
  8. Franzè, G.; Casavola, A.; Famularo, D. Distributed Receding Horizon Control of Constrained Networked Leader–Follower Formations Subject to Packet Dropouts. IEEE Trans. Control Syst. Technol. 2017, 25, 1798–1809. [Google Scholar] [CrossRef]
  9. Maxim, A.; Caruntu, C.F.; Lazar, C. Cruise and headway control for vehicle platooning using a distributed model predictive control algorithm. In Proceedings of the 21st International Conference on System Theory, Control and Computing, Sinaia, Romania, 19–21 October 2017; pp. 146–151. [Google Scholar]
  10. Sasaki, S.; Namerikawa, T. Traffic Signal Control Considering Switching Timing via Distributed Model Predictive Control. In Proceedings of the 19th International Conference on Control, Automation and Systems, Jeju, Korea, 15–18 October 2019; pp. 286–291. [Google Scholar]
  11. Zheng, Y.; Wei, Y.; Li, S. Coupling Degree Clustering-Based Distributed Model Predictive Control Network Design. IEEE Trans. Autom. Sci. Eng. 2018, 15, 1749–1758. [Google Scholar] [CrossRef]
  12. Meng, K.; Dong, Y.; Xu, Z.; Weller, S.R. Cooperation-Driven Distributed Model Predictive Control for Energy Storage Systems. IEEE Trans. Smart Grid 2015, 6, 2583–2585. [Google Scholar] [CrossRef]
  13. Lou, G.; Gu, W.; Xu, Y.; Cheng, M.; Liu, W. Distributed MPC-based secondary voltage control scheme for autonomous drop-control microgrids. IEEE Trans. Sustain. Energy 2017, 8, 792–804. [Google Scholar] [CrossRef]
  14. Lou, G.; Gu, W.; Sheng, W.; Song, X.; Gao, F. Distributed Model Predictive Secondary Voltage Control of Islanded Microgrids With Feedback Linearization. IEEE Access 2018, 6, 50169–50178. [Google Scholar] [CrossRef]
  15. Zhu, J.; Xue, B. Distributed Predictive Control of Multi-Agent Systems Based on Error Upper Bound Approach. IEEE Access 2021, 9, 11470–11478. [Google Scholar] [CrossRef]
  16. Tang, X.; Li, M.; Wei, S.; Ding, B. Event-triggered Synchronous Distributed Model Predictive Control for Multi-agent Systems. Int. J. Control. Autom. Syst. 2021, 19, 1–10. [Google Scholar] [CrossRef]
  17. Li, H.; Li, X. Distributed Model Predictive Consensus of Heterogeneous Time-Varying Multi-Agent Systems: With and Without Self-Triggered Mechanism. IEEE Trans. Circuits Syst. I Regul. Pap. 2020, 67, 5358–5368. [Google Scholar] [CrossRef]
  18. Maestre, J.M.; Negenborn, R.R. Distributed Model Predictive Control Made Easy; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar]
  19. Fele, F.; Maestre, J.M.; Camacho, E.F. Coalitional control: Cooperative game theory and control. IEEE Control Syst. 2017, 37, 53–69. [Google Scholar]
  20. Fele, F.; Debada, E.; Maestre, J.M.; Camacho, E.F. Coalitional Control of Self-Organizing Agents. IEEE Trans. Autom. Control. 2018, 63, 2883–2897. [Google Scholar] [CrossRef]
  21. Masero, E.; Fletcher, L.A.; Maestre, J.M. A Coalitional Model Predictive Control for the Energy Efficiency of Next-Generation Cellular Networks. Energies 2020, 13, 6546. [Google Scholar] [CrossRef]
  22. Masero, E.; Fletcher, L.A.; Maestre, J.M. A Coalitional Model Predictive Control Approach for Heterogeneous Cellular Networks. In Proceedings of the 2020 European Control Conference, Saint Petersburg, Russia, 12–15 May 2020; pp. 448–453. [Google Scholar]
  23. Masero, E.; Maestre, J.M.; Francisco, M.; Camacho, E.F. Coalitional MPC with predicted topology transitions. IFAC PapersOnLine 2020, 52, 3342–3347. [Google Scholar] [CrossRef]
  24. Baldavieso Monasterios, P.R.; Trodden, P.A.; Cannon, M. On feasible sets for coalitional MPC. In Proceedings of the 58th IEEE Conference on Decision and Control, Nice, France, 11–13 December 2019; pp. 4668–4673. [Google Scholar]
  25. Maestre, J.M.; Muñoz de la Peña, D.; Camacho, E.F. Distributed model predictive control based on a cooperative game. Optim. Control. Appl. Methods 2011, 32, 153–176. [Google Scholar] [CrossRef]
  26. Scattolini, R. Architectures for distributed and hierarchical Model Predictive Control—A review. J. Process. Control 2009, 19, 723–731. [Google Scholar] [CrossRef]
  27. Maxim, A.; Maestre, J.M.; Caruntu, C.F.; Lazar, C. Min-max coalitional model predictive control algorithm. In Proceedings of the 22nd International Conference on Control Systems and Computer Science, Bucharest, Romania, 28–30 May 2019. [Google Scholar]
  28. Maxim, A.; Maestre, J.M.; Caruntu, C.F.; Lazar, C. Robust coalitional distributed model predictive control algorithm with stability via terminal constraint. In Proceedings of the IEEE Conference on Control Technology and Applications, Copenhagen, Denmark, 21–24 August 2018; pp. 964–969. [Google Scholar]
  29. Maxim, A.; Pauca, O.; Caruntu, C.; Lazar, C. Distributed Model Predictive Control Algorithm with Time-Varying Communication Delays for a CACC Vehicle Platoon. In Proceedings of the 24th International Conference on System Theory, Control and Computing, Sinaia, Romania, 8–10 October 2020. [Google Scholar]
  30. Maxim, A.; Lazar, C.; Caruntu, C. Distributed Model Predictive Control Algorithm with Communication Delays for a Cooperative Adaptive Cruise Control Vehicle Platoon. In Proceedings of the 28th Mediterranean Conference on Control and Automation, Saint-Raphaël, France, 15–18 September 2020; pp. 909–914. [Google Scholar]
  31. Ogata, K. Modern Engineering Control, 5th ed.; Prentice Hall: Upper Saddle River, NJ, USA, 2010. [Google Scholar]
  32. Kwakernaak, H.; Sivan, R. Linear Optimal Control Systems; John Wiley & Sons, Inc.: New York, NY, USA, 1972. [Google Scholar]
  33. Maestre, J.M.; Muñoz de la Peña, D.; Camacho, E.F.; Alamo, T. Distributed model predictive control based on agent negotiation. J. Process. Control 2011, 21, 685–697. [Google Scholar] [CrossRef]
  34. Löfberg, J. YALMIP: A Toolbox for Modeling and Optimization in MATLAB. In Proceedings of the IEEE International Symposium on Computer Aided Control Systems Design, New Orleans, LA, USA, 2–4 September 2004; pp. 284–289. [Google Scholar]
Figure 1. Schematic diagram of a cyber-physical multi-agent system (CP-MAS).
Figure 1. Schematic diagram of a cyber-physical multi-agent system (CP-MAS).
Sensors 21 04041 g001
Figure 2. Depiction of the predefined invariant sets corresponding to sub-system S 1 , computed for uncertainty constraint limit value β = 0.1 .
Figure 2. Depiction of the predefined invariant sets corresponding to sub-system S 1 , computed for uncertainty constraint limit value β = 0.1 .
Sensors 21 04041 g002
Figure 3. Detail regarding the depiction of the predefined invariant sets corresponding to sub-system S 1 , computed for uncertainty constraint limit value β = 0.1 .
Figure 3. Detail regarding the depiction of the predefined invariant sets corresponding to sub-system S 1 , computed for uncertainty constraint limit value β = 0.1 .
Sensors 21 04041 g003
Figure 4. Reference tracking results (subplots 1–4), control efforts (subplots 5–8) and the corresponding coalitions formation (subplot 9) for a cyber-physical multi-agent system composed of 4 interconnected sub-systems.
Figure 4. Reference tracking results (subplots 1–4), control efforts (subplots 5–8) and the corresponding coalitions formation (subplot 9) for a cyber-physical multi-agent system composed of 4 interconnected sub-systems.
Sensors 21 04041 g004
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Maxim, A.; Caruntu, C.-F. A Coalitional Distributed Model Predictive Control Perspective for a Cyber-Physical Multi-Agent Application. Sensors 2021, 21, 4041.

AMA Style

Maxim A, Caruntu C-F. A Coalitional Distributed Model Predictive Control Perspective for a Cyber-Physical Multi-Agent Application. Sensors. 2021; 21(12):4041.

Chicago/Turabian Style

Maxim, Anca, and Constantin-Florin Caruntu. 2021. "A Coalitional Distributed Model Predictive Control Perspective for a Cyber-Physical Multi-Agent Application" Sensors 21, no. 12: 4041.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop