Multi-Agent-Based Data-Driven Distributed Adaptive Cooperative Control in Urban Traffic Signal Timing

Abstract

Data-driven intelligent transportation systems (D²ITSs) have drawn significant attention lately. This work investigates a novel multi-agent-based data-driven distributed adaptive cooperative control (MA-DD-DACC) method for multi-direction queuing strength balance with changeable cycle in urban traffic signal timing. Compared with the conventional signal control strategies, the proposed MA-DD-DACC method combined with an online parameter learning law can be applied for traffic signal control in a distributed manner by merely utilizing the collected I/O traffic queueing length data and network topology of multi-direction signal controllers at a single intersection. A Lyapunov-based stability analysis shows that the proposed approach guarantees uniform ultimate boundedness of the distributed consensus coordinated errors of queuing strength. The numerical and experimental comparison simulations are performed on a VISSIM-VB-MATLAB joint simulation platform to verify the effectiveness of the proposed approach.

1. Introduction

Conventional technology-driven ITS is on the brink of a revolution in the age of big data. Data-driven intelligent transportation system (D²ITS) is trending to become a more powerful way of improving traffic efficiency, enhancing traffic security, saving energy and providing more possibilities for different stakeholders [1].

In urban transportation, it is costly and unsustainable to add additional infrastructure to accommodate the increased number of vehicles due to the limited road resources. A more socially feasible option is the optimization of traffic signal timing in a data-driven and learning-based manner. Since urban traffic systems are becoming more and more complex, establishing an exact mechanistic model of road networks or even an intersection is difficult or impossible due to its higher order, strong nonlinearities, non-stationary nature and complicated structures. Furthermore, acquiring the traffic data regarding vehicle count, queue, occupancy and flow has become much easier, and a huge amount of online/offline data is collected every day from auxiliary heterogeneous traffic sensors, e.g., inductive-loop detectors, microwave detectors, video surveillance and RFID. Thereby the spatial–temporal relationship between traffic data should be considered when performing a D²ITS.

One promising way is to develop data-driven control (D²C) [2] for traffic signal timing. In the last decade, research on D²C has received much more attention, such as model free adaptive control (MFAC) [3,4], iterative learning control (ILC) [5], lazy learning control (LL) [6], virtual reference feedback tuning (VRFT) [7], and iterative feedback tuning (IFT) [8]. Without using the explicit information from the mathematical model, D²C is designed by utilizing I/O data directly obtained from the controlled system or data processing [9]. From an application point of view, D²C of urban traffic systems will play a key role in D²ITS due to the availability, reliability and integrity of traffic data and is closer to real-life traffic signal control problems than many model-based approaches [10].

A renowned fixed timing approach is the so-called Webster’s signal plan, which is used to calculate the green time and the cycle using offline traffic data [11]. While some adaptive mechanisms, such as SCOOT [12], SCAT [13] and GLIDE [14], have been successfully implemented in real urban traffic systems due to the merits of dynamic tuning of signal timing. In recent years, some D²C strategies were investigated addressing the performance optimization of complex dynamic traffic systems. Adaptive dynamic programming (ADP) and reinforcement learning (RL) methods [15] have emerged in solving Markov decision processes with uncertain conditions. In [16], for solving the problem of streetcar bunching, the multiple reinforcement learning agents were developed for a series of successive signalized intersections. A Q-learning algorithm was introduced for optimal control of heavily congested traffic across in [17]. To achieve the global optimization of urban traffic control, Salkham et al. [18] studied a collaborative reinforcement learning approach.

Although ADP-based learning control could achieve good results, the shortcomings of slow convergence rate, poor stability, curse of dimensionality and difficulty in selecting reward functions may extremely hinder its application. Therefore, some real-time, rapid, recursive D²C designs by using MFAC or ILC techniques have also been proposed for freeway traffic control systems with high nonlinearity and repetitive operations [19,20,21]. Unfortunately, those protocols are still deficient for realizing traffic signal D²C.

Moreover, some prominent features, such as multi-functionality, multi-sourcing and multi-users, have recently become a positive force for traffic control changes. Therefore, as a rapidly developing field of artificial intelligence, multi-agent-based urban traffic distributed control strategies have attracted significant importance because they provide a highly flexible and modular structure [22]. To avoid solving complex traffic control problem as a whole, they offer distributed protocols by dividing the original problem into many sub-problems to get an optimal solution, in particular, for a large-scale traffic network system.

By using a hybrid neural network, a new simultaneous perturbation stochastic approximation- based neural network (SPSA-NN) [23] is developed for each intersection. The total mean delay and the total mean stoppage time of each vehicle has been substantially reduced. However, for each traffic pattern, a large number of training samples is needed for the implementation of a SPSA-NN model. The number of decision-making modules and the slow convergence rate would reduce the system performance. Another distributed multi-agent-based control method is Type-2 fuzzy with dynamic reasoning (T2DR) [24]. A weighted type-2 fuzzy inference engine and separate belief model are proposed to calculate the weights for input. The problem in this case is that T2DR strongly depends on the accuracy of the trained belief model’s limiting the parameter learning capability. Similar to the T2DR, geometric fuzzy multi-agent system (GFMAS) is addressed by modifying the belief model to reduce the number of training samples needed, thereby geometric defuzzification is used to further reduce the computational requirements [25]. A large number of samples is however still needed to ensure better accuracy. Also, an unconstrained fixed sequence clearing strategy (UNCFSCS) [26] is adopted recently for all completely empty vehicles in a fixed sequence from the point of multi-phase coordination. It is very restricted in application due to the unsaturated intersections condition. Therefore, a sample independent distributed multi-agent-based control under constraints is indispensable as a supplement.

It was pointed out by Choy et al. [27] that a three layered hierarchical multi-agent architecture, intersection controller agents (ICA), zone controller agents (ZCA) and regional controller agents (RCA), is appropriate to analyze the real-time traffic signal control. In terms of ICA, the traffic data of queuing length, vehicle arrival rate and saturation flow rate collected from different sensors placed in the lanes is taken into account as each ICA input. Based on the obtained data, green time for a specific phase or direction in progress and the cooperation factor of multi-agents in intersections are computed to adjust the signaling and traffic routing pattern for alleviating traffic jams, reducing the environmental pollution and the consumption of energy.

In this paper, we focus on the lowest layer ICA signal timing problem. A novel multi-agent-based data-driven distributed adaptive corporative control (MA-DD-DACC) is investigated for multi-direction queuing strength balance with changeable cycles at single intersections in urban traffic signal control. The main contributions of this work can be summarized as follows:

(1)

A rapid and recursive distributed adaptive corporative control strategy with online parameter learning law is proposed by using the queuing length and the network topology of multi-direction signal controllers at single intersection;

(2)

Traffic signal direction-sequence control is adopted here to avoid phase conflicts;

(3)

A flexible changeable cycle signal timing strategy is analyzed satisfying maximum and minimum green time constraints;

(4)

Both the undersaturation and supersaturation traffic flow conditions can be handled by the proposed MA-DD-DACC without distinction.

The rest of this paper is distributed as follows: Section 2 briefly introduces the traditional store-and-forward model and multi-directional queuing strength model, respectively. Section 3 introduces the data-driven cooperative control technology for queuing strength balance in a single intersection network. Section 4 shows the Lyapunov-based stability analysis of MA-DD-DACC. Section 5 gives the simulation results obtained by a joint programming method, showing the effectiveness of the algorithm. Finally, Section 6 summarizes the paper.

2. Preliminaries

2.1. Store-and-Forward Model

As shown in Figure 1 branch $m$ connects intersection $i$ and $j$, and vehicles move from intersection $i$ to intersection $j$.

The traffic flow dynamic of branch $m$ is expressed as follows:

$$x_m(k+1)=x_m(k)+T\left[q_m(k)-u_m(k)+I_m(k)-O_m(k)\right]$$

(1)

where T is known as the sampling interval, $x_m(k)$ denotes the number of vehicles on branch $m$, $q_m(k)$ and $u_m(k)$ are defined as the input and output of branch $m$ in $[kT,(k+1)T]$, respectively. $I_m(k)$ and $O_m(k)$ represent traffic flow demand and dissipation of branch $m$, respectively.

Dissipative flow $O_m(k)$ satisfies the following equation:

$$O_m(k)=t_{m,0}{q}_m(k)$$

(2)

where $t_{m,0}$ is defined as the dissipation rate which is a known constant.

Input flow $q_m(k)$ satisfies the following equation:

$$q_m(k)={\displaystyle \sum _{n\in I_i}{t}_{n,m}{u}_n(k)}$$

(3)

where $t_{n,m}$ is the steering ratio of vehicles on branch $n$ entering branch $m$ through intersection j. Assuming that the traffic demand of branch $m$ is sufficient, the output flow $u_m(k)$ satisfies:

$$u_m(k)=\left(\raisebox{1ex}{$g_m\left(k\right)$}\!\left/ \!\raisebox{-1ex}{$C$}\right.\right)O_m$$

(4)

where $C$ is the signal cycle. $g_m(k)$ denotes the green time and satisfies:

$$g_m(k)={\displaystyle \sum _{l\in v_m}{g}_{j,l}\left(k\right)}$$

(5)

If the sampling interval T equals the cycle $C$, $u_m(k)$ equals the mean of traffic flow in the whole cycle, the actual traffic flow is $s_m$ or zero in the green or red period, so as long as the traffic demand is sufficient, there is a continuous and stable traffic flow in the road network.

By substituting Equations (2)–(5) into (1), it can be obtained that:

$$x_m(k+1)=x_m\left(k\right)+T\left[\left(1-t_{m,0}\right){\displaystyle \sum _{n\in I_i}{t}_{n,m}\frac{O_n{\displaystyle {\sum }_{l\in v_n}{g}_{i,l}\left(k\right)}}{c}}+I_m\left(k\right)-\frac{O_m{\displaystyle {\sum }_{l\in v_m}{g}_{j,l}\left(k\right)}}{c}\right]$$

(6)

The state equation of arbitrary dimension, topological structure and traffic flow characteristics can be rewritten by using Equation (6):

$$x(k+1)=x\left(k\right)+B\Delta I\left(k\right)-D\Delta g\left(k\right)$$

(7)

where the state vector $x(k)$ denotes the number of vehicles on each road in the road network during the k-th sampling period, $B$ and $D$ are stationary matrices reflecting road network characteristics. Denote $\Delta g(k)=g\left(k\right)-g^N$ , $\Delta I(k)=I\left(k\right)-I^N$. $g(k)$ is the green time vector of all intersections in the road network, $g^N$ is the corresponding initial green time nominal vector, $I(k)$ and $I^N$ are traffic demand and initial steady flow, respectively.

2.2. Multi-Directional Queuing Strength Model with Changeable Cycle

The multi-directional queuing strength balance problem for traffic signal timing is shown in Figure 2. Based on the store-and-forward model, the multi-directional queuing length model with changeable cycle at single intersection is as follows:

$$l_i(k+1)=l_i(k)+C(k)q_i(k)-S_i{g}_i(k)$$

(8)

where $i=1,\dots ,N$ and $N=4$. $i$ denotes the i-th direction of the intersection. $N$ denotes four directions; $C(k)$ denotes the k-th cycle which is changeable; $l_i(k)$ denotes the number of queuing vehicles of the k-th cycle in the i-direction; $q_i(k)$ denotes the vehicle arrival rate of the k-th cycle in the i-th direction, $S_i$ denotes the saturation flow rate in the i-th direction; $g_i(k)$ denotes the green time of the k-th cycle in the i-th direction.

The green time and traffic signal cycle in all directions at intersections satisfy the following constraints:

$$C(k)={\displaystyle \sum _{i=1}^4{g}_i(k)}+t_L$$

(9)

where $t_L$ denotes the total lost time at the intersection.

The queuing strength of an entrance line at an intersection is defined as the ratio of the queuing length of the entrance line (the number of queuing vehicles) to the length of its link (the storage capacity of vehicles) is as follows:

$$x_i(k)=\frac{l_i(k)}{l_{i,\max }}$$

(10)

where $x_i(k)$ denotes the queuing strength of the i-th entrance line in the k-th cycle, $l_{i,\max }$ denotes the storage capacity of vehicles of the i-th entrance line.

Therefore, multi-directional queuing strength model is given below:

$$x_i(k+1)=x_i(k)+\frac{C(k)q_i(k)}{l_{i,\max }}-\frac{S_i{g}_i(k)}{l_{i,\max }}$$

(11)

The degree of saturation of the k-th cycle in the i-th entrance line of the intersection is defined as:

$$d_i(k)=\frac{C(k)q_i(k)}{S_i{g}_i(k)}$$

(12)

According to the definition of the degree of saturation, traffic state of the k-th cycle in the i-th direction can be judged as follows: (1) $d_i(k)<1$ indicates undersaturation; (2) $d_i(k)=1$ indicates critical saturation; (3) $d_i(k)>1$ indicates supersaturation.

When traffic state of the intersection is undersaturation, i.e., $d_i(k)<1$, according to Equation (11), $x_i(k+1)$ decreases continuously and may eventually be reduced to a negative value. Due to the non-negative characteristics of traffic system variables, the undersaturation state in store-and-forward model will lose its physical and control significance at this time. In the traditional store-and-forward model analysis, only the traffic signal timing under the supersaturated traffic condition is usually considered. If more green time is allocated in an undersaturated situation, there will be a waste of green time. Therefore, only less green time is needed to ensure capacity. However, since most traffic signal control systems are fixed cycle control and meet the constraints of maximum and minimum green time, signal timing is often limited and cannot be perfectly compatible with undersaturated, critical saturated and supersaturated conditions. In order to avoid wasting green time, it is necessary introduce time-varying control signal cycle $C(k)$ leading to new green time constraints to ensure the queue strength satisfying $x_i(k+1)\ge 0$. Therefore, the green time and signal cycle under constraints become:

$${\overline{g}}_i(k)\triangleq sat(g_i(k))=\{\begin{array}{cc}\min \left\{[l_i(k)+C(k)q_i(k)]/S_i,g_{\max }\right\}& g_i(k)>\overline{g}\\ g_i(k)& \underset{\_}{g}\le g_i(k)\le \overline{g}\\ g_{\min }& g_i(k)<\underset{\_}{g}\end{array}$$

(13)

$$C(k)={\displaystyle \sum _{i=1}^4{\overline{g}}_i(k)+t_L}$$

(14)

Accordingly, the constrained multi-direction queuing strength model (11) with changeable cycle at single intersection would be rewritten as:

$$\{\begin{array}{c}{x}_i(k+1)=x_i(k)+f_i(k)-{\overline{u}}_i(k)\\ f_i(k)=\frac{C(k)q_i(k)}{l_{i,\max }}\\ {\overline{u}}_i(k)=\frac{S_i{\overline{g}}_i(k)}{l_{i,\max }}\end{array}$$

(15)

The proposed queueing strength model (15) can formulate three traffic flow conditions: undersaturation, critical saturation and supersaturation, simultaneously.

3. Problem Formulation and Controller Design

3.1. Cooperative Control Problem Describtion

When the queuing strength balance is achieved at single intersection, the desired queueing strength can be expressed as:

$$\{\begin{array}{c}{x}_r(k+1)=x_r(k)+f_r(k)-u_r(k)\\ x_r(k)=\frac{1}{4}{\displaystyle \sum _{h=1}^4{x}_i(k)}\\ f_r(k)=\frac{1}{4}C(k){\displaystyle \sum _{h=1}^4\frac{q_h(k)}{l_{h,\max }}}\\ u_r(k)=\frac{1}{4}C(k){\displaystyle \sum _{h=1}^4\frac{S_h{\overline{g}}_h(k)}{l_{h,\max }}}\end{array}$$

(16)

where $x_r(k)\in R$ is the average queue strength of four directions in k-th cycle; $f_r(k)\in R$ and $u_r(k)\in R$ are the average increase and decrease of queuing strength in the k-th signal cycle.

Considering each direction queuing strength as an agent, distributed consensus coordinated error of queuing strength in the i-th direction of one intersection is given below:

$$e_i(k)={\displaystyle \sum _{j\in N_i}^{}{a}_{ij}(x_j(k)-x_i(k))}+b_i(x_r(k)-x_i(k))$$

(17)

where $A=[a_{ij}]$ denotes the adjacency matrix in network topology of signal controllers at single intersection, $b_i$ denotes the connection coefficient between the $i$-direction queuing strength and the desired one. Moreover, define the in-degree of the ICA as $d_i={\displaystyle {\sum }_{j=1}^N{a}_{ij}}$ and in-degree matrix as $D=diag\left\{d_i\right\}\in R^{N\times N}$. Then Laplacian matrix in network topology of multi-direction signal controller graph is $L=D-A$ .

The unknown nonlinear parameterized functions $f_i(k)$ and $f_r(k)$ satisfy the relation:

$$f_i(k)-f_r(k)=\frac{C(k)q_i(k)}{l_{i,\max }}-\frac{1}{4}C(k){\displaystyle \sum _{h=1}^4\frac{q_h(k)}{l_{h,\max }}}=\frac{1}{4}C(k){\displaystyle \sum _{h=1,h\ne i}^4\left(\frac{q_i(k)}{l_{i,\max }}-\frac{q_h(k)}{l_{h,\max }}\right)}$$

(18)

and consequently, satisfy the following inequality conditions by incorporating the parameter separation technique [28]:

$$\left|f_i(k)-f_r(k)\right|\le \left|{\phi }_i(k)\right|\left|{\theta }_i(k)\right|\left|e_i(k)\right|\le \overline{\phi }\left|{\theta }_i(k)\right|\left|e_i(k)\right|$$

(19)

where ${\theta }_i(k)$ denotes an unknown function related to vehicle arrival rate $q_i(k)$, ${\phi }_i(k)$ denotes a known function related to signal cycle $C(k)$, and meets $0<\left|{\phi }_i(k)\right|\le \overline{\phi }$. According to Equation (18), we choose ${\phi }_i(k)=\frac{1}{4}C(k)<\frac{1}{4}\overline{C}=\overline{\phi }$.

The global multi-direction and desired queuing strength as well as consensus coordinated error can be described as follows, respectively:

$$\mathit{x}(k+1)=\mathit{x}(k)+\mathit{f}(k)-\overline{\mathit{u}}(k)$$

(20)

$${\mathit{x}}_r(k+1)={\mathit{x}}_r(k)+{\mathit{f}}_r(k)-{\mathit{u}}_r(k)$$

(21)

$$\mathit{e}(k)=-(L+B)(\mathit{x}(k)-{\mathit{x}}_r(k))=-(L+B)\tilde{\mathit{x}}(k)$$

(22)

where $\mathit{x}={[x_1,\dots ,x_N]}^T\in R^N$ , ${\mathit{x}}_r=\underset{\_}{1}{x}_r\in R^N$, $\mathit{f}={[f_1,\dots ,f_4]}^T\in R^N$, ${\mathit{f}}_r=\underset{\_}{1}{f}_r\in R^N$, ${\mathit{u}}_r=\underset{\_}{1}{u}_r\in R^N$, $\overline{\mathit{u}}={[{\overline{u}}_1,\dots ,{\overline{u}}_N]}^T\in R^N$, $\mathit{e}={[e_1,\dots ,e_N]}^T\in R^N$, $\tilde{\mathit{x}}=\mathit{x}-{\mathit{x}}_r\in R^N$, $\underset{\_}{1}={[1,\dots ,1]}^T\in R^N$ are defined as N-Dimensional vector composed of element 1, and $B=diag(b_i)\in R^{N\times N}$ denotes a diagonal matrix.

Moreover, the difference dynamics of global consensus coordinated error is as follows:

$$\mathit{e}(k+1)=\mathit{e}(k)-(L+B)(\tilde{\mathit{x}}(k+1)-\tilde{\mathit{x}}(k))=\mathit{e}(k)-(L+B)(\mathit{f}(k)-{\mathit{f}}_r(k)+{\mathit{u}}_r(k)-\overline{\mathit{u}}(k))$$

(23)

Remark 1.

$\tilde{\mathit{x}}=\mathit{x}-{\mathit{x}}_r$is a centralized global coordinated error of queueing strength. We assume that at least one directional signal controller can obtain centralized global information $\tilde{\mathit{x}}$. The global coordinated errors e, rather than centralized global information is more suitable for Lyapunov-based distributed data-driven adaptive controller design.

Remark 2.

Considering that the digraph of each direction queueing strength at single intersection is strongly connected and at least one signal controller can communicate with others. Considering that the digraph contains a spanning tree for all directions at single intersection, that is to say, there is at least one $b_i\ne 0$, then$(L+B)$matrix is an irreducible principal diagonal$M$matrix and is a nonsingular matrix. All the poles of$M$matrix are distributed in the right half plane.

Remark 3.

Define that the eigenvalue of a matrix $M$can be expressed as$\sigma (M)$. Then$\overline{\sigma }(M)$and$\underset{\_}{\sigma }(M)$represent the maximum and minimum eigenvalues, respectively. Frobenius norm is defined as${\Vert M\Vert }_F=\sqrt{tr\left\{M^{T}M\right\}}$, where$tr\{\cdot \}$. represents the trace of a matrix. Frobenius inner product of two matrices is defined as ${\langle M_1,M_2\rangle }_F=\sqrt{tr\left\{M_1^T{M}_2\right\}}$.

For the convenience of the following discussion, we first discuss the three properties of saturation functions and two lemmas in multi-agent systems in terms of queuing strength model.

Lemma 1.

If a digraph is strongly connected and $B=diag\left\{b_i\right\}\ne 0$, then L of digraphs is irreducible and there is at least one positive diagonal element$b_i>0$in matrix$B$, then matrix$(L+B)$is a nonsingular$M$-matrix. Define:

$$\Vert \tilde{\mathit{x}}\Vert \le \Vert \mathit{e}\Vert /\underset{\_}{\sigma }(L+B)$$

(24)

where$\underset{\_}{\sigma }(L+B)$denotes the minimum eigenvalue of matrix$(L+B)$, when $\mathit{e}=0$, if and only if the queue strength in all directions reaches balance.

Lemma 2.

If the Laplacian matrix L of digraphs is irreducible and there is at least one positive diagonal element$b_i>0$in matrix$B$, then matrix$(L+B)$is a nonsingular$M$-matrix. Define:

$$q={[q_1,\cdots ,q_1]}^T={(L+B)}^{-1}\underset{\_}{1}$$

(25)

$$P=diag\left\{p_i\right\}\equiv diag\left\{1/q_i\right\}$$

(26)

then $P>0$, the definition of matrix $Q$ is:

$$Q={(L+B)}^{T}P(L+B)$$

(27)

then $Q>0$.

For $g,h^{*}\in R$, satisfying $\left|g\right|\le h^{*}$, then

Property 1

[5].

$${[g-sat(h,h^{*})]}^2\le {[g-h]}^2$$

(28)

Property 2

[5]. For $h=sat(g,h^{*})+d$, there are

$$\left|sat(h,h^{*})-h\right|\le \left|d\right|$$

(29)

Property 3

[29].

$$[(\gamma +1)g-(\gamma h+sat(h,h^{*}))][h-sat(h,h^{*})]\le 0$$

(30)

where $\gamma \ge 0$.

3.2. Data-Driven Distributed Adaptive Cooperative Control Design

Given the following standard definitions of multi-direction queueing strength balance for distributed cooperative control problem.

Definition 1.

The global consensus coordinated error  $\mathit{e}(k)\in R^N$ is uniformly ultimately bounded (UUB). If there exists a compact set $\mathsf{\Omega }\subset R^N$, so that $\forall \mathit{e}(k_0)\in \mathsf{\Omega }$, there exists a boundary $B$ and a time $k_t(B,\mathit{e}(k_0))$, which are independent of $k_0\ge 0$, then $\Vert \mathit{e}(k)\Vert \le B$ and $\forall k\ge k_0+k_t$ hold.

Definition 2.

Given the desired queueing strength model (16), whose state $x_r(k)$is uniformly ultimately bounded under coordination. If there exists a compact set$\mathsf{\Omega }\subset R^N$, so that$\forall (x_i(k_0)-x_r(k_0))\in \mathsf{\Omega }$, there exists a boundary$B$and a time$k_t(B,(x_i(k_0)-x_r(k_0)))$, which are independent of$k_0\ge 0$, then$\Vert x_i(k_0)-x_r(k_0)\Vert \le B$and$\forall i,\forall k\ge k_0+k_t$hold.

Considering the queuing strength model (15), the following data-driven queuing strength cooperative control strategy for multi-agent single intersection is constructed as follows:

$${\overline{u}}_i(k)=\frac{S_i{\overline{g}}_i(k)}{l_{i,\max }}=\frac{p_i{S}_i}{(p_i+\mu )l_{i,\max }}\left[{\overline{g}}_i(k-1)-{v^{\prime }}_i(k)\right]=\frac{p_i}{(p_i+\mu )}\left[{\overline{u}}_i(k-1)-v_i(k)\right]$$

(31)

where $v_i(k)=\frac{S_i}{l_{i,\max }}{v^{\prime }}_i(k)$ represents an arbitrary control signal designed by Lyapunov technology later. The global cooperative control strategy can be written as follows:

$$\overline{\mathit{u}}(k)={(1+\mu P^{-1})}^{-1}(\overline{\mathit{u}}(k-1)-\mathit{v}(k))$$

(32)

where $\mu >0$ denotes a learning gain, $\mathit{v}={[v_1,\dots ,v_N]}^T\in R^N$.

Theorem 1.

Considering the multi-agent based queuing strength system (15) and (16) at single intersection, digraph in network topology of multi-direction signal controllers is strongly connected, and there is at least one $b_i\ne 0$. According to the distributed consensus coordinated error (17), the distributed adaptive cooperative control term $v_i(k)$ is selected as follows:

$$v_i(k)=c{\displaystyle \sum _{j\in N^i}^{}{a}_{ij}(e_i-e_j)}+(c{b}_i+\overline{\phi }{\widehat{\theta }}_i(k))e_i(k)$$

(33)

where $c>0$ denotes the error learning gain.

The data-driven distributed adaptive cooperative control strategy for signal timing ${\overline{u}}_i(k)$ at single intersection is designed as follows:

$${\overline{u}}_i(k)=\frac{p_i}{(p_i+\mu )}\left[{\overline{u}}_i(k-1)-c{\displaystyle \sum _{j\in N^i}^{}{a}_{ij}(e_i-e_j)}-(c{b}_i+\overline{\phi }{\widehat{\theta }}_i(k))e_i(k)\right]$$

(34)

where $\mu >0$ denotes the cooperative control gain, $p_i>0$ , and the parameter learning law is designed as follows:

$${\widehat{\theta }}_i(k)=\frac{1}{1+\kappa F_i}\left[{\widehat{\theta }}_{i-1}(k)+\overline{\phi }{p}_i{F}_i{e}_i\left({\displaystyle \sum _{j\in N^i}^{}{a}_{ij}(e_i-e_j)+b_i{e}_i}\right)\right]$$

(35)

where $F_i={\Pi }_i>0$. $\kappa >0$ denotes parameter learning gain;

The control and parameter learning gains satisfy the following conditions:

$$c>\frac{{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)}{\underset{\_}{\sigma }(Q)}>0$$

(36)

$$\kappa >\frac{1}{2}c\underset{\_}{\sigma }(Q)+\frac{1}{2c}\left[{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)+\overline{\phi }\overline{\mathsf{\Theta }}\left(\overline{\sigma }(P)+\overline{\sigma }(Q)\right)\right]-\frac{1}{2}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)>0$$

(37)

$$\mu =\frac{1}{2}\left(c^2\underset{\_}{\sigma }(Q)+c(1-{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2)\overline{\sigma }(Q)+{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)\right)>\frac{c}{2}\overline{\sigma }(Q)>0$$

(38)

where $P=P^T\in R^{N\times N}>0$, $Q=Q^T\in R^{N\times N}>0$ are positive definite matrices defined in Lemma 2.

Then the distributed consensus coordinated error of queueing strength $e_i(k)$ of all directions is uniformly ultimately bounded (UUB) and converges to the desired queuing strength at single intersection. The upper bound of consensus coordinated error can be reduced by increasing the error learning gain $c$.

Remark 4.

It is worth noting that the proposed queuing strength balance control method is almost model-free. The traffic model parameters, such as the saturation flow rate, the vehicle arrival rate in model (8) are not used both in the distributed adaptive cooperative control (34) and the parameter learning law (35). Only the queueing length $l(k)$ of output data and network topology L and B of multi-direction signal controllers at signal intersection are used here to adjust the green time of input data. Especially, network topology information is easily obtained from adjacent relationship of traffic infrastructures and irrelevant to the real-traffic flow model. Therefore, the proposed MA-DD-DACC is data-driven.

4. Stability Analysis

Proof of Theorem 1.

Define a Lyapunov function candidate as follows:

$$V(k)=\frac{1}{2}{\mathit{e}}^T(k)P\mathit{e}(k)+\frac{1}{2}tr\left\{{\tilde{\mathsf{\Theta }}}^T(k-1)F^{-1}\tilde{\mathsf{\Theta }}(k-1)\right\}+\frac{1}{2c}{\widetilde{\overline{\mathit{u}}}}^T(k-1)P\widetilde{\overline{\mathit{u}}}(k-1)$$

(39)

where $F^{-1}\in R^{N\times N}>0$ is a diagonal matrix and $F=diag(F_i)$.

Difference of (39):

$$\Delta V(k+1)=V(k+1)-V(k)=\Delta V_1(k+1)+\Delta V_2(k)+\Delta V_3(k)$$

(40)

where:

$$\Delta V_1(k+1)=\frac{1}{2}{\mathit{e}}^T(k+1)P\mathit{e}(k+1)-\frac{1}{2}{\mathit{e}}^T(k)P\mathit{e}(k)$$

(41)

$$\Delta V_2(k)=\frac{1}{2}tr\left\{{\tilde{\mathsf{\Theta }}}^T(k)F^{-1}\tilde{\mathsf{\Theta }}(k)\right\}-\frac{1}{2}tr\left\{{\tilde{\mathsf{\Theta }}}^T(k-1)F^{-1}\tilde{\mathsf{\Theta }}(k-1)\right\}$$

(42)

$$\Delta V_3(k)=\frac{1}{2c}{\widetilde{\overline{\mathit{u}}}}^T(k)P\widetilde{\overline{\mathit{u}}}(k)-\frac{1}{2c}{\widetilde{\overline{\mathit{u}}}}^T(k-1)P\widetilde{\overline{\mathit{u}}}(k-1)$$

(43)

Equation (41) equals:

$$\begin{array}{cc}\hfill \Delta V_1(k+1)& =-{\mathit{e}}^T(k){(L+B)}^{T}P(\mathit{f}(k)-{\mathit{f}}_r(k)+{\mathit{u}}_r(k)-\overline{\mathit{u}}(k))\hfill \\ & +\frac{1}{2}{(\mathit{f}(k)-{\mathit{f}}_r(k)+{\mathit{u}}_r(k)-\overline{\mathit{u}}(k))}^{T}Q(\mathit{f}(k)-{\mathit{f}}_r(k)+{\mathit{u}}_r(k)-\overline{\mathit{u}}(k))\hfill \end{array}$$

(44)

where $Q={(L+B)}^{T}P(L+B)$.

According to (19), one has:

$$-{\mathit{e}}^T(k){(L+B)}^{T}P(\mathit{f}(k)-{\mathit{f}}_r(k))\le \overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\mathsf{\Theta }(k)\mathit{e}(k)$$

(45)

where $\mathsf{\Theta }(k)=diag(\left|{\theta }_i(k)\right|)$.

Based on Equations (32), (33) and (45), the first term at right-hand of (44) satisfies:

$$\begin{array}{l}-{\mathit{e}}^T(k){(L+B)}^{T}P(\mathit{f}(k)-{\mathit{f}}_r(k)+{\mathit{u}}_r(k)-\overline{\mathit{u}}(k))\hfill \\ \le -c{\mathit{e}}^T(k)Q\mathit{e}(k)+\overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\tilde{\mathsf{\Theta }}(k)\mathit{e}(k)-{\mathit{e}}^T(k){(L+B)}^{T}P(\mu P^{-1}\overline{\mathit{u}}(k)-\overline{\mathit{u}}(k-1)+{\mathit{u}}_r(k))\hfill \\ =\overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\widehat{\mathsf{\Theta }}(k)\mathit{e}(k)+\overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\tilde{\mathsf{\Theta }}(k)\mathit{e}(k)+{\mathit{e}}^T(k){(L+B)}^{T}P\widetilde{\overline{\mathit{u}}}(k)\hfill \end{array}$$

(46)

where $\tilde{\mathsf{\Theta }}(k)=diag({\tilde{\theta }}_i(k))=diag(\left|{\theta }_i(k)\right|-{\widehat{\theta }}_i(k))$, $\widetilde{\overline{\mathit{u}}}(k)=\overline{\mathit{u}}(k)-{\mathit{u}}_r(k)$.

Substituting (46) into (44):

$$\begin{array}{cc}\hfill \Delta V_1(k+1)& \le \overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\widehat{\mathsf{\Theta }}(k)\mathit{e}(k)+\overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\tilde{\mathsf{\Theta }}(k)\mathit{e}(k)\hfill \\ & +{\mathit{e}}^T(k){(L+B)}^{T}P\widetilde{\overline{\mathit{u}}}(k)+\frac{1}{2}{(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))}^{T}Q(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))\hfill \end{array}$$

(47)

Equation (42) can be rewritten as:

$$\begin{array}{cc}\hfill \Delta V_2(k)& =-tr\left\{{\tilde{\mathsf{\Theta }}}^T{F}^{-1}\left[\widehat{\mathsf{\Theta }}(k)-\widehat{\mathsf{\Theta }}(k-1)\right]\right\}-\frac{1}{2}tr\left\{{\left[\widehat{\mathsf{\Theta }}(k)-\widehat{\mathsf{\Theta }}(k-1)\right]}^T{F}^{-1}\left[\widehat{\mathsf{\Theta }}(k)-\widehat{\mathsf{\Theta }}(k-1)\right]\right\}\hfill \\ & \le -tr\left\{{\tilde{\mathsf{\Theta }}}^T{F}^{-1}\left[\widehat{\mathsf{\Theta }}(k)-\widehat{\mathsf{\Theta }}(k-1)\right]\right\}\hfill \end{array}$$

(48)

Given the parameter learning law as follows:

$${\widehat{\theta }}_i(k)=\frac{1}{1+\kappa F_i}\left[{\widehat{\theta }}_{i-1}(k)+\overline{\phi }{p}_i{F}_i{e}_i\left({\displaystyle \sum _{j\in N^i}^{}{a}_{ij}(e_i-e_j)+b_i{e}_i}\right)\right]$$

(49)

with global form:

$$\widehat{\mathsf{\Theta }}(k)={(I+\kappa F)}^{-1}\left[\widehat{\mathsf{\Theta }}(k-1)+F\overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\mathit{e}(k)\right]$$

(50)

It can be easily derived that

$$\widehat{\mathsf{\Theta }}(k)-\widehat{\mathsf{\Theta }}(k-1)=-\kappa F\widehat{\mathsf{\Theta }}(k)+F\overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\mathit{e}(k)$$

(51)

where $\kappa >0$ is the parameter learning gain.

Then we have:

$$\begin{array}{cc}\hfill \Delta V_1(k+1)+\Delta V_2(k)& \le \overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\widehat{\mathsf{\Theta }}(k)\mathit{e}(k)+{\mathit{e}}^T(k){(L+B)}^{T}P\widetilde{\overline{\mathit{u}}}(k)\hfill \\ & +tr\left\{\kappa {\tilde{\mathsf{\Theta }}}^T(k)(\mathsf{\Theta }(k)-\tilde{\mathsf{\Theta }}(k))\right\}+\frac{1}{2}{(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))}^{T}Q(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))\hfill \end{array}$$

(52)

Equation (43) can be further described as:

$$\begin{array}{cc}\hfill \Delta V_3(k)& =-\frac{1}{c}{(\overline{\mathit{u}}(k-1)-\overline{\mathit{u}}(k))}^{T}P(\overline{\mathit{u}}(k)-{\mathit{u}}_r)-\frac{1}{2c}{(\overline{\mathit{u}}(k)-\overline{\mathit{u}}(k-1))}^{T}P(\overline{\mathit{u}}(k)-\overline{\mathit{u}}(k-1))\hfill \\ & \le -\frac{1}{c}{(\overline{\mathit{u}}(k-1)-\overline{\mathit{u}}(k))}^{T}P\widetilde{\overline{\mathit{u}}}(k)-\frac{1}{2c}{\left[\mu P^{-1}\overline{\mathit{u}}(k)+(c(L+B)+\overline{\phi }\widehat{\mathsf{\Theta }}(k))\mathit{e}(k)\right]}^{T}P\hfill \\ & \times \left[\mu P^{-1}\overline{\mathit{u}}(k)+(c(L+B)+\overline{\phi }\widehat{\mathsf{\Theta }}(k))\mathit{e}(k)\right]\hfill \\ & \le -\frac{1}{c}{(\overline{\mathit{u}}(k-1)-\overline{\mathit{u}}(k))}^{T}P\widetilde{\overline{\mathit{u}}}(k)-\frac{\mu }{c}{\overline{\mathit{u}}}^T(k)(c(L+B)+\overline{\phi }\widehat{\mathsf{\Theta }}(k))\mathit{e}(k)-\frac{1}{2c}{\mathit{e}}^T(k)Q^{\prime }\mathit{e}(k)\hfill \end{array}$$

(53)

where $Q^{\prime }={(c(L+B)+\overline{\phi }\widehat{\mathsf{\Theta }}(k))}^{T}P(c(L+B)+\overline{\phi }\widehat{\mathsf{\Theta }}(k))$.

Combining Equations (52) and (53):

$$\begin{array}{cc}\hfill \Delta V(k+1)& \le \overline{\phi }{\mathit{e}}^T(k){(L+B)}^{T}P\widehat{\mathsf{\Theta }}(k)\mathit{e}(k)+tr\left\{\kappa {\tilde{\mathsf{\Theta }}}^T(k)(\mathsf{\Theta }(k)-\tilde{\mathsf{\Theta }}(k))\right\}\hfill \\ & +\frac{1}{2}{(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))}^{T}Q(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))+{\mathit{e}}^T(k){(L+B)}^{T}P\widetilde{\overline{\mathit{u}}}(k)\hfill \\ & -\frac{1}{c}{(\overline{\mathit{u}}(k-1)-\overline{\mathit{u}}(k))}^{T}P\widetilde{\overline{\mathit{u}}}(k)-\frac{\mu }{c}{\overline{\mathit{u}}}^T(k)(c(L+B)+\overline{\phi }\widehat{\mathsf{\Theta }}(k))\mathit{e}(k)-\frac{1}{2c}{\mathit{e}}^T(k)Q^{\prime }\mathit{e}(k)\hfill \end{array}$$

(54)

Rearranging terms in (54):

$$\begin{array}{cc}\hfill \Delta V(k+1)& \le -\frac{1}{2}{\mathit{e}}^T(k)\left(\frac{1}{c}{Q}^{\prime }-2\overline{\phi }{(L+B)}^{T}P\widehat{\mathsf{\Theta }}(k)\right)\mathit{e}(k)+tr\left\{\kappa {\tilde{\mathsf{\Theta }}}^T(k)(\mathsf{\Theta }(k)-\tilde{\mathsf{\Theta }}(k))\right\}\hfill \\ & +\frac{1}{2}{(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))}^{T}Q(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))\hfill \\ & -\frac{1}{c}\overline{\phi }{\mathit{e}}^T(k)\widehat{\mathsf{\Theta }}(k)P\widetilde{\overline{\mathit{u}}}(k)-\frac{\mu }{c}{(\widetilde{\overline{\mathit{u}}}(k)+{\mathit{u}}_r(k))}^T\widetilde{\overline{\mathit{u}}}(k)\hfill \end{array}$$

(55)

According to (19) again, one has:

$$\frac{1}{2}{(\mathit{f}(k)-{\mathit{f}}_r(k))}^{T}Q(\mathit{f}(k)-{\mathit{f}}_r(k))\le \frac{1}{2}{\overline{\phi }}^2{\mathit{e}}^T(k){\mathsf{\Theta }}^T(k)Q\mathsf{\Theta }(k)\mathit{e}(k)$$

(56)

and:

$${(\mathit{f}(k)-{\mathit{f}}_r(k))}^{T}Q({\mathit{u}}_r(k)-\overline{\mathit{u}}(k))\le \overline{\phi }\overline{\sigma }(Q)\Vert \mathsf{\Theta }(k)\Vert \Vert \mathit{e}(k)\Vert \Vert \widetilde{\overline{\mathit{u}}}(k)\Vert \le \overline{\phi }\overline{\sigma }(Q)\overline{\mathsf{\Theta }}\Vert \mathit{e}(k)\Vert \Vert \widetilde{\overline{\mathit{u}}}(k)\Vert$$

(57)

where $\overline{\mathsf{\Theta }}\ge \Vert \mathsf{\Theta }(k)\Vert$.

The third term at the right-hand of (55) becomes:

$$\begin{array}{l}\frac{1}{2}{(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))}^{T}Q(\mathit{f}(k)-{\mathit{f}}_r(k)-\widetilde{\overline{\mathit{u}}}(k))\hfill \\ \le \frac{1}{2}{\overline{\phi }}^2{\mathit{e}}^T(k){\mathsf{\Theta }}^T(k)Q\mathsf{\Theta }(k)\mathit{e}(k)+\overline{\phi }\overline{\sigma }(Q)\overline{\mathsf{\Theta }}\Vert \mathit{e}(k)\Vert \Vert \widetilde{\overline{\mathit{u}}}(k)\Vert +\frac{1}{2}{\widetilde{\overline{\mathit{u}}}}^T(k)Q\widetilde{\overline{\mathit{u}}}(k)\hfill \end{array}$$

(58)

Then Equation (55) is derived:

$$\begin{array}{cc}\hfill \Delta V(k+1)& \le -\frac{1}{2}{\mathit{e}}^T(k)\left(\frac{1}{c}{Q}^{\prime }-2\overline{\phi }{(L+B)}^{T}P\widehat{\mathsf{\Theta }}(k)\right)\mathit{e}(k)+tr\left\{\kappa {\tilde{\mathsf{\Theta }}}^T(k)(\mathsf{\Theta }(k)-\tilde{\mathsf{\Theta }}(k))\right\}\hfill \\ & -\frac{1}{c}\overline{\phi }{\mathit{e}}^T(k)\widehat{\mathsf{\Theta }}(k)P\widetilde{\overline{\mathit{u}}}(k)-\frac{\mu }{c}{(\widetilde{\overline{\mathit{u}}}(k)+{\mathit{u}}_r(k))}^T\widetilde{\overline{\mathit{u}}}(k)\hfill \\ & +\frac{1}{2}{\overline{\phi }}^2{\mathit{e}}^T(k){\mathsf{\Theta }}^T(k)Q\mathsf{\Theta }(k)\mathit{e}(k)+\overline{\phi }\overline{\sigma }(Q)\overline{\mathsf{\Theta }}\Vert \mathit{e}(k)\Vert \Vert \widetilde{\overline{\mathit{u}}}(k)\Vert +\frac{1}{2}{\widetilde{\overline{\mathit{u}}}}^T(k)Q\widetilde{\overline{\mathit{u}}}(k)\hfill \end{array}$$

(59)

By using $\Vert \tilde{\mathsf{\Theta }}(k)\Vert \le {\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F$, $\Vert \mathsf{\Theta }(k)\Vert \le \Vert \widehat{\mathsf{\Theta }}(k)\Vert \le \overline{\mathsf{\Theta }}$, (59) is:

$$\begin{array}{cc}\hfill \Delta V(k+1)& \le -\frac{1}{2}\left(c\underset{\_}{\sigma }(Q)+\frac{1}{c}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)-{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)\right){\Vert \mathit{e}(k)\Vert }^2\hfill \\ & +\kappa \overline{\mathsf{\Theta }}{\Vert \tilde{\mathsf{\Theta }}\Vert }_F-\kappa {\Vert \tilde{\mathsf{\Theta }}\Vert }_F^2+\frac{1}{c}\overline{\phi }\overline{\mathsf{\Theta }}\Vert \mathit{e}(k)\Vert \left(\overline{\sigma }(P)+\overline{\sigma }(Q)\right)\Vert \widetilde{\overline{\mathit{u}}}(k)\Vert \hfill \\ & -\left(\frac{\mu }{c}I-\frac{1}{2}Q\right){\Vert \widetilde{\overline{\mathit{u}}}(k)\Vert }^2+\mu {\overline{\mathit{u}}}_r\Vert \widetilde{\overline{\mathit{u}}}(k)\Vert \hfill \end{array}$$

(60)

which could be rewritten as:

$$\begin{array}{l}\Delta V(k+1)\le -\left[\begin{array}{ccc}\Vert \mathit{e}(k)\Vert & {\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F& \Vert \widetilde{\overline{\mathit{u}}}(k)\Vert \end{array}\right]\hfill \\ \left[\begin{array}{ccc}\frac{1}{2}\left(c\underset{\_}{\sigma }(Q)+\frac{1}{c}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)-{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)\right)& 0& -\frac{1}{2c}\overline{\phi }\overline{\mathsf{\Theta }}\left(\overline{\sigma }(P)+\overline{\sigma }(Q)\right)\\ 0& \kappa & 0\\ -\frac{1}{2c}\overline{\phi }\overline{\mathsf{\Theta }}\left(\overline{\sigma }(P)+\overline{\sigma }(Q)\right)& 0& \left(\frac{\mu }{c}-\frac{1}{2}\overline{\sigma }(Q)\right)\end{array}\right]\left[\begin{array}{c}\Vert \mathit{e}(k)\Vert \\ {\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F\\ \Vert \widetilde{\overline{\mathit{u}}}(k)\Vert \end{array}\right]\hfill \\ +\left[\begin{array}{ccc}0& \kappa \overline{\mathsf{\Theta }}& \mu {\overline{\mathit{u}}}_r\end{array}\right]\left[\begin{array}{c}\Vert \mathit{e}(k)\Vert \\ {\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F\\ \Vert \widetilde{\overline{\mathit{u}}}(k)\Vert \end{array}\right]=-z^{T}Rz+{\mathit{r}}^{T}z\hfill \end{array}$$

(61)

If $R$ is a positive definite matrix, and:

$$\Vert z\Vert >\frac{\Vert \mathit{r}\Vert }{\underset{\_}{\sigma }(R)}$$

(62)

then $\Delta V(k+1)\le 0$.

Based on (39), we have:

$$\frac{1}{2}\underset{\_}{\sigma }(P){\Vert \mathit{e}\Vert }^2+\frac{1}{2{\Pi }_{\max }}{\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F^2+\frac{1}{2}\underset{\_}{\sigma }(P){\Vert \tilde{\mathit{u}}\Vert }^2\le V\le \frac{1}{2}\overline{\sigma }(P){\Vert \mathit{e}\Vert }^2+\frac{1}{2{\Pi }_{\min }}{\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F^2+\frac{1}{2}\overline{\sigma }(P){\Vert \tilde{\mathit{u}}\Vert }^2$$

(63)

then:

$$\begin{array}{l}\frac{1}{2}\left[\begin{array}{ccc}\Vert \mathit{e}(k)\Vert & {\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F& \Vert \tilde{\mathit{u}}(k)\Vert \end{array}\right]\left[\begin{array}{ccc}\underset{\_}{\sigma }(P)& 0& 0\\ \ast & \frac{1}{{\Pi }_{\max }}& 0\\ \ast & \ast & \underset{\_}{\sigma }(P)\end{array}\right]\left[\begin{array}{c}\Vert \mathit{e}\Vert \\ {\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F\\ \Vert \tilde{\mathit{u}}\Vert \end{array}\right]\le V\hfill \\ \le \frac{1}{2}\left[\begin{array}{ccc}\Vert \mathit{e}(k)\Vert & {\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F& \Vert \tilde{\mathit{u}}(k)\Vert \end{array}\right]\left[\begin{array}{ccc}\overline{\sigma }(P)& 0& 0\\ \ast & \frac{1}{{\Pi }_{\min }}& 0\\ \ast & \ast & \overline{\sigma }(P)\end{array}\right]\left[\begin{array}{c}\Vert \mathit{e}(k)\Vert \\ {\Vert \tilde{\mathsf{\Theta }}(k)\Vert }_F\\ \Vert \tilde{\mathit{u}}(k)\Vert \end{array}\right]\hfill \end{array}$$

(64)

Since:

$$\frac{1}{2}{z}^T\underset{\_}{S}z\le V\le \frac{1}{2}{z}^T\overline{S}z$$

(65)

where $\underset{\_}{S}=\left[\begin{array}{ccc}\underset{\_}{\sigma }(P)& 0& 0\\ \ast & \frac{1}{{\Pi }_{\max }}& 0\\ \ast & \ast & \underset{\_}{\sigma }(P)\end{array}\right]$, $\overline{S}=\left[\begin{array}{ccc}\overline{\sigma }(P)& 0& 0\\ \ast & \frac{1}{{\Pi }_{\min }}& 0\\ \ast & \ast & \overline{\sigma }(P)\end{array}\right]$, and then

$$\frac{1}{2}\underset{\_}{\sigma }(\underset{\_}{S}){\Vert z\Vert }^2\le V\le \frac{1}{2}\overline{\sigma }(\overline{S}){\Vert z\Vert }^2$$

(66)

It can be obtained that:

$$V>\frac{1}{2}\frac{\overline{\sigma }(\overline{S}){\Vert \mathit{r}\Vert }^2}{{\underset{\_}{\sigma }}^2(R)}$$

(67)

which leads to Equation (62).

In Equation (61), we define the matrix $R$ as follows:

$$R=\left[\begin{array}{ccc}{a}_1& 0& a_3\\ 0& b_1& 0\\ a_3& 0& c_1\end{array}\right]$$

(68)

where $a_1=\frac{1}{2}\left(c\underset{\_}{\sigma }(Q)+\frac{1}{c}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)-{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)\right)$, $a_3=-\frac{1}{2c}\overline{\phi }\overline{\mathsf{\Theta }}\left(\overline{\sigma }(P)+\overline{\sigma }(Q)\right)$, $b_1=\kappa$, $c_1=\left(\frac{\mu }{c}-\frac{1}{2}\overline{\sigma }(Q)\right)$.

Firstly, select the learning gain satisfying:

$$c\underset{\_}{\sigma }(Q)+\frac{1}{c}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)-{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)>0$$

(69)

$$\kappa >0$$

(70)

$$\mu >\frac{c}{2}\overline{\sigma }(Q)$$

(71)

such that $R>0$.

Then the characteristic polynomial of matrix $R$ is given as:

$$(\lambda -a_1)(\lambda -b_1)(\lambda -c_1)-a_3^2(\lambda -b_1)=0$$

(72)

we obtain all the eigenvalues as:

$${\lambda }_1=b_1=\kappa$$

(73)

$${\lambda }_2=\frac{(a_1+c_1)+\sqrt{{(a_1-c_1)}^2+4a_3^2}}{2}$$

(74)

$${\lambda }_3=\frac{(a_1+c_1)-\sqrt{{(a_1-c_1)}^2+4a_3^2}}{2}$$

(75)

Let $\mu =\frac{1}{2}\left(c^2\underset{\_}{\sigma }(Q)+c(1-{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2)\overline{\sigma }(Q)+{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)\right)$, then:

$$a_1-c_1=\frac{1}{2}\left(c\underset{\_}{\sigma }(Q)+\frac{1}{c}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)-{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)\right)-\left(\frac{\mu }{c}-\frac{1}{2}\overline{\sigma }(Q)\right)=0$$

(76)

Now the third eigenvalue is:

$$\begin{array}{cc}\hfill {\lambda }_3& =\frac{a_1+c_1-2a_3}{2}=a_1-a_3\hfill \\ & =\frac{1}{2}c\underset{\_}{\sigma }(Q)+\frac{1}{2c}\left[{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)+\overline{\phi }\overline{\mathsf{\Theta }}\left(\overline{\sigma }(P)+\overline{\sigma }(Q)\right)\right]-\frac{1}{2}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)\hfill \end{array}$$

(77)

Select an appropriate $\kappa$ , such that:

$$\kappa >\frac{1}{2}c\underset{\_}{\sigma }(Q)+\frac{1}{2c}\left[{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)+\overline{\phi }\overline{\mathsf{\Theta }}\left(\overline{\sigma }(P)+\overline{\sigma }(Q)\right)\right]-\frac{1}{2}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)>0$$

(78)

We have:

$$\underset{\_}{\sigma }(R)={\lambda }_3=\frac{1}{2}c\underset{\_}{\sigma }(Q)+\frac{1}{2c}\left[{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)+\overline{\phi }\overline{\mathsf{\Theta }}\left(\overline{\sigma }(P)+\overline{\sigma }(Q)\right)\right]-\frac{1}{2}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)$$

(79)

and error and parameter learning gains satisfy:

$$c>\frac{{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)}{\underset{\_}{\sigma }(Q)}>0$$

(80)

$$\kappa >\frac{1}{2}c\underset{\_}{\sigma }(Q)+\frac{1}{2c}\left[{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)+\overline{\phi }\overline{\mathsf{\Theta }}\left(\overline{\sigma }(P)+\overline{\sigma }(Q)\right)\right]-\frac{1}{2}{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\overline{\sigma }(Q)>0$$

(81)

$$\mu =\frac{1}{2}\left(c^2\underset{\_}{\sigma }(Q)+c(1-{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2)\overline{\sigma }(Q)+{\overline{\phi }}^2{\overline{\mathsf{\Theta }}}^2\underset{\_}{\sigma }(P)\right)>\frac{c}{2}\overline{\sigma }(Q)$$

(82)

As a result, 𝔃(k) is UUB [30]. □

5. Simulation

In this paper, a VISSIM-VB-MATLAB joint simulation platform is used to build a more realistic experiment. VISSIM is a professional traffic simulation software that simulates all modes of traffic and analyses their interactions. It consists of a traffic simulator and a signal state generator, which exchange detector data and signal state information through an interface. It can not only generate visualized traffic conditions online, but also output various realistic and statistical data offline, such as travel time, queue length, etc. The simulation data obtained by VISSIM is transferred into MATLAB through VB. In MATLAB, the MA-DD-DACC signal timing algorithm is used to realize the cooperative control of queue strength in four directions at single intersection. A single intersection is build by using the traffic evaluation module provided by VISSIM to output traffic evaluation data as shown in Figure 3.

Using multi-directional queuing strength model with changeable cycle at intersection (15), the simulation is conducted. The initial parameters are shown in

Table 1. Simulation parameter table of queuing strength cooperative control at single intersection.

	East	North	West	South
Arrival flow (q/pcu*h−1)	250	250	250	250
Initial cycle (Co/s)	128
Initial queuing strength (Lm(0)/pcu)	10	20	30	40
Saturation flow rate (s/pcu*h−1)	1200	1200	1200	1200
Initial green time(g/s)	10	10	10	10
Lost time (tl/s)	2	2	2	2
The maximum queue value (lmax/pcu)	100	200	300	400
The maximum green time (gmax/s)	70	70	70	70
The minimum green time (gmin/s)	10	10	10	10
Quotiety (alpha)	0.99	0.99	0.99	0.99

.

Two experiments are carried out. The first one uses the UNCFSCS [26] which is designed to release vehicles in fixed sequence at unsaturated intersections and to keep all phases completely empty. The second one adopts the MA-DD-DACC signal timing strategies presented in this paper. The total queuing delays in all directions will be recorded at the end of each cycle in the simulation running process, as shown in Figure 4. It is not difficult to find out that the queuing delays of all directions in UNCFSCS decreases eventually to zero and to keep all phases completely empty under the premise that undersaturation. While the proposed cooperative control scheme can adjust the cycle and green ratio adaptively for different traffic conditions, undersaturation, saturation and supersaturation, respectively. It should be noted that the total delays in UNCFSCS is almost six times that in MA-DD-DACC.

From Figure 5, the queueing strength of south direction in UNCFSCS is more than 80. However, the queueing strength of all directions in MA-DD-DACC is only less than 50. By calculation, the queueing strength mean value of four directions in MA-DD-DACC has dropped from 25 to 10 during 2190 s. While, the queueing strength mean value in UNCFSCS has dropped from 35 to 10 during 2731 s. It is obvious that the queueing strength balance is achieved which implies the proposed scheme has the merit of high convergence rate and low energy consumption as displayed.

The VISSIM delay evaluation index is obtained from two simulations. It can be found that green time of each direction could be effectively adjusted by using the MA-DD-DACC such that the average vehicle delay is balanced. The performance in average delays of our proposed control is much better than that of the UNCFSCS control, as shown in Figure 6. Furthermore, the fuel consumption of MA-DD-DACC is less than that of the UNCFSCS as displayed in Figure 7. The experimental results verify the validity of the proposed MA-DD-DACC.

6. Conclusions

This work proposed a novel MA-DD-DACC method to deal with urban traffic signal timing problem concerning phase conflict, green time constraints, changeable cycle, and network topology of multi-direction signal controllers, which achieved a fairly good performance for queuing strength balance under different kind of traffic conditions, as shown in a VISSIM-VB-MATLAB joint simulation platform. Although much future research needs to be done, it can be expected that data-driven corporative control methodology in a distributed manner for urban traffic multi-agent systems will provide a powerful tool for realizing D²ITS with the goal of reducing traffic congestion and energy consumption.