AI-Driven Consensus: Modeling Multi-Agent Networks with Long-Range Interactions Through Path-Laplacian Matrices

Abstract

Extended connectivity in graphs can be analyzed through k-path Laplacian matrices, which permit the capture of long-range interactions in various real-world networked systems such as social, transportation, and multi-agent networks. In this work, we present several alternative methods based on machine learning methods (LSTM, xLSTM, Transformer, XGBoost, and ConvLSTM) to predict the final consensus value based on directed networks (Erdös–Renyi, Watts–Strogatz, and Barabási–Albert) and on the initial state. We highlight how different k-hop interactions affect the performance of the tested methods. This framework opens new avenues for analyzing multi-scale diffusion processes in large-scale, complex networks.

1. Introduction

Consensus algorithms are protocols allowing an agent network to agree on certain quantities of interest. Early work on consensus in social systems was introduced by DeGroot [1], who modeled individuals iteratively updating their opinions to the average of their neighbors’ opinions. However, it is known that other individuals who are not neighbors are also influential in people’s opinions [2] and behaviour [3,4]. In control theory, consensus gained prominence for coordinating multi-agent systems (e.g., fleets of autonomous vehicles or sensor networks) [5,6]. How this consensus is reached heavily depends on the initial state and network structure.

Let $G=(V,E)$ be a network with V as the set of nodes and E as the set of nodes. If A is the adjacency matrix of G and D is its degree matrix (a diagonal matrix with the degree of the node $v_i\in V$ in the position $d_{ii}$), the Laplacian matrix is defined as $L=D-A$. Laplacian matrices and their spectral properties have been extensively studied due to their appeal when studying graph connectivity, spreading, and community detection [7,8]. This facilitates measuring robustness, spectral clustering, and partition of networks.

While a Laplacian matrix captures direct (one-hop) connections connecting adjacent nodes by an edge, Estrada extended the notion of adjacency to include paths of length k between nodes generalized to incorporate longer path-based connections, rather than only immediate neighbors by introducing k-path Laplacian matrices [9]. These matrices are constructed from k-path adjacency matrices that count the shortest paths of length k between any pair of nodes, providing a multi-hop representation of the network connectivity. We recall that the case of $k=1$ returns classical Laplacian matrices. If the multiplicity of the zero eigenvalue of $L^k$ is one, then the network is k-path connected (analogous to one-hop connectivity for the standard Laplacian matrix). These path-based extensions provide a more global view of connectivity beyond immediate neighbors.

The introduction of path-based Laplacians permits the incorporation of long-range connections and the study of how they impact dynamic network processes. This connectivity extension is particularly relevant for complex networks such as social [10] or transportation systems [11], where indirect connections (friends-of-friends in a social network or multi-hop routes in a transportation network) significantly influence network behavior.

For example, considering $k>1$, we can accelerate synchronization and consensus in networks by effectively reducing distances between nodes [12,13]. Additionally, by weighting connections along paths, path-Laplacian matrices generalize a foundational graph-theoretic concept and provide a richer framework to study connectivity and diffusion processes on networks.

In this work, we introduce several machine learning methods for estimating the consensus in networks comprising long-range interactions through k-path-Laplacian matrices instead of using numerical methods. We study three different types of networks: Erdös–Renyi [14], Watts–Strogatz [15], and Barabási–Albert [16]. We also test several machine learning methods to estimate the consensus finally obtained in these networks as time passes.

The manuscript is organized as follows: In Section 2, we review some aspects on mechanisms on multi-agent systems modeled as networks. Useful machine learning models to study dynamics on networks, mainly based on recurrent neural networks, are described in Section 3. The mathematical formalism of consensus through path-Laplacian matrices for directed and undirected networks is shown in Section 4. In Section 5, we describe the data generation for the process which will be used for training, validation, and testing the models. We describe the performance of the models in Section 6 and we outline the conclusions in Section 7.

2. Consensus Mechanisms in Multi-Agent Systems

Multi-agent systems can be modeled through a network in which agents interacting with each other are directly connected by an edge. Connectivity is a crucial property for reaching a consensus in a network [6,17,18,19]. These foundational works lay on the idea that graph connectivity (often quantified by properties of the Laplacian matrix L) is a crucial condition for consensus. For instance, if the agents’ communication graph has a spanning tree across time, the agents’ states will converge to a common value. Connectivity can also be linked to spectral theory. The second-smallest eigenvalue of L, known as the Fiedler value or algebraic connectivity, relates to how well the graph is connected [20]. In addition, continuous or, at least, frequent enough connectivity is also required to reach the consensus state. For static (fixed-topology) networks, consensus algorithms can be analyzed directly using Laplacian matrices. The standard first-order consensus update is given by

$$\mathbf{x}(t+1)=\mathbf{x}\left(t\right)-ϵL\mathbf{x}\left(t\right);(\mathrm{discrete}\mathrm{time}),\mathrm{or}$$

(1)

$$\dot{\mathbf{x}}\left(t\right)=-ϵL\mathbf{x}\left(t\right);(\mathrm{continuous}\mathrm{time}),$$

(2)

where L is the Laplacian matrix of the network, which drives all agent states $\mathbf{x}$ to the agreement (consensus) on an average initial value when the graph is connected.

The Fiedler value also provides the convergence rate to consensus; that is, a larger Fiedler eigenvalue yields faster consensus [21].

Beyond one-hop (nearest-neighbor) interactions, recent studies consider whether leveraging multi-hop information can improve consensus speed or resilience. Traditional consensus uses only direct neighbor states; however, nodes in many real networks can also be indirectly influenced by more distant nodes (e.g., a friend-of-a-friend in a social network). Path-based Laplacians have been applied to consensus dynamics to capture such influences. Estrada [9] showed that allowing agents to utilize not only their immediate neighbors’ states but also the states of nodes at a distance k (via the k-path Laplacian) can accelerate agreement; see also [12,13,22].

Long-range interactions provide additional communication “shortcuts” in the network, speeding up information spread and consensus formation. This concept has been extended in control protocols [23]. Such approaches are particularly relevant in systems where influence extends beyond immediate contacts. In opinion dynamics, models incorporating second- or third-neighborhood influences (sometimes called “social power” or indirect influence) help explain how global consensus or polarization can emerge [24].Therefore, as network connectivity drives consensus, it is natural to consider how modern data-driven methods can further capture and predict such complex dynamics.

3. Machine Learning Models for Studying Dynamics on Networks

Modeling the dynamics of processes on networks often requires capturing both temporal patterns and network structure. In the past decade, researchers have increasingly turned to machine learning—and particularly deep learning models—to model and predict network dynamics. Recurrent neural networks (RNNs), especially Long Short-Term Memory (LSTM) networks [25], have shown notable success in forecasting traffic flow and travel speeds by effectively learning long-range temporal dependencies [26]. Similarly, LSTMs have been applied to social networks to predict the evolution of user engagement and information diffusion over time, leveraging their strength in sequence modeling to anticipate bursts of activity or link formations. Complementary methods such as eXtreme Gradient Boosting (e.g., XGBoost [27]), consisting of the use of ensembles of decision trees to model nonlinear relationships, have been applied either as standalone models or in hybrid schemes with LSTMs to refine predictions.

More recently, the Transformer architecture [28] has revolutionized sequence modeling by leveraging self-attention to capture both local and global interactions—yielding improved performance in traffic forecasting and social-interaction modeling [29]. Transformers rely on self-attention mechanisms to capture relationships in sequential data, either temporal patterns or correlations between locations, and they are highly parallelizable, enabling very large models. In social-network analysis, Transformers (and their variant, graph Transformers) have been employed to model sequences of interactions or events (like sequences of posts or the evolution of relationships), benefiting from attention to identify critical earlier interactions that influence later outcomes [30].

Hybrid approaches combine spatial feature extraction with temporal dynamics, including convolutional Transformers [31,32] and convolutional LSTM networks [33,34,35]. The idea behind them is to use convolutional layers to capture local spatial structure (for example, nearby roads on a map or local clusters in a network) and Transformer/LSTM layers to capture broader contextual or temporal patterns. Variants like graph-structured LSTMs (applying ConvLSTM, also known as CNN-LSTM, ideas to graph adjacency instead of image grids) have been developed to handle network data more directly [36]. Similarly, for social networks, one could imagine using CNNs on graph-structured data (via graph convolution operations) combined with Transformers for temporal sequences to model phenomena like epidemic spreading or information cascades with high fidelity [37].

Finally, emerging models such as Extended LSTM (xLSTM) [38] push the limits of recurrent architectures by introducing modifications such as exponential memory gating or parallelizable memory units. Early experiments in sequence modeling (primarily language modeling benchmarks) suggest that, with proper scaling and gating enhancements, xLSTMs can achieve competitive performance to Transformers [38] without the training difficulties that vanilla LSTMs face. Robust synthetic data generation becomes indispensable to effectively evaluate and validate these machine learning methods, as detailed in the following section.

4. Mathematical Formalism of Consensus Through Path-Laplacian Matrices

Consider a simple undirected network $G=(V,E)$ with $n=\left|V\right|$ nodes (vertices) and $m=\left|E\right|$ edges. Two nodes $i,j\in V$ are said to be adjacent if an edge $(i,j)\in E$ is connecting them. The adjacency matrix A of G is the $n\times n$ matrix with entries $A_{ij}=1$ if $(i,j)\in E$ (and $A_{ij}=0$ otherwise). Let $deg\left(i\right)$ denote the degree of node i (number of neighbors). The diagonal degree matrix is $D=diag(deg(1),\cdots ,deg(n\left)\right)$. We recall that the Laplacian matrix is defined as

$$L=D-A,$$

(3)

which is a symmetric positive semi-definite matrix of size $n\times n$ that has real, non-negative eigenvalues $0={\lambda }_1\le {\lambda }_2\le \cdots \le {\lambda }_n$ [39,40]. When modeling, we assume that a node represents an agent, and its adjacent nodes will be connected to it with edges. Classical consensus models assume that each agent is influenced only by its nearest neighbors (direct connections). In what follows, we introduce a generalized Laplacian formalism that allows for incorporating longer-range interactions beyond immediate neighbors. This generalization is based on graph paths and will lead to an extended family of Laplacian matrices capturing k-hop relationships in the network.

4.1. Path-Based Laplacian Matrices

A path of length k in G is a sequence of $k+1$ distinct vertices $(v_{i_0},v_{i_1},\cdots ,v_{i_k})$ such that each consecutive pair $(v_{i_{r-1}},v_{i_r})$ is an edge in E. The distance $d(i,j)$ between two nodes i and j is defined as the length of the shortest path connecting i to j. Let $d_{max}$ denote the graph’s diameter (the maximum distance over all node pairs). For each integer k with $1\le k\le d_{max}$, we define the k-path adjacency matrix, denoted $P_k$, as the $n\times n$ (symmetric) matrix with entries

$${\left(P_k\right)}_{ij}=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}d(i,j)=k,\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(4)

In other words, $P_k(i,j)=1$ if and only if there exists at least one shortest length path k between node i and node j. This matrix generalizes the ordinary adjacency matrix ($P_1=A$) to connections via longer paths. We also define the k-path degree of node i as

$${\delta }_k\left(i\right)=\sum _{j=1}^n{\left(P_k\right)}_{ij}.$$

(5)

i.e., the number of nodes at distance exactly k from i in the network [9]. By construction, ${\delta }_1\left(i\right)=deg\left(i\right)$ is the usual degree. We introduce the k-path Laplacian matrix $L_k$ of the network G as follows [9]:

$${\left(L_k\right)}_{ij}=\left\{\begin{array}{cc}-1,\hfill & \mathrm{if}d(i,j)=k,\hfill \\ {\delta }_k\left(i\right),\hfill & \mathrm{if}i=j,\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(6)

Equivalently, we can write $L_k=D_k-P_k$, where $D_k=diag({\delta }_k\left(1\right),\cdots ,{\delta }_k\left(n\right))$ is the diagonal matrix of k-path degrees. For $k=1$, $L_1=D_1-P_1=D-A$, which is exactly the classical Laplacian L. Each $L_k$ is symmetric and positive semi-definite. In fact, one can verify that for any real vector $y\in {\mathbb{R}}^n$,

$$y^{\top }{L}_{k}y={\displaystyle \frac{1}{2}}\sum _{\begin{array}{c}i,j\in V\\ d(i,j)=k\end{array}}{\left(y_i-y_j\right)}^2\ge 0.$$

confirming that all eigenvalues of $L_k$ are non-negative. Moreover, $L_k$ shares the zero-eigenvalue property of the standard Laplacian: since each off-diagonal $-1$ in row i of $L_k$ is balanced by the contribution to the diagonal entry ${\delta }_k\left(i\right)$, it follows that $L_k\mathbf{1}=0$. Thus, 0 is an eigenvalue of $L_k$ with eigenvector $\mathbf{1}={[1,1,\dots ,1]}^T$ for every k. The multiplicity of this zero eigenvalue reflects the number of disconnected components in the graph when considering only k-distance connections. In particular, if the network G is such that any node can reach any other node through a sequence of hops of length $\le k$ (sometimes said to be k-path connected’, then $L_k$ has a unique zero eigenvalue. In general, m zero eigenvalues of $L_k$ indicate that the graph can be partitioned into m disjoint subgraphs that are not linked by any k-length path. This is a natural generalization of the connectivity interpretation of the ordinary Laplacian ($L_1$). The family $L_k:1\le k\le d_{max}$ thus provides a spectrum of Laplacian-like operators, each capturing network connectivity at a different path-length scale. Importantly, higher values of k account for more “long-range” links in the network, and as we will see, incorporating such long-range interactions can significantly impact dynamic processes like consensus.

4.2. Consensus Dynamics on Undirected Networks

We start reviewing the standard consensus process on an undirected network and then describe its extension using the multi-hop Laplacians introduced above. Consider an undirected network of n agents (nodes) whose goal is to reach agreement on some vector ${\mathit{\varphi }}_0={[{\varphi }_{01},\dots ,{\varphi }_{0n}]}^T\in {\mathbb{R}}^n$. Let ${\varphi }_i\left(t\right)$ denote the state (value of $\varphi$) held by agent i at time t. In a consensus algorithm, each agent updates its state by interacting with its neighbors, gradually reducing differences between agents’ values. We distinguish two common frameworks: continuous-time and discrete-time consensus dynamics. Later, we will see how to introduce multi-hop interactions.

4.2.1. Continuous-Time Consensus

In the first-order continuous-time model, each agent i adjusts its state with a rate proportional to the discrepancies with its neighbors’ states. This is described by the following system of ordinary differential equations

$${\dot{\varphi }}_i\left(t\right)=\sum _{j\in V_i}\left({\varphi }_j\left(t\right)-{\varphi }_i\left(t\right)\right),$$

(7)

where $V_i$ denotes the set of neighbors of node i (all $1\le j\le n$, $i\ne j$, such that $(i,j)\in E$). Equivalently, in vector form, one can write

$${\displaystyle \frac{d\mathit{\varphi }\left(t\right)}{dt}}=-L\mathit{\varphi }\left(t\right).$$

(8)

where $\mathit{\varphi }\left(t\right)={[{\varphi }_1\left(t\right),\cdots ,{\varphi }_n\left(t\right)]}^{\top }$ and L is the Laplacian of the network [21]. Equation (8) is a linear system whose solution is $\mathit{\varphi }\left(t\right)=e^{-Lt}·\mathit{\varphi }\left(0\right)$. For a connected graph, L has a single zero eigenvalue and all other eigenvalues ${\lambda }_2,\cdots ,{\lambda }_n>0$. In this case, the system (8) drives the agents to a consensus: ${\varphi }_i\left(t\right)$ converges to ${\varphi }_{0i}$ as $t\to \infty$ for all i. Indeed, one can show ${lim}_{t\to \infty }\mathit{\varphi }\left(t\right)=c·\mathbf{1}$, where c is a constant equal to the average of the initial states ${\varphi }_i\left(0\right)$ [21,41]. This average is an invariant of (8) because ${\mathbf{1}}^{\top }L=\mathbf{0}$, so ${\displaystyle \frac{d}{dt}}{\sum }_i{\varphi }_i\left(t\right)={\mathbf{1}}^{\top }\dot{\mathit{\varphi }}\left(t\right)=-{\mathbf{1}}^{\top }L\mathit{\varphi }\left(t\right)=\mathbf{0}$. The eigenvalues of L govern the convergence rate of continuous consensus; in particular, the algebraic connectivity) determines the slowest decay mode and hence the asymptotic convergence speed [7,21].

4.2.2. Discrete-Time Consensus

In discrete time, consensus is achieved by iterative averaging. At each time step, $t\to t+1$, every agent $v_i$ updates its state as a convex combination of its own and its neighbors’ previous states ($v\in V_i$). A common discrete model can be formulated as

$${\varphi }_i(t+1)={\varphi }_i\left(t\right)+\epsilon \sum _{j\in V_i}\left({\varphi }_j\left(t\right)-{\varphi }_i\left(t\right)\right),$$

(9)

where $0<\epsilon <1$ is a small step size, also interpreted as an interaction weight [21]. In matrix form, this update can be expressed as

$$\mathit{\varphi }(t+1)=\left(I-\epsilon L\right)\mathit{\varphi }\left(t\right),$$

(10)

where I is the identity of order n. Let $W=I-\epsilon L$ denote the update’s weight matrix. Other generalizations of diffusion that include subdiffusion and super diffusion in networks were stated in [42].

By construction, each row of W sums to 1 and $W_{ij}>0$ if $j\in V_i$ (with $W_{ii}>0$ as well). If $0<\epsilon <1/{deg}_{max}$, where ${deg}_{max}$ is the maximum degree in the network, then W has all non-negative entries and no entry exceeds 1, meaning that W is a stochastic matrix. In such a matrix, only the rows are normalized to sum to 1. On the one hand, if W is doubly stochastic, that is, both its rows and its columns sum to 1, then the average of the states is invariant under the update, that is ${\sum }_{i=1}^n{\varphi }_i(t+1)={\sum }_{i=1}^n{\varphi }_i\left(t\right)$ for all $t\ge 0$, and the consensus is reached at ${\mathit{\varphi }}_{\mathbf{0}}$ with ${\varphi }_{01}=\dots ={\varphi }_{0n}=c$, with c equalling the initial average ${\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{\varphi }_i\left(0\right)$. On the other hand, if W is only row-stochastic, then the limiting consensus state will be a weighted average determined by the left eigenvector of W corresponding to eigenvalue 1 [43]. The convergence of (10) is again related to the spectrum of L analogously to the continuous case.

4.2.3. Multi-Hop Consensus Dynamics

The previous models can be extended to incorporate long-range interactions using the k-path Laplacians $L_k$. To model such phenomena, we introduce additional coupling terms that account for neighbors at distances $2,3$, etc. Suppose we fix an integer $k_0\ge 1$ and assign a weight coefficient ${\alpha }_k\ge 0$ to interactions via k-hop neighbors (where $k=1,2,\cdots ,k_0$). It is natural to assume ${\alpha }_1\ge {\alpha }_2\ge \cdots \ge {\alpha }_{k_0}$, reflecting that closer neighbors exert a stronger influence than far neighbors. In some cases, the weights are considered as ${\alpha }_k=e^{-\beta k}$, with $\beta >0$, representing an exponential decay of the influence concerning distance. These transformations do not qualitatively transform the diffusion dynamics, i.e., for any parameter, the dynamics remain standard diffusion. However, the Mellin transformation (power law) transforms from normal diffusion to superdiffusion for certain values of the exponent [22,44]. In a continuous-time setting, generalized consensus dynamics can be written as

$${\displaystyle \frac{d\mathit{\varphi }}{dt}}\left(t\right)=-\sum _{k=1}^{k_0}{\alpha }_k{L}_k\mathit{\varphi }\left(t\right).$$

(11)

We recall that the standard nearest-neighbor model is obtained from (8) when $k_0=1$ and ${\alpha }_1=1$). The matrix $L_{\mathrm{tot}}:={\sum }_{k=1}^{k_0}{\alpha }_k{L}_k$ from (11) acts as an effective Laplacian matrix for the multi-hop coupling. Like each $L_k$, the combined matrix $L_{\mathrm{tot}}$ is symmetric, positive, and semi-definite, and $L_{\mathrm{tot}}\mathbf{1}=0$. Analogously, a multi-hop consensus update can be formulated for the discrete case shown in (10):

$$\mathit{\varphi }(t+1)=P_c\mathit{\varphi }\left(t\right),$$

(12)

where $P_c$ denotes the combined update matrix $I-\epsilon {\sum }_{k=1}^{k_0}{\alpha }_k{L}_k$, which generalizes the Perron matrix of the standard model. As before, for $P_c$ to define a proper consensus update, it should have non-negative entries, and each row should sum to 1. Sufficient conditions can be derived from the step-size $\epsilon$ and the network’s k-path degrees. For instance, if we take ${\alpha }_k=1$ for all k, a suitable choice of the step size would be

$$0<\epsilon <{\displaystyle \frac{1}{{max}_{i\in V}{\sum }_{k=1}^{k_0}{\delta }_k\left(i\right)}}.$$

(13)

4.3. Consensus Dynamics on Directed Networks

Suppose that $G=(V,E)$ is now a directed network with A as an adjacency matrix. For each node $v_i\in V$, the out-degree of $v_i$ is defined as $d_i^{\mathrm{out}}={\sum }_{j=1}^n{A}_{ij}$ and the in-degree as $d_i^{\mathrm{in}}={\sum }_j{A}_{ji}$. Then, the directed Laplacian is given by

$$L^{\mathrm{out}}=D^{\mathrm{out}}-A,$$

(14)

where $D^{\mathrm{out}}=diag(d_1^{\mathrm{out}},\cdots ,d_n^{\mathrm{out}})$. Note that $L^{\mathrm{out}}$ satisfies $L^{\mathrm{out}}\mathbf{1}=0$, but, in general, it is not symmetric, so its spectral properties differ from those of the undirected case [45]. We recall that a directed network is weight-balanced if $d_i^{\mathrm{in}}=d_i^{\mathrm{out}}$ for all $1\le i\le n$. In this case, we can show that $W^{-1}{L}^{\mathrm{out}}W$ is symmetric. Consequently, $L^{\mathrm{out}}$ has real eigenvalues and a simple zero eigenvalue with the eigenvector $\mathbf{1}$ (on both left and right) [46]. In the continuous-time case, this guarantees that the total sum of states remains invariant, so the final consensus state is exactly the arithmetic mean of the initial conditions. In contrast, if the network is not balanced, the consensus state will be a weighted average determined by the stationary distribution of the row-stochastic matrix $W=I-\epsilon L^{\mathrm{out}}$ [43]. The discrete-time case is similar to the continuous one. The case of multi-hop interactions is expressed through the k-hop Laplacian defined as

$$L_k^{\mathrm{out}}=D_k^{\mathrm{out}}-A^{\left(k\right)},$$

(15)

with $D_k^{\mathrm{out}}$ as the diagonal matrix of k-hop out-degrees and $A^{\left(k\right)}$.

5. Synthetic Data Generation for Network-Based Analyses

Synthetic data generation is essential for developing and testing network algorithms under controlled conditions. Traditional random graph models have long generated synthetic networks with controlled properties. The Erdős–Rényi model produces random graphs by independently connecting nodes with a fixed probability, yielding networks that serve as null models for graph algorithms [47]. Watts and Strogatz’s small-world networks interpolate between regular lattices and random graphs, capturing the high clustering and short path lengths already observed in real-life examples [15]. Barabási and Albert’s scale-free model introduced preferential attachment, producing graphs with a power-law degree distribution capturing nature’s essence in forming networks [16]. These foundational models allow researchers to synthetically create social or transportation network topologies with desired global properties (randomness, small-world characteristics, heavy-tailed connectivity), which are helpful in testing network behavior theories and algorithms’ robustness (e.g., community detection or consensus). It is worth mentioning that other techniques, such as Stochastic Block Models (SBMs) [48] and the Lancichinetti–Fortunato–Radicchi (LFR) [49], can capture community structure and realistic degree heterogeneity. In addition, deep generative models enable us to improve synthetic graph generation by incorporating structural patterns from a set of training graphs (e.g., real social networks or road networks) as GraphRNN [50], NetGAN [51], or GraphVAE [52] do. However, we will restrict ourselves to the aforementioned three seminal network models in this work.

Below, we present two distinct procedures for synthetic data generation based on the consensus dynamics outlined previously. The first scenario considers a local (one-hop) interaction model, while the second scenario extends interactions through an exponential (multi-hop) model. Both approaches begin to generate a network and are subsequently converted into directed networks by assigning orientations to edges, allowing bidirectional connections with a given probability.

5.1. Base Case: Local (One-Hop) Interactions

In the local interaction scenario, the synthetic network generation process begins by sampling an undirected random graph $\tilde{G}$ with n nodes according to the chosen random model (Erdös–Renyi, Watts–Strogatz, or Barabàsi–Albert). Each undirected network acts as the structural backbone for the subsequent directed network construction.

Let $G=(V,E)$ be an undirected network and $G^{\prime }=(V^{\prime },E^{\prime })$ be the new directed network. Here, $V=V^{\prime }$. We assign a direction to each edge $(u,v)\in E$, creating an initial directed edge $(u,v)\in E^{\prime }$. Additionally, the reverse edge ($(v,u)$) is introduced probabilistically (e.g., with probability 0.3), thus resulting in a directed network $G^{\prime }$. The rest of the calculations are performed over $G^{\prime }$. We compute its adjacency matrix A and the diagonal out-degree matrix $D^{\mathrm{out}}$, from which the directed Laplacian matrix is derived as

$$L=D^{\mathrm{out}}-A.$$

(16)

Consistent with the previously described consensus dynamics, we introduce the auxiliary matrix $K=L+A$ and determine the iterative update matrix P as:

$$P=I-ϵL,\mathrm{where}ϵ={\displaystyle \frac{1}{cmax\left(K\right)}},$$

(17)

with c typically chosen around 100 to ensure numerical stability. The initial state vector $\varphi \left(0\right)$ is constructed from the diagonal elements of K (i.e., ${\varphi }_i\left(0\right)=K_{ii}$). The consensus evolution then iterates according to:

$$\varphi (t+1)=P\varphi \left(t\right),$$

(18)

until the state convergence reaches a predefined tolerance $\tau$ or until the maximum allowed iterations are performed. Only the first ten state vectors and the final convergent mean state are retained to ensure computational efficiency and manageable data volume.

5.2. Exponential Case: Extended (Multi-Hop) Interactions

The exponential (multi-hop) interaction scenario generalizes the one-hop procedure by explicitly considering higher-order node interactions. The network diameter $d_{max}$ of $G^{\prime }$ (defined on the underlying undirected topology) is calculated, determining the maximum interaction range. For each interaction distance k from 1 to $d_{max}$, the corresponding k-hop adjacency matrix $A^{\left(k\right)}$ is computed through repeated matrix multiplication. Each $A^{\left(k\right)}$ leads to the associated k-path Laplacian matrix:

$$L_k=D_k-A^{\left(k\right)},$$

(19)

where $D_k$ is a diagonal matrix containing the out-degree sums of $A^{\left(k\right)}$. The interactions across longer paths are weighted by an exponentially decaying factor $e^{-\alpha k}$, where $\alpha >0$, which controls the influence decay rate. Thus, the multi-hop consensus iterative update matrix $P_{\mathrm{multi}}$ is defined as:

$$P_{\mathrm{multi}}=I-ϵ\sum _{k=1}^{d_{max}}{e}^{-\alpha k}{L}_k,$$

(20)

where the parameter $ϵ$ is carefully chosen to maintain non-negative matrix entries and row normalization, ensuring stochastic-like behavior.

The iterative consensus evolution begins from an initial state $\varphi \left(0\right)$ constructed from the diagonal entries of the one-hop matrix $K=L+A$. The state vector is then updated iteratively according to:

$$\varphi (t+1)=P_{\mathrm{multi}}\varphi \left(t\right),$$

(21)

and the procedure continues until convergence (within tolerance $\tau$) or the maximum iteration limit is reached. As before, only the initial ten states and the final mean consensus state are recorded, ensuring computational tractability while capturing essential dynamic features.

5.3. Pseudocode of the Data Generation

Algorithm 1 summarizes the synthetic data generation steps for local and exponential consensus cases, explicitly detailing graph construction, edge orientation, Laplacian calculations, iterative consensus procedures, and data storage considerations.

Algorithm 1 Synthetic data generation with base (one-hop) and exponential (multi-hop) consensus

Require: Number of nodes n; random graph model $\mathcal{M}\in \{\mathrm{BA},\mathrm{WS},\mathrm{ER}\}$; case type $\in \{\mathtt{base},\mathtt{exponential}\}$; bidirectional probability $p_b$; tolerance $\tau$; maximum iterations ${\mathit{iter}}_{max}$; decay parameter $\alpha$ (for multi-hop); record length $T=10$ Ensure: Time-series states $\left\{\varphi \right(0),\cdots ,\varphi (T-1\left)\right\}$; final consensus value ${\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{\varphi }_i(\infty )$        1: Generate random undirected graph $\tilde{G}$ (BA, WS, or ER).        2: Create directed graph G from $\tilde{G}$, orienting edges and adding reverse edges with probability $p_b$.        3: Compute adjacency matrix A and out-degree diagonal matrix $D^{\mathrm{out}}$ from G.        4: Form directed Laplacian $L=D^{\mathrm{out}}-A$; set $K=L+A$, $ϵ=1/(100·max(K\left)\right)$.        5: if case type = base then        6:     Set $P=I-ϵL$.        7: else        8:     Compute $d_{max}=\mathrm{diameter}(\tilde{G})$.        9:     Calculate matrices $A^k$ and Laplacians $L_k$ for $1\le k\le d_{max}$.      10:     Set $P=I-ϵ{\sum }_{k=1}^{d_{max}}{e}^{-\alpha k}{L}_k$.      11: end if      12: Initialize $\varphi \left(0\right)=\mathrm{diag}\left(K\right)$.      13: for $t=0$ to ${\mathit{iter}}_{max}-1$ do      14:     $\varphi (t+1)=P\varphi \left(t\right)$      15:     if ${\parallel \varphi (t+1)-\varphi \left(t\right)\parallel }_{\infty }\le \tau$ then      16:         break      17:     end if      18: end for      19: Record $\left\{\varphi \right(0),\cdots ,\varphi (min(T-1,t)\left)\right\}$ and ${\displaystyle \frac{1}{n}}{\sum }_i{\varphi }_i\left(t\right)$.

6. Results

In this section, we present the performance assessment of our proposed models under different directed graph configurations, focusing on accurately predicting the final consensus value while accommodating the extended connectivity provided by k-path Laplacian matrices. We generated, for every graph type (Erdős–Rényi, Watts–Strogatz, and Barabási–Albert), node size (25, 50, 100, 200, and 300), and case (base and exponential), a fixed-size training set of 2400 samples and a test set of 600 samples. Each model—XGBoost, LSTM, Extended LSTM, Transformer, and ConvLSTM—was exposed to identical training and test data to allow fair comparisons. Unless otherwise stated, we reserve 20% of the training data for validation, leading to an 80/20 train–validation split. Al the models and training data are accessible at https://github.com/yusef320/AI-Driven-Consensus (accessed on 30 March 2025).

We systematically tuned hyperparameters through predefined search grids tailored to each approach. We also performed some preliminary experiments to fine-tune these ranges and ensure robust performance across different model types. For instance, maximum tree depth, number of estimators, and learning rate were explored for XGBoost; hidden layer sizes, projection sizes, and multiple LSTM layers were considered for the recurrent-based architectures; and varying numbers of heads, hidden dimensions, and layers were tested for the Transformer models. A mean-squared-error objective-guided training, combined with early stopping triggered after 30 consecutive epochs (or boosting rounds for XGBoost) without validation loss improvement. The 30-epoch threshold was specifically selected based on our observation that during the first epoch, with our chosen hyperparameters, the model did not demonstrate a stable learning trend.

Final evaluations were performed on the 600-sample holdout test set. The main performance metrics include Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE).

Post training, predictive performance was evaluated primarily using the RMSE and the MAPE. RMSE provides a clear representation of the typical magnitude of prediction errors in the original units of the target variable. At the same time, MAPE offers a normalized error measure by expressing the average absolute error as a percentage of the true values.

In the following subsections, we compare predictive accuracy and computational overhead for each candidate model, emphasizing how long-range interactions (via higher-order path-Laplacian matrices) influence the quality of the consensus estimation. As discussed previously, larger values of k allow greater leverage of indirect connectivity, potentially strengthening information flow in highly sparse or disconnected graphs. The empirical findings highlight the importance of carefully chosen hyperparameters and how each architecture scales in the presence of multi-hop influences.

6.1. Validation

Figure 1 presents the RMSE achieved by five predictive models—ConvLSTM, LSTM, Extended LSTM, Transformer, and XGBoost—for both the base (left column) and exponential (right column) scenarios in three network types: Barabási–Albert (top row), Erdős–Rényi (middle row), and Watts–Strogatz (bottom row). Each chart compares the models’ RMSE for increasing node sizes (25, 50, 100, 200, 300). Overall, the exponential scenario reduces the RMSE relative to the base scenario, suggesting that multi-hop interactions improve predictive accuracy. These patterns highlight how network topology, including long-range interactions and model architecture, each influence consensus prediction accuracy.

In Figure 2, we show the Mean Absolute Percentage Error (MAPE) across different network sizes (25, 50, 100, 200, 300 nodes) for three topologies: Barabási–Albert, Erdős–Rényi, and Watts–Strogatz. The bars indicate that at smaller scales (25 and 50 nodes), Erdős–Rényi networks exhibit substantially higher MAPE values, suggesting more variability or unpredictability in their consensus dynamics. Watts–Strogatz networks present intermediate error levels, while Barabási–Albert remains comparatively lower even for smaller node counts. As the number of nodes increases, the MAPE for all three topologies converges to very low values, implying that larger networks become easier to predict regardless of their underlying structure. This pattern highlights the strong influence of network topology—especially the randomness in Erdős–Rényi—on predictability at small scales, whereas scale-free (Barabási–Albert) and small-world (Watts–Strogatz) networks remain more stable in smaller configurations. In addition, the higher MAPE variability at smaller scales, particularly for models like Extended LSTM, Transformer, and XGBoost, is likely due to the models’ sensitivity to network sparsity at smaller node sizes. These models have a greater capacity to capture complex relationships, but at smaller scales, they may overfit to the specific realizations of the network, leading to higher variability in predictions.

Figure 3 displays box-and-whisker plots of the Mean Absolute Percentage Error (MAPE) for five predictive models—ConvLSTM, LSTM, Extended LSTM, Transformer, and XGBoost—across varying network sizes (25, 50, 100, 200, 300). Each box illustrates the distribution of prediction errors, with taller boxes and whiskers indicating more considerable variability. At small network sizes (25–50 nodes), specific methods (e.g., Extended LSTM, Transformer, and XGBoost) seem to produce slightly higher median MAPEs than ConvLSTM and LSTM. When you get to 200–300 nodes, the boxes for most methods become narrower and cluster closer together. In other words, as the network size increases, all models tend to converge toward lower MAPE values, reflecting improved stability and predictability. It is worth mentioning that the lack of hubs and strong clustering in Erdös–Renyi networks results in more uniform and homogeneous structures, where most nodes have similar degrees. This reduces the presence of highly connected nodes or local communities that can accelerate or concentrate the spreading in the network, resulting in a more variable and unpredictable diffusion.

Overall, the results emphasize that both model architecture and mostly network size significantly affect prediction accuracy.

6.2. Computational Efficiency

Figure 4 illustrates the average prediction time (in milliseconds) for five models—ConvLSTM, LSTM, Extended LSTM, Transformer, and XGBoost—across networks of increasing size (25, 50, 100, 200, 300 nodes). The graph shows that ConvLSTM’s prediction time scales significantly as the number of nodes grows, surpassing 0.25 ms at 300 nodes. In contrast, LSTM, Extended LSTM, Transformer, and XGBoost remain relatively stable and low in prediction time, indicating they handle larger networks more efficiently. This highlights the importance of computational overhead when selecting a model for real-time or large-scale scenarios.

7. Conclusions

Our study introduces a novel framework that significantly advances the modeling of consensus dynamics in multi-agent networks by incorporating machine learning methods to include the effect of long-range interactions through path-Laplacian matrices. By extending the classical one-hop Laplacian formulation to account for k-hop connections, we have demonstrated that capturing extended connectivity not only estimates the final consensus state, but also enhances the robustness of consensus protocols. Such an approach results in particular interest in directed networks, when the consensus is not just the average of components and depends on the inner structure of the network.

The experimental analysis—conducted on synthetic networks generated using Erdös–Rényi, Watts–Strogatz, and Barabási–Albert models—underscores the pivotal role of multi-hop interactions. In the base case of local interactions, traditional consensus mechanisms perform adequately; however, the exponential (multi-hop) approach reveals a marked improvement in consensus speed and accuracy. This suggests that considering indirect, long-range influences provides additional communication “shortcuts”, thereby effectively reducing the distance between nodes and promoting faster agreement.

Moreover, our comparative study of various machine learning techniques—including LSTM, xLSTM, Transformer, XGBoost, and ConvLSTM—highlights the considerable potential of data-driven methods in predicting the final consensus state. Notably, these advanced architectures excel in capturing both local and global temporal dynamics inherent in complex networks. Integrating deep learning with advanced graph-theoretic constructs paves the way for more efficient and adaptive consensus algorithms, particularly relevant in applications such as autonomous systems and distributed sensor networks. We also point out that the extension of this approach to larger scales of nodes have to be carefully addressed due to (linear) complexity growth, the vanishing gradient problem that makes the training unstable, and the batch-size reduction that makes the training slower.

In summary, the most compelling outcomes of our work are the successful extension of the Laplacian framework to encapsulate long-range interactions and the demonstrated efficacy of state-of-the-art machine learning models in predicting consensus dynamics. These insights enrich our theoretical understanding and offer practical avenues for enhancing consensus mechanisms in real-world multi-agent systems.