Synchronization Dynamics in Non-Normal Networks: The Trade-Off for Optimality

Abstract

Synchronization is an important behavior that characterizes many natural and human made systems that are composed by several interacting units. It can be found in a broad spectrum of applications, ranging from neuroscience to power-grids, to mention a few. Such systems synchronize because of the complex set of coupling they exhibit, with the latter being modeled by complex networks. The dynamical behavior of the system and the topology of the underlying network are strongly intertwined, raising the question of the optimal architecture that makes synchronization robust. The Master Stability Function (MSF) has been proposed and extensively studied as a generic framework for tackling synchronization problems. Using this method, it has been shown that, for a class of models, synchronization in strongly directed networks is robust to external perturbations. Recent findings indicate that many real-world networks are strongly directed, being potential candidates for optimal synchronization. Moreover, many empirical networks are also strongly non-normal. Inspired by this latter fact in this work, we address the role of the non-normality in the synchronization dynamics by pointing out that standard techniques, such as the MSF, may fail to predict the stability of synchronized states. We demonstrate that, due to a transient growth that is induced by the structure’s non-normality, the system might lose synchronization, contrary to the spectral prediction. These results lead to a trade-off between non-normality and directedness that should be properly considered when designing an optimal network, enhancing the robustness of synchronization.

1. Introduction

Systems in nature are often constituted by a large number of small parts that continuously interact with each other [1,2]. Although it might be possible to accurately know the dynamics that characterize each of the individual constituents, it is, in general, nontrivial to figure out the collective behavior of the systems as a whole that results from the individual/local interactions. A relevant example is provided by a system that is composed by an ensemble of coupled non-linear oscillators, which behave in unison, being driven by the non-local interaction; the system is said to be synchronized [2,3]. Synchronization has been extensively studied in snetworkcience as a paradigm of dynamical processes on a complex network, mainly due to the essential role of the coupling topology in the collective dynamics [1]. Its generic formulation allowed for researchers to use it to model several applications, ranging from biology, e.g., neurons firing in synchrony, to engineering, e.g., power grids [4]. The ubiquity of synchronization in many natural or artificial systems has naturally raised questions regarding the stability and robustness of synchronized states [5,6,7,8]. In their seminal work, Pecora & Caroll [9] introduced a method, known as Master Stability Function (MSF), in order to help understand the role that the topology of interactions has on system stability. Assuming a diffusive-like coupling among the oscillators, the MSF relates the stability of the synchronous state to the nontrivial spectrum of the (network) Laplace matrix; in particular, it has been proven that the latter should lie in the region where the Lyapunov exponent that characterizes the MSF takes negative values [2,10]. For a family of models (e.g., Rössler, Lorenz, etc.), whose stable part of the MSF has a continuous interval where the (real part of the) Laplacian eigenvalues can lie, it has been proven that they maximize their stability once the coupling network satisfies particular structural properties. Such optimal networks should be directed, spanning trees and without loops [5,6]. These networks have the peculiarity of possessing a degenerate spectrum of the Laplacian matrix and laying in the stability domain that is provided by the MSF. The Laplacian degeneracy is also often associated with a real spectrum or with considerably low imaginary parts when compared to the real ones [11,12].

The vast interest in complex networks in recent years has also provided an abundance of data on empirical networked systems that initiated a large study of their structural properties [1]. From this perspective, it has been recently shown that many real networks are strongly directed, namely they possess a highly asymmetric adjacency matrix [13]. Most of these networks present an extremely hierarchical, almost-DAG (Directed Acyclic Graph), structure. This property potentially makes the real networks suitable candidates for optimally synchronized dynamical systems that are defined on top of them. Another aspect that is unavoidably associated with the high asymmetry of real networks is their non-normality [13], namely their adjacency matrix $\mathbf{A}$ satisfies the condition $\mathbf{A}{\mathbf{A}}^T\ne {\mathbf{A}}^T\mathbf{A}$ [12]. The non-normality can be critical for the dynamics of networked systems [13,14,15,16,17,18]. In fact, in the non-normal dynamics regime, a finite perturbation regarding a stable state can undergo a transient instability [12], which, because of the nonlinearities, could never be reabsorbed [13,14]. The effect of non-normality in dynamical systems has been studied in several contexts, such as hydrodynamics [19], ecosystems stability [20], pattern formation [21], chemical reactions [22], etc. However, it is only recently that the ubiquity of non-normal networks and the related dynamics have been put to the fore [13,14,15,16,17,18]. In this paper, we will elaborate on these lines showing the impact of non-normality on the stability of a synchronous state. We first show that a strongly non-normal network has, in general, a spectrum that is very close to a real one, and that this, in principle, should imply a larger domain of parameters for which stability occurs, for systems with a generic shaped MSF. For illustration purposes, we will consider the Brusselator model [23,24], a two-species system with a discontinuous interval of stability in the MSF representation. We will also examine the limiting cases of our analysis to two simple network models [25], namely a (normal) bidirected circulant network and a (non-normal) chain, both with tunable edge weights in such a way to allow for a continuous adjustment, respectively, of the directedness and non-normality. In the Appendix A, we will extend such results to a family of non-normal random networks thus showing the generality of such behavior.

The MSF relies on the computation of the (real part of the maximum) Lyapunov exponent and, thus, in the case of time-dependent systems, it does not possess the full predictability power that it has in the autonomous case (fixed point in/stability). For this reason, we will use a homogenization method, whose validity is limited to a specific region of the model parameters, allowing for us to transform the linearized periodic case problem into a time-independent one [26]. This way, we remap our problem to an identical one that was studied in the context of pattern formation in directed networks where spectral techniques provide significant insight [25,27]. Such an approach allows for us on one side to assess the quantitative evaluation of the role of the imaginary part of the Laplacian spectrum in the stability problem. On the other, it permits the use of numerical methods, such as the pseudo-spectrum [12] in the study of the non-normal dynamics. To the best of our knowledge, this is the first attempt to use such techniques in the framework of time-varying systems, with the theory of non-normal dynamical systems being limited so far to autonomous systems [12]. As expected, the non-normality plays against the stability of the synchronized ensemble of oscillators. Furthermore, a high non-normality translates to a high spectral degeneracy, which brings a large pseudo-spectrum, indicating a high sensibility towards the instability.

Clearly, the directionality and non-normality stand on two parallel tracks regarding the stability of synchronized states and their robustness. The results that we present here show that the previously optimal networks are not practically as good as was thought, since the synchronization dynamics are frail to small external perturbations. In fact, their highly directed structure amplifies such perturbations and eventually causes the underlying system to desynchronize. Furthermore, such an outcome bypasses the picture that is described by the classical spectral analysis. As a conclusion of our work, we show that the most optimal design should be looked at as a trade-off between a high and low directionality/non-normality. Such a choice should depend either on the magnitude of perturbation or the ratio directed vs. non-normal of the network structure.

2. Optimal Synchronization: Directed vs. Non-Normal Networks

We consider a network that is constituted of N nodes (e.g., the idealized representation of a cell), and we assume a metapopulation framework, where the species dynamics inside each node is described by the Brusselator model, a portmanteau term for Brussels and oscillator. It has been initially introduced by Prigogine & Nicolis in order to capture the autocatalytic oscillation [23] phenomenon, which results from a Hopf bifurcation curve in the parameter plane. This will be the framework that we will consider in the following, thus neglecting the fixed point regime. Species can migrate across nodes with a diffusion-like mechanism. In formulae, this model translates to a reaction–diffusion set of equations:

$$\left\{\begin{array}{c}{\displaystyle {\displaystyle \frac{d{\phi }_i}{dt}}=1-(b+1){\phi }_i+c{\phi }_i^2{\psi }_i+D_{\phi }\sum _{j=1}^N{\mathcal{L}}_{ij}{\phi }_j}\hfill \\ {\displaystyle {\displaystyle \frac{d{\psi }_i}{dt}}=b{\phi }_i-c{\phi }_i^2{\psi }_i+D_{\psi }\sum _{j=1}^N{\mathcal{L}}_{ij}{\psi }_j,\forall i=1,\cdots ,N,}\hfill \end{array}\right.$$

(1)

where ${\phi }_i$ and ${\psi }_i$ indicate the concentration of the two species per node, $D_{\phi }$, $D_{\psi }$ are their corresponding diffusion coefficients, and b, c are the model parameters. The coupling is represented by the matrix $𝓦$, whose non-negative entries ${\mathcal{W}}_{ij}$ represent the strength of the edge pointing form node j to node i. The entries of the Laplacian matrix $𝓛$ are given by ${\mathcal{L}}_{ij}={\mathcal{W}}_{ij}-k_i^{in}{\delta }_{ij}$, where $k_i^{in}={\sum }_j{\mathcal{W}}_{ij}$ stands for the incoming degree of node i, i.e., the number of all the entering edges into node i. Here, we want to emphasize that many other coupling operators are also possible; nevertheless, most of them will reduce at the linear level to a Laplacian involving the differences of the observable among coupled nodes [2], i.e., ${\sum }_{j=1}^N{\mathcal{L}}_{ij}{x}_j={\sum }_{j=1}^N{\mathcal{W}}_{ij}(x_j-x_i)$. This form ensures that the coupling is only in action when the observables assume different values in two coupled nodes.

The reason for choosing such a model, as mentioned earlier, is mainly due to the discontinuous interval of the stability domain that is provided by the MSF of the problem. In order to proceed with the stability analysis, we first need to identify the homogeneous periodic solution, ${\mathbf{\phi }}^{*}\left(t\right)$ and ${\mathbf{\psi }}^{*}\left(t\right)$, hereby called the synchronized manifold and then to linearize the system around this. Let us introduce the perturbations for the i–th node by $\delta {\phi }_i$ and $\delta {\psi }_i$, and then the linearized equations describing their evolution are given by:

$$\begin{array}{ccc}\hfill {\displaystyle \frac{d\left(\delta {\phi }_i\right)}{dt}}& =& \left[f_{{\phi }_i}{\delta }_{ij}+D_{\phi }\sum _{j=1}^N{\mathcal{L}}_{ij}\right]\delta {\phi }_j+f_{{\psi }_i}\delta {\psi }_i\hfill \\ \hfill {\displaystyle \frac{d\left(\delta {\psi }_i\right)}{dt}}& =& {\mathcal{g}}_{{\phi }_i}\delta {\phi }_i+\left[{\mathcal{g}}_{{\psi }_i}{\delta }_{ij}+D_{\psi }\sum _{j=1}^N{\mathcal{L}}_{ij}\right]\delta {\psi }_j\hfill \\ \hfill \forall i=& 1& ,\cdots ,N,\hfill \end{array}$$

(2)

where the partial derivatives are given by $f_{{\phi }_i}=-(b+1)+2c{\phi }^{*}\left(t\right){\psi }^{*}\left(t\right)$, $f_{{\psi }_i}=c{\phi }^{*}{\left(t\right)}^2$, ${\mathcal{g}}_{{\phi }_i}=b-2c{\phi }^{*}\left(t\right){\psi }^{*}\left(t\right)$, and ${\mathcal{g}}_{{\psi }_i}=-c{\phi }^{*}{\left(t\right)}^2$. Notice that the partial derivatives of the reaction part are evaluated on the synchronized manifold. This translates into a time-dependent Jacobian matrix, due to the periodicity of the solutions and, thus, to a non-autonomous linear system. In order to make a step forward, let us introduce the following compact notation; let $\mathbf{x}={(\delta {\phi }_1,\dots ,\delta {\phi }_N,\delta {\psi }_1,\dots ,\delta {\psi }_N)}^{\mathrm{T}}$ be the $2N$-dimensional perturbations vector, $𝓓$ the diagonal diffusion coefficients matrix, and $𝓙\left(t\right)$ the time-dependent Jacobian matrix; hence, Equation (2) can be rewritten as

$$\dot{\mathbf{x}}=\left(𝓙\left(t\right)+𝓓\odot 𝓛\right)\mathbf{x},$$

(3)

where ⊙ is the coordinatewise multiplication operator. Subsequently, we proceed by diagonalizing the linearized system while using the basis of eigenvectors of the network Laplace operator $𝓛$. Notice that this is not always possible, because the Laplacian matrix of directed networks might not have linearly independent eigenvectors. We will assume such a basis to exist for the time being, and we will consider such an issue again when discussing the non-normal case. Denoting, by $\mathsf{\xi }$, the transformed perturbations vector, Equation (3) becomes

$$\dot{\mathsf{\xi }}=\left(𝓙\left(t\right)+𝓓\odot \mathsf{\Lambda }\right)\xi ,$$

(4)

where $\mathsf{\Lambda }$ denotes the diagonal matrix of the Laplacian eigenvalues. The (real part of the) largest Lyapunov exponent of Equation (4), which is known in the literature as the Master Stability Function [1,2,9,10], is thus a function of the eigenvalues $\mathsf{\Lambda }$. Let us stress that the study of the stability of a general non-autonomous system is normally not possible through the classical spectral analysis, and one, therefore, has to resort to the MSF.

Before proceeding in the quest for the optimal network topological features that minimize the MSF, we will introduce two simple network models, as shown in Figure 1, for which we can tune the directionality and the non-normality acting on a single parameter. In the first case, Figure 1a, we consider a bidirectional circulant network, i.e., a network whose adjacency matrix is circulant [28], made by two types of links, one of weight 1 forming a clockwise ring and the other winding a counterclockwise ring of tunable weights $ϵ$. The latter can vary in the interval $\epsilon \in [0,1]$, exploring, in this way, the possible topologies from a fully symmetric case when $\epsilon =1$ to a totally mono-directed network when $\epsilon =0$. Because such a network is circulant, the adjacency matrix will be normal, a property that is inherited by the Laplace operator. On the contrary, if we remove two reciprocal links, respectively, of weights 1 and $\epsilon$, we, instead, obtain a non-normal network, as depicted in Figure 1b. In this case, the adjacency matrix is non-normal [12], a feature that is also reflected on the Laplacian matrix. Even in this case, we can tune the non-normality by varying the $\epsilon$ parameter in the unitary interval, as for the previous case, this can be appreciated from the results that are shown in Figure 1c, where we report the normalized Henrici index, a well-known proxy of non-normality, as a function of $\epsilon$. The main advantage of using the above network models is the existence of a basis of eigenvectors for the Laplacian matrix. In the first network model, this is due to the normality of the graph Laplacian, while, in the second one, it is because of the tridiagonal form of the coupling operator [29]. This property is essential for the applicability of the MSF analysis, which is impossible otherwise.

3. The Case of Normal Directed Networks

We start by considering the bidirected circular network and studying the linear stability of the synchronized state while using the MSF analysis. The results that are shown in Figure 2a indicate that the network topology increasingly contrasts the stability of the synchronous manifold when the directionality increases. In fact, when the MSF computed for the directed network is compared to the symmetric case used as reference line, which lies in the magenta curve (constructed for a real continuous interval of non-positive values of ${\mathsf{\Lambda }}_{\alpha }$), we can always observe larger values, which, moreover, increase as $\epsilon$ decreases (for the same fixed Laplacian eigenvalue). Because of the circulant property of the Laplace matrix, its spectrum can be explicitly computed [25] ${\mathsf{\Lambda }}_{\alpha }=1+\epsilon +(1+\epsilon )cos(2\alpha \pi /N)+i(1-\epsilon )sin(2\alpha \pi /N)$. One can easily notice that, for $\epsilon =0$, the spectrum distributes uniformly onto the unitary circle centered at $(1,0)$, as also shown in Figure 2b in blue stars, where the instability region in the plane $[-$Re$\left({\mathsf{\Lambda }}_{\alpha }\right)$, ± Im$\left({\mathsf{\Lambda }}_{\alpha }\right)]$, where the Lyapunov exponent that takes positive values is shown in magenta. On the other side, when $\epsilon =1$, the network turns symmetric, which makes the spectrum real.

The MSF formalism ultimately relies on the maximum Lyapunov exponent, which, despite having proved its validity in ruling out the chaotic behavior of dynamical system [3], remains grounded on numerical methods. In order to improve our analytical understanding of the problem, we proceed by transforming Equation (4) into an autonomous one, allowing, in this way, to deploy the spectral analysis tools. This method is part of the broader set of homogenization methods that aim at averaging a time-dependent system in order to obtain a time-independent one [26]. Such methods have been found to also be useful for the stability analysis of synchronized states [30,31]. The resulting autonomous version of the MSF is sometimes referred to as the dispersion relation [21]. The mathematical validity of the proposed approximation is grounded on the Magnus series expansion truncated at the first order [31]; hence, the set of model parameters for which we expect a good agreement with the original model corresponds to the case when higher-order terms are negligible. For more details, the interested reader should consult [31]. In formula, it translates to

$$𝓙\left(t\right)⟶{\langle 𝓙\rangle }_T=\frac{1}{T}{\int }_0^T𝓙\left(\tau \right)d\tau .$$

(5)

Remarkably, as shown in Figure 3, this approximation yields qualitative results that are in excellent agreement with the original model for a specific range of parameters. An alternative to this approach is to apply a perturbative expansion near the bifurcation point, obtaining, in this way, the time-independent Ginzburg–Landau normal form [32]. However, the effectiveness of the latter method is exclusively limited to parameter values that are very close to the stability threshold. In this sense, our approach is more general, both from allowing a larger set of parameters where the method remains valid, and at the same time, it is independent of the choice of the model compared to previous works [33]. The passage to an autonomous system is also essential in explaining the effect of the imaginary part of the Laplacian eigenvalues in the newly obtained stability function, the dispersion relation. It has been rigorously shown in [25,27] that the dispersion relation increases proportionally to the magnitude of the imaginary part of the spectrum. We already observed similar results for the case of the MSF that is presented in Figure 2. We can, in this way, conclude that the averaging method sheds light on the role of the directed topology in the destabilization of a synchronized regime.

The Case of Non-Normal Directed Networks

The analysis that was performed in the previous section is based on the study of the linearized system; in some cases, however, such an analysis is not sufficient for understanding the outcome of the nonlinear system. In Figure 4, we again consider the MSF that is computed for the directed chain previously introduced (panel b) of Figure 1. From Figure 4b, one might naively conclude that the system will synchronize, since the MSF is non-positive for all values of −Re $\left({\mathsf{\Lambda }}_{\alpha }\right)$. Moreover, the spectrum is completely real (see panel b) and, thus, there cannot be any contribution from the imaginary part of the spectrum. However, a direct inspection of the orbit behavior (panel c) that was obtained by numerically integrating the network of coupled Brusselator oscillators clearly shows that the system does not synchronize. This diversity of behavior is related to the non-normal property of the considered network; indeed, it has been recently proved that such a structural property can strongly alter the asymptotic behavior of networked systems [34]. In a linear regime, a finite perturbation regarding a stable equilibrium goes through a transient amplification (blue curve in Figure 4d) that is proportional to the level of non-normality before it is eventually reabsorbed in the steady state [12], while, in the full non-linear system, the finite perturbation could persist indefinitely (red curve in Figure 4d). Up to now, this analysis has been limited to the case of autonomous systems; in this paper, for the first time we extend it to the periodic time-dependent case, making use of the homogenization process. This explains the permanent instability, as shown in Figure 4, causing the loss of stability for the synchronized state.

The non-normal dynamics study cannot be straightforwardly tackled with the analytical methods of the local stability, mostly because the instability occurs in a highly nonlinear regime. Such condition require a global analysis that can be obtained while using the numerical technique that is based on a spectral perturbation concept that is known as the pseudo-spectrum. For a given matrix $\mathbf{A}$, the latter is defined as $\sigma \left({\mathbf{A}}_{\delta }\right)=\sigma \left(\mathbf{A}+\Delta \right),$ for all $\left|\right|\Delta \left|\right|\le \delta$, for where $\sigma (·)$ represents the spectrum and $\left|\right|·\left|\right|$ a given norm. The package EigTool [35] allows for us to compute and draw in the complex plane the level curves of the pseudo-spectrum for a given value of $\delta$. Although the pseudo-spectrum is not sufficient to fully explain the system behavior, it is certainly of great utility in estimating the role of non-normality in the dynamics outcomes. In particular, in panel (a) of Figure 5, we report level curves of the pseudo-spectrum for three different values of the parameter $\epsilon$ representing the reciprocal links of the directed chain. Notice that, by increasing the non-normality of the toy network, the pseudo-spectrum will also increase the chances of intersection with the instability region. In panel (b) of Figure 5, we have shown a comparison between a proxy of the presence of a synchronized state, i.e., the standard deviation S [36] of the asymptotic orbit behavior and the MSF demonstrating a clear different behavior. For all of the considered values of $\epsilon$, the MSF is always negative, which suggests a stable synchronized state; on the other hand, S becomes positive and large for small enough $\epsilon$, testifying a loss of synchronization. The dependence on the different values of the initial conditions is further shown in panels ($c_1$) and ($c_2$). As expected, the instability is more probable for both larger values of non-normality and magnitude of the initial conditions. In particular, it can be observed that the synchronization basin of attraction is strongly reduced for the non-normal network as compared to the normal one and, moreover, its width varies along the limit cycle, which implies that desynchronization will also depend on the point at which the perturbation starts.

4. Conclusions

In this paper, we have studied the quest for the optimal conditions ensuring the stability of synchronization dynamics in directed networks. Such conditions determine the design of a networked system that makes the synchronization regime as robust as possible. Previous results have proven that a strictly directed topology is necessary for the synchronized state’s robustness. Based on the well-known Master Stability Function, it has been shown that directed tree-like networks are optimal for models with a discontinuous interval of the Laplacian spectrum in the stability range of MSF. Here, we have extended such results, proving that they are generally independent of the dynamic model. While using an averaging procedure, we transformed the problem from a time-dependent (non-autonomous) to a time-invariant (autonomous) one. This method allows for proving that networks whose Laplacian matrix exhibits a spectrum that lacks an imaginary part are the most optimal. In general, the loss of synchronization increases with the magnitude of the imaginary part of the spectrum. However, our primary focus has been on the effect that the non-normality, a ubiquous feature of many real-world networks, as recent findings show [13], has on the collective dynamics of synchronization. This latter feature can play a very important role in the linear dynamics influencing the local stability of the synchronized state through a strong transient amplification of the perturbations. We have extended the idea of non-normal dynamics to the case of non-autonomous synchronization dynamics, revealing how network non-normality can drive the system to instability, thus increasing the understanding of synchronization in complex networks. We have also numerically quantified the effect of non-normality in driving the instability through the pseudo-spectrum technique. Although, throughout this paper, for purely pedagogical reasons, we have illustrated our results for a toy network model with tuneable non-normality, we have also shown (see the Appendix A) that such results extend to a general case of non-normal random networks. In conclusion, we have analytically and numerically demonstrated that there is no compelling recipe for optimal network architecture in order to conserve the synchronized state, but rather a trade-off between the network directedness and its non-normality. Our results clearly show that networks previously thought to be optimal regarding synchronization are not such, but, on the contrary, the stability of the associated synchronous solution is quite fragile to small perturbations, which makes their role in the synchronization dynamics apparently different from what was previously intuited in the literature [5,6]. Additionally, the non-normality makes standard techniques, such as the Master Stability Function, fail by a large amount. We are aware that the interesting outcomes of the interaction of structural non-normality networks with the fascinating synchronization phenomenon require deeper and further investigation (e.g., synchronization basin). In this sense, we hope that we can initiate a new direction of research of the synchronization problem with our work through this paper.