Information-Theoretic Sequential Framework to Elicit Dynamic High-Order Interactions in High-Dimensional Network Processes

Abstract

Complex networks of stochastic processes are crucial for modeling the dynamics of interacting systems, particularly those involving high-order interactions (HOIs) among three or more components. Traditional measures—such as mutual information (MI), interaction information (II), the redundancy-synergy index (RSI), and O-information (OI)—are typically limited to static analyses not accounting for temporal correlations and become computationally unfeasible in large networks due to the exponential growth of the number of interactions to be analyzed. To address these challenges, first a framework is introduced to extend these information-theoretic measures to dynamic processes. This includes the II rate (IIR), RSI rate (RSIR), and the OI rate gradient ($\Delta \mathrm{OIR}$), enabling the dynamic analysis of HOIs. Moreover, a stepwise strategy identifying groups of nodes (multiplets) that maximize either redundant or synergistic HOIs is devised, offering deeper insights into complex interdependencies. The framework is validated through simulations of networks composed of cascade, common drive, and common target mechanisms, modelled using vector autoregressive (VAR) processes. The feasibility of the proposed approach is demonstrated through its application in climatology, specifically by analyzing the relationships between climate variables that govern El Niño and the Southern Oscillation (ENSO) using historical climate data.

1. Introduction

The complexity of physical, biological, and technological systems arises from the intricate structural and functional interactions among their components. Modern science has shifted away from the reductionist view that the behavior of a networked system—composed of multiple potentially interacting units—can be fully understood by examining each unit in isolation. Additionally, the traditional approach of modeling networked systems through pairwise measures of coupling or causality [1,2] is being increasingly questioned, as it often fails to fully capture the emergent dynamics of the system. Growing evidence indicates that network dynamics cannot be explained solely by pairwise couplings, as higher-order interactions (HOIs)—those involving multiple system components—play a crucial role in shaping the overall behavior [3,4]. As a result, new methods are being developed to incorporate these HOIs into generalized network models [5]. Higher-order relationships involving three or more systems can be redundant (where multiple interacting systems share the same information) or synergistic (where a set of systems holds some information that cannot be seen from any subset). However, implementing such approaches remains challenging, particularly when interactions need to be inferred from data, as is often the case with biological or functional networks that represent the dynamic activity of physical systems.

Information-theoretic measures have been extensively used to assess higher-order interdependences in multivariate dynamical systems across diverse fields including neuroscience [6,7], finance [8], physiology [9,10], and climatology [11], where redundancies and synergies have been found to play meaningful roles [12,13,14,15,16]. The partial information decomposition (PID) framework has emerged as a prominent tool for analyzing complex higher-order statistical interdependencies [12,17]. PID is able to quantify as separate quantities the synergistic and the redundant information terms together with the unique information that is held by each source but not by the others. Several approaches have been proposed relying on the PID concept and can be divided into two families: directed approaches, which decompose information provided by source systems about a target, and undirected methods disentangle shared information among stochastic processes without assigning distinct roles, capturing the interplay between redundancy and synergy while remaining invariant under permutations [18,19]. Despite the advantages of PID approaches, a key limitation is the super-exponential growth of its atoms, making full decomposition impractical for large systems. To mitigate this challenge, coarse-grained PID metrics have been introduced that bypass the computation of individual atoms by focusing on their linear combinations [20,21,22]. However, this approach still struggles with scalability as the number of processes in the network increases. Another approach recently introduced by Stramaglia et al. [23] proposes a method based on partial conditioning in multivariate datasets for decomposing transfer entropy (TE) [24]; the method is very versatile, as it has been extended to assess feature importance in explainable artificial intelligence [25]. Specifically, this approach bridges dyadic and polyadic methodologies, providing a complementary perspective to those focused on assessing HOIs [12,20,21,22]. While these methods aim to decompose the total information about the target from multiple sources, the proposed framework focuses on analyzing how the environment modifies the TE between two processes of interest.

This work builds upon [23] by introducing a stepwise strategy to identify, starting from a target node in a complex network, groups of nodes (multiplets) that maximize either redundant or synergistic HOIs. Measures traditionally applied to static analyses of random variables—such as interaction information (II) [26], the redundancy-synergy index (RSI) [27], and O-information (OI) [20]—are herein extended to the analysis of stochastic processes. Specifically, the proposed method incorporates the interaction information rate (IIR), the redundancy-synergy index rate (RSIR), and the O-information rate gradient ($\Delta \mathrm{OIR}$) to enable the dynamic analysis of these interactions. The approach is validated through simulations of interaction mechanisms that generate HOIs, including cascade effects, common driver, and common target motifs. Finally, its feasibility is demonstrated through the analysis of a real-world climate network that captures the large-scale climate pattern of the El Niño–Southern Oscillation (ENSO), a key driver of global climate variability [28,29,30].

The article is structured as follows: Section 2 provides an overview of the proposed sequential framework, detailing its components, the mathematical formulation of the information-theoretic measures employed, and the linear parametric estimation method; Section 3 assesses the framework using two simulation scenarios, a five-variable autoregressive model and a collection of randomly generated networks with varying connection densities; finally, Section 4 illustrates the framework application to a climate network associated with ENSO variability.

The Matlab Toolbox implementing the greedy approach proposed in this work is freely available for download from the repository https://github.com/helderpinto97/GS-HOIs (accessed on 9 June 2025).

2. Sequential Method for Quantifying Higher-Order Interactions

This section outlines the methodology of the information-theoretic approach used to identify and quantify HOIs in complex networks. The section begins with a detailed presentation of the framework, introducing the IIR, RSIR, and the $\Delta \mathrm{OIR}$. It also includes a comprehensive discussion of the linear estimation approach used to compute these measures, highlighting its assumptions, limitations, and applicability to empirical data.

2.1. Sequential Procedure Outline

Let us consider a set consisting of M processes, $\mathbf{X}=\{X_1,\cdots ,X_M\}$. This approach aims to identify, in a computationally efficient manner, a subset of processes within $\mathbf{X}$ that form an informational circuit which is the most redundant or synergistic. To achieve this, the algorithm involves the following steps:

1.

Given the set $\mathbf{X}$, first identify the triplet that exhibits the highest level of redundancy or synergy according to a predefined metric.

2.

Expand the selected triplet iteratively by adding one process at a time, ensuring that its inclusion results in the maximal statistically significant increase in the overall redundancy or synergy of the joint multiplet according to the chosen metric. Repeat this until no additional inclusion produces a statistically significant increase in redundancy or synergy.

The measures employed in this study to evaluate the prevalence of redundancy or synergy within a subset of processes are as follows: (i) the interaction information rate (IIR) [10]; (ii) the redundancy-synergy index (RSI) [31], extended in this work to its rate form (RSIR); and (iii) the O-information rate gradient $(\Delta \mathrm{OIR})$ [10]. The three measures are formulated in Equations (1), (7) and (8), respectively. The estimation details of these measures are presented in Section 2.2. Importantly, the RSIR and $\Delta \mathrm{OIR}$ measures reduce to the IIR when applied to scenarios involving $M=3$ processes.

Therefore, the IIR serves as the basic measure used in this work to implement Step 1 of our algorithm. The IIR for the triplet of processes ${\mathbf{X}}_{\mathbf{1}}=\{X_{i_1},X_{i_2},X_{i_3}\}$, is defined as [10]:

$$\begin{array}{cc}\hfill I_{{\mathbf{X}}_1}=I_{X_{i_1};X_{i_2};X_{i_3}}& =I_{X_{i_1};X_{i_2}}+I_{X_{i_1};X_{i_3}}-I_{X_{i_1};X_{i_2},X_{i_3}}\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill & \hspace{1em} =I_{X_{i_1};X_{i_2}}-I_{X_{i_1};X_{i_2}|X_{i_3}},\hfill \end{array}$$

(2)

where $I_{(·);(·)}$ is the mutual information rate (MIR), which quantifies the information shared between the two processes per unit of time [32], and $I_{(·);(·)\left|\right(·)}$ is the conditional MIR (cMIR). The IIR $I_{X_{i_1};X_{i_2};X_{i_3}}>0$ indicates the prevalence of redundancy, while $I_{X_{i_1};X_{i_2};X_{i_3}}<0$ indicates the prevalence of synergy. Specifically, in the former case, the knowledge of $X_{i_3}$ reduces the information shared by $X_{i_1}$ and $X_{i_2}$, indicating that the statistical dependence between $X_{i_1}$ and $X_{i_2}$ is suppressed when $X_{i_3}$ is observed. Conversely, in the latter case, the knowledge of $X_{i_3}$ increases the information shared by $X_{i_1}$ and $X_{i_2}$, indicating that the statistical dependence between $X_{i_1}$ and $X_{i_2}$ emerges when $X_{i_3}$ is observed. Accordingly, step 1—identifying the most redundant or synergistic triplet—involves finding the maximum or minimum values of the IIR measure across all possible triplets within the set $\mathbf{X}$, respectively. The maximally redundant and maximally synergistic triplets identified in this way are denoted as ${\mathbf{X}}_1^M$ and ${\mathbf{X}}_1^m$, respectively.

The algorithm proceeds by iteratively selecting, for each step k ($k\ge 2$), the process $X_{i_{k+2}}^m$ such that

$$X_{i_{k+2}}^m=\underset{X_j\in \mathbf{X}\setminus {\mathbf{X}}_{k-1}}{arg\; min}{\mathrm{\Phi }}_{X_j}$$

(3)

in the case of a search for synergy, and selecting the process $X_{i_{k+2}}^M$ such that

$$X_{i_{k+2}}^M=\underset{X_j\in \mathbf{X}\setminus {\mathbf{X}}_{k-1}}{arg\; max}{\mathrm{\Phi }}_{X_j}$$

(4)

in the case of a search for redundancy. If the measure ${\mathrm{\Phi }}_{X_j}$ is statistically significant, the process satisfying either Equations (3) or (4) is retained, updating the multiplet to ${\mathbf{X}}_k^m=\{{\mathbf{X}}_{k-1}^m,X_{i_{k+2}}^m\}$ or ${\mathbf{X}}_k^M=\{{\mathbf{X}}_{k-1}^M,X_{i_{k+2}}^M\}$, respectively, and the procedure proceeds to the next step. If ${\mathrm{\Phi }}_{X_j}$ is not statistically significant, the procedure terminates, resulting in the final multiplet: ${\mathbf{X}}^m={\mathbf{X}}_{k-1}^m$ in the case of synergy, or ${\mathbf{X}}^M={\mathbf{X}}_{k-1}^M$ in the case of redundancy. The statistical significance of ${\mathrm{\Phi }}_{X_j}$ is assessed using the surrogate-based approach [23]. Processes are added one at a time to the current multiplet to construct a set that either maximizes or minimizes ${\mathrm{\Phi }}_{X_j}$. The greedy search stops when any observed change in ${\mathrm{\Phi }}_{X_j}$ can reasonably be attributed to chance. To determine this, the probability is estimated that the observed increase (or decrease) in ${\mathrm{\Phi }}_{X_j}$ is smaller (or bigger) than what would result from adding a process with similar individual statistical properties but not coupled with the current multiplet ${\mathbf{X}}_{k-1}$.

The function ${\mathrm{\Phi }}_{(·)}$ applied to a process $X_j\in \mathbf{X}\setminus {\mathbf{X}}_1$ is the objective function of the minimization (or maximization), and is chosen in three different ways as follows. In the case (i), the conditional IIR (cIIR) is employed, where for the step $k=2$:

$${\mathrm{\Phi }}_{X_j}^{\mathrm{IIR}}=I_{{\mathbf{X}}_1|X_j},$$

(5)

and for $k\ge 3$:

$${\mathrm{\Phi }}_{X_j}^{\mathrm{IIR}}=I_{{\mathbf{X}}_1|X_{i_4},\dots ,X_{i_{k+1}},X_j}.$$

(6)

In the case (ii), the RSIR is employed, which, at step k is defined as [31,33]:

$${\mathrm{\Phi }}_{X_j}^{\mathrm{RSIR}}=\sum _{X_i\in {\mathbf{X}}_{k-1}}{I}_{X_j;X_i}-I_{X_j;{\mathbf{X}}_{k-1}}.$$

(7)

Finally, analogously, in case (iii), $\Delta \mathrm{OIR}$ is used [10]:

$${\mathrm{\Phi }}_{X_j}^{\Delta \mathrm{OIR}}=\sum _{X_i\in {\mathbf{X}}_{k-1}}{I}_{X_j;{\mathbf{X}}_{k-1}\setminus X_i}-k{I}_{X_j;{\mathbf{X}}_{k-1}}.$$

(8)

In both Equations (7) and (8), $I_{(.);(.)}$ denotes the MIR.

In case (i), adding processes to the conditional vector can either increase or decrease the cIIR, revealing how the added processes interact in a synergistic or redundant way with the initial triplet ${\mathbf{X}}_1$. For this specific measure, the final multiplets ${\mathbf{X}}^M$ and ${\mathbf{X}}^m$ is obtained appending the initial triplet ${\mathbf{X}}_1$ to the conditioning vector. Regarding cases (ii) and (iii), Equations (7) and (8), the RSIR [31,33], and $\Delta \mathrm{OIR}$ [10] quantify how the tested process $X_j$ contributes to the balance between redundant and synergistic interdependencies with the current multiplet ${\mathbf{X}}_{k-1}$.

Before delving into the technical details regarding the estimation of these information measures, it is essential to highlight that each objective function—${\mathrm{\Phi }}_{X_j}^{\mathrm{IIR}}$, ${\mathrm{\Phi }}_{X_j}^{\mathrm{RSIR}}$, and ${\mathrm{\Phi }}_{X_j}^{\Delta \mathrm{OIR}}$—used to identify the most synergistic or redundant multiplets can be expressed as a sum of MIR terms. In particular, for ${\mathrm{\Phi }}_{X_j}^{\mathrm{RSIR}}$ and ${\mathrm{\Phi }}_{X_j}^{\Delta \mathrm{OIR}}$, this follows directly from Equations (7) and (8). Conversely, the cIIR measure can be expressed as a sum of cMIR terms. For instance, for $k=2$, Equation (5) can be decomposed as:

$$I_{{\mathbf{X}}_{\mathbf{1}}\mid X_j}=I_{X_{i_1};X_{i_2}\mid X_j}+I_{X_{i_1};X_{i_3}\mid X_j}-I_{X_{i_1};X_{i_2},X_{i_3}\mid X_j}.$$

(9)

Moreover, each cMIR term can be expressed in terms of MIR [34], leading to the following reformulation:

$$I_{{\mathbf{X}}_{\mathbf{1}}\mid X_j}=I_{X_{i_1};X_{i_2},X_{i_3},X_j}-I_{X_{i_1};X_{i_2},X_j}-I_{X_{i_1};X_{i_3},X_j}+I_{X_{i_1};X_j}.$$

(10)

The same reasoning applies at each step k. Therefore, for each step, it suffices to estimate only MIR terms to compute the functions ${\mathrm{\Phi }}^{\mathrm{IIR}},$${\mathrm{\Phi }}^{\mathrm{RSIR}}$ and ${\mathrm{\Phi }}^{\Delta \mathrm{OIR}}$. For this purpose, a parametric approach based on the vector autoregressive (VAR) model [35] is employed to estimate the MIR terms involved in the estimation of the three measures, the details of which are presented in the next section.

Another key point to emphasize is that, when analyzing real-world complex systems, the focus is often on a specific process and its interaction with the surrounding environment—particularly how other processes influence this target process, either synergistically or redundantly. This consideration can be integrated into Step 1 of the algorithm, where the selected triplet must include the predefined target process.

2.2. Linear Parametric Estimation of Higher-Order Interaction Information Measures

Consider a general system of M units, $\mathcal{S}=\{{\mathcal{S}}_1,\dots ,{\mathcal{S}}_M\}$ and focus on the two units $\mathcal{X}={\mathcal{S}}_i$ and $\mathcal{Y}={\mathcal{S}}_j$ while collecting the remaining $M-2$ units in the group $\mathcal{Z}=\mathcal{S}\setminus \{\mathcal{X},\mathcal{Y}\}$. Since the system $\mathcal{S}$ exhibits dynamic behavior, it is best described as a vector random process $S_n=[S_{1,n},\dots ,S_{M,n}]$, where the temporal dependence is indicated by the time index n, and $S_{i,n}$ represents the sampled value of the $i^{\mathrm{th}}$ process at time step n ($i=1,\dots ,M$, $n\in \mathbb{Z}$ for discrete-time processes). The equivalent notation evidencing the dynamics of the units $\{\mathcal{X},\mathcal{Y},\mathcal{Z}\}$ is $S_n=\left[X_n,Y_n,{\mathbf{Z}}_n\right]$. A widely used method to model the dynamics of $\mathbf{S}=\{X,Y,\mathbf{Z}\}$ is the linear vector autoregressive (VAR) model [35], given by:

$${\mathbf{S}}_n=\sum _{k=1}^p{\mathbf{A}}_k{\mathbf{S}}_{n-k}+U_n$$

(11)

where ${\mathbf{S}}_n$ and ${\mathbf{S}}_{n-k}$ represent M-dimensional variables that capture the state of the process at the current time step n and k steps in the past, respectively. The matrix ${\mathbf{A}}_k$ is an $M\times M$ coefficient matrix, while $U_n$ denotes an independent and identically distributed (i.i.d.) innovation process, with an $M\times M$ covariance matrix ${\Sigma }_U$. Given the full model, represented in Equation (11), restricted models can be used to describe the dynamics of any given subset of processes. Specifically, the dynamics of processes X, Y, and their joint dynamics in the multivariate process $\mathbf{W}=\{X,Y\}$ are represented by:

$$\begin{array}{cc}\hfill X_n& =\sum _{k=1}^q{A}_k^{\left(x\right)}{X}_{n-k}+U_{x,n}\hfill \\ \hfill Y_n& =\sum _{k=1}^q{A}_k^{\left(y\right)}{Y}_{n-k}+U_{y,n}\hfill \\ \hfill {\mathbf{W}}_n& =\sum _{k=1}^q{\mathbf{A}}_k^{\left(w\right)}{\mathbf{W}}_{n-k}+{\mathbf{U}}_{w,n}\hfill \end{array}$$

(12)

where q is the model order (theoretically $q\to \infty$). The parameters $A_k^{\left(x\right)},A_k^{\left(y\right)},{\mathbf{A}}_k^{\left(w\right)}$, and the residual covariances ${\Sigma }_{U_x},{\Sigma }_{U_y},{\Sigma }_{U_w}$, can be derived from the full model parameters ${\mathbf{A}}_k$ and ${\Sigma }_U$ by solving the Yule–Walker equations and reorganizing the covariance structure of $\mathbf{S}$ to relate it to X, Y, or $\mathbf{W}$ [9].

Assuming joint Gaussianity of the overall process $\mathbf{S}$, the MIR can be directly determined from the covariances of the residuals of the restricted models:

$$I_{X;Y}={\displaystyle \frac{1}{2}}log{\displaystyle \frac{{\Sigma }_{U_x}·{\Sigma }_{U_y}}{\left|{\Sigma }_{U_w}\right|}}.$$

(13)

The aforementioned procedure can be iterated to construct restricted models that characterize the dynamics of the vector process $\mathbf{Z}$ and the joint processes $\mathbf{V}=\{X,\mathbf{Z}\}$ and $\mathbf{R}=\{Y,\mathbf{Z}\}$. These models are formulated using VAR representations, as outlined in Equation (12), enabling the derivation of expressions for the MIR terms $I_{X;\mathbf{Z}}$ and $I_{X;Y,\mathbf{Z}}=I_{X;\mathbf{R}}$:

$$I_{X;\mathbf{Z}}={\displaystyle \frac{1}{2}}log{\displaystyle \frac{{\Sigma }_{U_x}\left|{\Sigma }_{U_z}\right|}{\left|{\Sigma }_{U_v}\right|}},$$

(14)

$$I_{X;Y,\mathbf{Z}}=I_{X;\mathbf{R}}={\displaystyle \frac{1}{2}}log{\displaystyle \frac{{\Sigma }_{U_x}\left|{\Sigma }_{U_r}\right|}{\left|{\Sigma }_U\right|}}.$$

(15)

Then, the formulations presented in Equations (13)–(15) are directly applied in Equations (10), (7) and (8) to compute ${\mathrm{\Phi }}^{\mathrm{IIR}}$, ${\mathrm{\Phi }}^{\mathrm{RSIR}}$, and ${\mathrm{\Phi }}^{\Delta \mathrm{OIR}}$, respectively, at each step of the algorithm. In practical analyses, these metrics are computed from synchronous time series realizations of the processes X, Y, and $\mathbf{Z}$. The parameters of the full model (11) are estimated using least squares [35], and these estimates are used to derive the covariance matrices of the residuals from the restricted models [9]. These covariance matrices are then employed in Equations (13)–(15) to estimate the MIR terms. The order p of the full model is selected using the Akaike Information Criterion (AIC) [36], while the order q of the restricted models is fixed at a high value $q=22$, in this work, to capture the decay of correlations at increasing lags [9,37]. This choice is supported by previous studies showing that, when evaluating information dynamics, all measures tend to stabilize to constant values even at relatively small lags, typically around $q=10$ [38].

While linear approaches—such as the VAR-based method used in this study—are inherently limited to capturing linear dependencies, they offer significant advantages in analytical tractability and computational efficiency compared to model-free alternatives. Importantly, estimating information-theoretic measures using VAR models assumes a linear model, not a linear process—a subtle but crucial distinction. The linear structure of the model does not require the underlying time series to be strictly linear. Nonlinearities in the data may not compromise model stability but can result in an incomplete representation of the system dynamics [39]. Nevertheless, as long as the fitted VAR model is stable—that is, all characteristic roots (or poles) lie strictly inside the unit circle in the complex plane [35]—the resulting MIR based measures remain valid. In this setting, the model acts as a linear approximation of the system. Wold’s theorem supports this view, showing that any stationary process can be represented by a linear model, potentially of infinite order, making it potentially non-parsimonious for time series generated by a nonlinear process [40,41]. Thus, the impact of nonlinearity on the estimation is nuanced and complex. The critical question remains whether the linear model, when applied to nonlinear data, provides a sufficiently accurate representation of the underlying dynamics to make valid inferences. Though preliminary research suggests the advantages of linear models, this remains an interesting and open area for future research.

The statistical significance of the measures is assessed at each step of the algorithm. Surrogate realizations of the candidate process for the multiplet are generated using the iterative amplitude-adjusted Fourier transform (IAAFT) method, which destroys cross-correlation while preserving autocorrelation and the power spectrum [42,43,44]. Moreover, the IAAFT surrogates preserve the amplitude distribution of the original data, including any non-Gaussian components when present [45]. Depending on whether redundancy or synergy is being evaluated, the process is retained if the corresponding probability is above or bellow a predefined threshold, respectively. In this work, the 95% and 5% percentiles of the surrogates distribution are used as thresholds. This procedure ensures that the value obtained at each step of the algorithm is statistically significant by comparing it to surrogate data. Specifically, the same surrogate-based significance testing is applied in Step 1 to evaluate the initial triplet interaction, which is measured using the IIR. If the observed interaction passes this significance test, the algorithm proceeds; otherwise, it terminates immediately, and the selected redundancy/synergy measure are set to zero.

3. Simulation Studies

The proposed sequential method for detecting and quantifying HOIs in a network of interacting processes is validated using theoretical examples based on simulated VAR models. In these simulations, various interaction measures are computed directly from the theoretical model parameters, ensuring the evaluation of the proposed method. These simulations demonstrate how our approach can effectively identify processes that interact in a redundant or synergistic manner with respect to a predefined target process.

3.1. Five-Dimensional VAR Model

First, the proposed method is evaluated using a five-variate VAR model of order 2, defined by the following equations [7]:

$$\begin{array}{cc}\hfill & X_{1,n}=2{\rho }_{1}cos\left(2\pi f_1\right)X_{1,n-1}-{\rho }_1^2{X}_{1,n-2}+\left(1-c\right)X_{2,n-1}+c{X}_{3,n-2}+U_{1,n},\hfill \\ \hfill & X_{2,n}=2{\rho }_{2}cos\left(2\pi f_2\right)X_{2,n-1}-{\rho }_2^2{X}_{2,n-2}+X_{4,n-1}+U_{2,n},\hfill \\ \hfill & X_{3,n}=2{\rho }_{3}cos\left(2\pi f_3\right)X_{3,n-1}-{\rho }_3^2{X}_{3,n-2}+X_{4,n-2}+U_{3,n},\hfill \\ \hfill & X_{4,n}=2{\rho }_{4}cos\left(2\pi f_4\right)X_{4,n-1}-{\rho }_4^2{X}_{4,n-2}+U_{4,n},\hfill \\ \hfill & X_{5,n}=2{\rho }_{5}cos\left(2\pi f_5\right)X_{5,n-1}-{\rho }_5^2{X}_{5,n-2}+U_{5,n},\hfill \end{array}$$

(16)

where ${\mathbf{U}}_n={\left[U_{1,n},\dots ,U_{5,n}\right]}^{\top }$ represents a zero-mean white Gaussian noise with covariance matrix ${\Sigma }_U$. The node $X_5$ represents an isolated node that does not interact with other processes in the network. The graphical representation of the model (16) is presented in Figure 1a. The parameter design in the set of Equation (16) is chosen to enable autonomous oscillations in all five processes. This is achieved by placing complex-conjugate poles in the complex plane representation of the transfer function of the vector process, with amplitudes ${\rho }_1,{\rho }_2,{\rho }_3,{\rho }_4,{\rho }_5$ and normalized frequency $f_1,f_2,f_3,f_4,f_5$. Causal interactions between processes $X_2$ and $X_1$ are controlled by the coupling parameter $1-c$, while interactions between $X_3$ and $X_1$ are modulated by the parameter c. Specifically, the parameters were set as ${\rho }_1={\rho }_2={\rho }_3={\rho }_3={\rho }_5=0.8$, ${\displaystyle \frac{f_1}{f_s}}={\displaystyle \frac{f_2}{f_s}}={\displaystyle \frac{f_3}{f_s}}={\displaystyle \frac{f_4}{f_s}}={\displaystyle \frac{f_5}{f_s}}=0.1$, which results in oscillatory activity at 0.1 Hz when the sampling frequency $f_s=1$ is considered. The parameter c was allowed to vary freely between 0 and 1.

The median and the 5th–95th percentile range of ${\mathrm{\Phi }}_{\mathrm{IIR}}$, ${\mathrm{\Phi }}_{\mathrm{RSI}}$, and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}},$ considering the process $X_1$ as target, are shown in Figure 1b, Figure 1c, and Figure 1d, respectively, along with the most frequently observed redundant (red) and synergistic (blue) multiplets across the 100 generated realizations. In these heatmaps, each cell is color-coded to indicate whether the process was part of the corresponding multiplet or not. The three information measures used in the ${\mathrm{\Phi }}_{(.)}$ function exhibit a similar pattern: increasing c enhances synergy up to $c=0.5$, after which it declines. Redundancy follows the opposite trend, decreasing until $c=0.5$ and then increasing. This behavior is expected, as low c values lead to redundancy dominance due to the cascade $X_4\to X_2\to X_1$ [7]. At intermediate c, synergy and redundancy coexist, with synergy peaking at $c=0.5$ due to the common child structure $X_2\to X_1\leftarrow X_3$, while redundancy reaches its minimum but persists due to the cascades $X_4\to X_2\to X_1$ and $X_4\to X_3\to X_1$. Beyond this point, redundancy increases while synergy declines to zero, driven by the dominance of $X_4\to X_3\to X_1$. This trend is captured by all three measures, though differences arise in the multiplets they identify. ${\mathrm{\Phi }}_{\mathrm{IIR}}$ clearly detects the shift between the cascades $X_4\to X_2\to X_1$ and $X_4\to X_3\to X_1$ as c increases. Meanwhile, ${\mathrm{\Phi }}_{\mathrm{RSIR}}$ and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$ identify $[X_1,X_2,X_3,X_4]$ as the most redundant multiplet, capturing both cascades and the common driver structure $X_2\leftarrow X_4\to X_3$. Regarding synergy, notable differences are observed among the ${\mathrm{\Phi }}_{\mathrm{IIR}}$, ${\mathrm{\Phi }}_{\mathrm{RSIR}}$, and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$ multiplets. In all three measures, the common empty multiplet appears at both low and high values of c, as expected, since synergy is absent in these cases and the network is dominated by cascade effects. However, at intermediate values of c, the multiplet identified by ${\mathrm{\Phi }}_{\mathrm{IIR}}$ differs from those found using ${\mathrm{\Phi }}_{\mathrm{RSIR}}$ and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$. The latter two measures consistently identify the multiplet $[X_1,X_2,X_3]$, revealing the common child structure $X_2\to X_1\leftarrow X_3$. In contrast, ${\mathrm{\Phi }}_{\mathrm{IIR}}$ also includes the process $X_4$ in the multiplet. Specifically, for this measure, the final multiplet is formed by extending the initial triplet with the conditioning vector (see Section 2), where $X_4$ is part of the conditioning vector. By incorporating $X_4$ into the conditioning vector, the visibility of the structure $X_2\to X_1\leftarrow X_3$ is enhanced. Thus, the knowledge of $X_4$ helps clarify the interactions between $X_1$, $X_2$, and $X_3$. This distinction can explain the differences observed in the multiplets identified for redundancy when using IIR when compared to RSIR and $\Delta \mathrm{OIR}$. In RSIR and $\Delta \mathrm{OIR}$, where a new process is added to the multiplet at each step, both methods detect three redundant motifs: the two cascade structures and the common driver structure $X_2\leftarrow X_4\to X_3$. However, since IIR is formulated in terms of conditioning, it selects processes that enhance the visibility of the interaction detected in the initial triplet rather than simply adding a new process at each step. For instance, for lower values of c, IIR identifies the dominant cascade effect $X_4\to X_2\to X_1$. However, no additional process is included in the conditioning vector because neither $X_3$ nor $X_5$ contains information relevant to the cascade effect detected in the first step. A similar reasoning applies to higher values of c.

3.2. Randomly Connected Networks

The simulated datasets were generated according to different ground-truth networks by means of an VAR model used as a generator filter [46]. The simulated multivariate time series ($M=13$) were generated as realizations of a VAR process of order $p=3$ fed by Gaussian noise with variance equal to 1. Two different matrices of AR parameters were generated by the SEED-G software according with a predefined network density of 10% and 30% (16 and 48 out of 156 possible connections). The parameters were randomly assigned in the range $[1,p]$ and their values were selected in the interval $[-0.9,0.9]$. This value is imposed only at one lag among the possible p lags, randomly selected.

Figure 2a,b illustrate a randomly generated network with 13 nodes, featuring edge densities of 10% and 30%, respectively. The increase in density enhances connectivity, which can influence both the structural and dynamic properties of the network. Figure 2c presents the distribution of redundancy and synergy, computed using ${\mathrm{\Phi }}_{\mathrm{IIR}}$, ${\mathrm{\Phi }}_{\mathrm{RSIR}}$, and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$, based on 20 realizations of the network shown in Figure 2a. These distributions are evaluated for all possible target nodes, $X_1,\dots ,X_{13}$. Similarly, Figure 2d displays the redundancy and synergy distributions for the same three measures, obtained from 20 realizations of the denser network in Figure 2b, again considering all possible target processes, $X_1,\dots ,X_{13}$. For the network with 10% density (Figure 2c), the results show a predominance of redundancy over synergy across all information measures used in the function ${\mathrm{\Phi }}_{(·)}$. However, notable synergy peaks for the three measures—${\mathrm{\Phi }}_{\mathrm{IIR}}$, ${\mathrm{\Phi }}_{\mathrm{RSIR}}$, and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$—are observed when $X_2$, $X_8$, and $X_{11}$ are considered as target nodes. This can be explained by the common child structure $X_2\to X_{11}\leftarrow X_8$, as seen in Figure 2a.

Increasing the network density parameter to 0.3 results in a greater number of connections, as shown in Figure 2b. Additionally, Figure 2d illustrates an increase in redundancy across all three measured parameters compared to the lower-density network, which is summarized in Figure 2c. Furthermore, as shown in Figure 2d, synergy arises when $X_5$, $X_7$, and $X_{12}$ are considered as target processes, which can be explained by the common child structure $X_5\to X_{12}\leftarrow X_7$. Alternatively, this can be explained by comparing the prevalence of synergistic structures involving these three nodes with that of redundant structures. All three measures considered—${\mathrm{\Phi }}_{IIR}$, ${\mathrm{\Phi }}_{RSIR}$, and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$—identified the same initial triplet $[X_5,X_7,X_{12}]$. However, while RSIR and $\Delta \mathrm{OIR}$ retained this triplet as the final multiplet, IIR included additional processes, $X_2$ and $X_{13}$, in the conditioning vector (result omitted here for simplicity). Although these two processes are associated with redundancy, as they follow the cascade $X_{13}\to X_2\to X_{12}$, their inclusion provides valuable insight into the synergistic interactions within the initial triplet. Another important observation is that the estimated values of redundancy and synergy, calculated for the three measures across both random networks, show minimal bias when compared to the theoretical values, presented as

Table 1. Theoretical values for redundancy and synergy, for ${\mathrm{\Phi }}_{\mathrm{IIR}}$, ${\mathrm{\Phi }}_{\mathrm{RSIR}}$, and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$, obtained from the random network with parameter densities of 0.1 and 0.3, illustrated in Figure 2a and Figure 2b, respectively.

		Measure/Target	${\mathit{X}}_1$	${\mathit{X}}_2$	${\mathit{X}}_3$	${\mathit{X}}_4$	${\mathit{X}}_5$	${\mathit{X}}_6$	${\mathit{X}}_7$	${\mathit{X}}_8$	${\mathit{X}}_9$	${\mathit{X}}_{10}$	${\mathit{X}}_{11}$	${\mathit{X}}_{12}$	${\mathit{X}}_{13}$
Density 0.1	Redundancy	${\mathrm{\Phi }}_{\mathrm{IIR}}$	0.0391	0.0710	0.0783	0.0783	0.0690	0.0783	0.2430	0	0.2430	0.0835	0.0975	0.2430	0
		${\mathrm{\Phi }}_{\mathrm{RSIR}}$	0.0932	0.2589	0.0783	0.0783	0.2574	0.0783	0.2126	0.1174	0.2126	0.2482	0.3154	0.2126	0.1174
		${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$	0.0783	0.2209	0.0783	0.0783	0.2206	0.0783	0.2126	0.1025	0.2126	0.2118	0.2252	0.2126	0.1025
	Synergy	${\mathrm{\Phi }}_{\mathrm{IIR}}$	−0.0026	−0.0968	−0.0457	−0.0169	−0.0419	−0.0150	−0.0296	−0.0968	−0.0457	−0.0440	−0.0968	−0.0457	0
		${\mathrm{\Phi }}_{\mathrm{RSIR}}$	−0.0377	−0.0968	−0.0457	−0.0169	−0.0419	−0.0150	−0.0296	−0.0968	−0.0457	−0.0440	−0.0968	−0.0457	−0.0227
		${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$	−0.0348	−0.0968	−0.0457	−0.0169	−0.0419	−0.0150	−0.0296	−0.0968	−0.0457	−0.0440	−0.0968	−0.0457	−0.0208
Density 0.3	Redundancy	${\mathrm{\Phi }}_{\mathrm{IIR}}$	0.2544	0.2624	0.1728	0.3388	0.3388	0.2514	0.1921	0.2781	0.2091	0.3102	0.3388	0.2564	0.2395
		${\mathrm{\Phi }}_{\mathrm{RSIR}}$	1.7048	1.7048	1.6904	1.7048	1.7048	1.7048	1.7843	1.7048	1.6856	1.7048	1.7048	1.7048	1.7048
		${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$	0.3912	0.4293	0.4472	0.3609	0.3609	0.3819	0.3739	0.3982	0.3787	0.3895	0.3609	0.3848	0.4211
	Synergy	${\mathrm{\Phi }}_{\mathrm{IIR}}$	0	0	0	0	−0.1368	0	−0.1368	0	0	0	0	−0.1368	0
		${\mathrm{\Phi }}_{\mathrm{RSIR}}$	0	0	0	0	−0.0533	0	−0.0533	0	0	0	0	−0.0533	0
		${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$	0	0	0	0	−0.0533	0	−0.0533	0	0	0	0	−0.0533	0

.

Overall our simulations show that increasing network density leads to a significant rise in HOIs, as expected, because a denser network offers more connections, which helps collective behaviors to emerge. Specifically, we found that more densely connected networks exhibit a predominance of redundant interactions, which can obscure underlying synergistic effects. This behavior can be explained by the presence of common-child structures, whereas redundant interactions are more strongly linked to cascade and common-driver effects—a finding consistent with prior studies [7,10].

4. Application to a Climate Network

The feasibility of the proposed framework is demonstrated through an exemplary case study in climate science, i.e., the network of interactions among the most representative indices descriptive of El Niño and the Southern Oscillation (ENSO) [29,30]. El Niño and the ENSO is a climate pattern involving periodic warming (El Niño) and cooling (La Niña) of the central and eastern Pacific Ocean, affecting global weather patterns through atmospheric pressure shifts [28]. ENSO is considered the most prominent form of interannual climate variability on Earth [29]. Since the exact mechanisms triggering ENSO warm and cool events are not fully understood, analyzing the statistical relationships among climate variables is crucial. The resulting network of interactions is tightly connected through various feedback loops that influence climate patterns over multiple years [30,47]. This highlights the need to study HOIs using a broader, dynamic approach.

The network considered was composed by 13 time series publicly available [30], covering the period of 66 years from 1950 to 2016: NINO34 (the East Central Tropical Pacific sea surface temperature anomaly), SOI (Southern Oscillation Index), AIR (All Indian Rainfall), AMO (Atlantic Multidecadal Oscillation), GMT (Global Mean Temperature anomaly), HURR (total number of hurricanes or named tropical storms per month in the Atlantic region), NOA (North Atlantic Oscillation of pressure anomalies over the Atlantic), NP (North Pacific pattern of sea level pressure), NTA (North Tropical Atlantic), PDO (Pacific Decadal Oscillation), QBO (Quasi-Biennial Oscillation), Sahel (Sahel Standardized Rainfall), and TSA (Tropical Southern Atlantic Index). It is important to note that these variables may not represent entirely distinct processes, as underlying latent factors could influence multiple indices simultaneously. This highlights the necessity of an analytical approach that accounts for shared information and potential interdependencies among the variables, like the one presented in this work.

As noted in Section 2.2, although the VAR model only partially captures the system’s dynamics—given that climate data often exhibit nonlinear behavior—it still provides valuable insights by serving as a linear approximation. Previous studies on climate networks have demonstrated that linear models can yield meaningful representations of underlying dynamics [48], particularly in large-scale geophysical systems such as those governing ENSO [30,49,50]. Nevertheless, the results should be interpreted with appropriate caution due to the model inherent limitations. Additionally, our findings are compared with those of previous studies, especially [30], which employed a similar VAR-based approach to estimate causality measures, allowing for a direct and meaningful comparison.

Prior to the analysis, the time series were detrended and deseasonalized whenever necessary and rescaling the series to have zero mean and unitary variance [30]. The VAR model order was fixed at $p=3$ based on AIC minimization, balancing fit and complexity. The fitted model was stable, with all characteristic roots inside the unit circle [35]. Stationarity of all 13 variables was confirmed by the augmented Dickey–Fuller test at $\alpha =0.05$, with p-values below 0.001 [51]. Residual normality assessed by the Lilliefors test [52] showed non-normality for some variables, notably HURR, Sahel and AIR with high kurtosis, while others (e.g., NP, SOI, NTA) exhibited mild, symmetric deviations due to heavy tails. Despite these departures, the residuals approximate symmetry and zero mean justify cautious use of the VAR model in subsequent analysis. Although the multivariate Portmanteau test was rejected [53] (p-value = 0.035), visual inspection revealed no severe autocorrelation or cross-correlation issues (additional figures that support this diagnostics can be found in Supplementary Material).

Figure 3 summarizes the redundant multiplets (first row in red) and synergistic multiplets (second row in blue) for all possible target processes, considering ${\mathrm{\Phi }}_{\mathrm{IIR}}$, ${\mathrm{\Phi }}_{\mathrm{RSIR}}$, and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$. First, we focus on one of the most important processes in the network: the Southern Oscillation index (SOI). A key observation is that the initial triplet identified for all three measures, regarding redundancy, was the same: $[\mathrm{SOI},\mathrm{GMT},\mathrm{NINO}34]$. This corroborates in some extent the results presented in [30], where authors found that SOI may act as a common driver influencing both NINO34 and GMT. For ${\mathrm{\Phi }}_{\mathrm{IIR}}$, no additional redundant processes were included in the multiplet. However, for ${\mathrm{\Phi }}_{\mathrm{RSIR}}$ and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$, the final multiplet expanded significantly to $[\mathrm{SOI},\mathrm{GMT},\mathrm{NINO}34,\mathrm{NTA},\mathrm{AMO},\mathrm{NP},\mathrm{PDO}]$. This expansion suggests that RSIR and $\Delta \mathrm{OIR}$ capture a broader set of redundant motifs, potentially uncovering cascading effects such as $\mathrm{NINO}34\to \mathrm{AMO}\to \mathrm{SOI}$ and $\mathrm{NINO}34\to \mathrm{NTA}\to \mathrm{SOI}$, as observed in [30]. Furthermore, this finding supports that the triplet $[\mathrm{NINO}34,\mathrm{PDO},\mathrm{SOI}]$ interacts in a redundant manner, consistent with the results reported in [54] using the partial information rate decomposition framework. Regarding synergy, for all three measures, the initial triplet that contributed to the final multiplet was the same regardless of the considered approach: $[\mathrm{SOI},\mathrm{TSA},\mathrm{AIR}]$. This finding supports the presence of a common child structure, where SOI is influenced by the joint effects of AIR and TSA, potentially indicating an underlying atmospheric teleconnection pattern: $[\mathrm{AIR},\mathrm{TSA}]\to \mathrm{SOI}$ [30]. These results also partially align with the findings of [23], where the authors observed that the pair NTA and TSA contribute to the synergy in the pairwise interaction $\mathrm{NINO}34\to \mathrm{SOI}$. The differences observed in this study could be attributed to the methodological approaches: the measures used in the greedy framework presented here are undirected and capture HOIs in a more general sense, whereas [23] employed the conditioning of TE [24], a pairwise causality measure, to infer the presence of HOIs.

Interestingly, the HURR process was not present in any redundant or synergistic multiplet for any target considered, suggesting that this process does not play a role in the emergence of higher-order effects. This could be related with the fact is strongly influenced by ENSO, but it is generally considered an outcome rather than an active interacting variable in redundancy or synergy analyses but more as a pairwise interaction [55,56].

From Figure 3 and

Table 2. Values of redundancy and synergy obtained using ${\mathrm{\Phi }}_{\mathrm{IIR}}$, ${\mathrm{\Phi }}_{\mathrm{RSIR}}$, and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$, considering all possible network processes as targets.

	Measure\Target	Sahel	NAO	NP	GMT	HURR	AMO	SOI	TSA	QBO	PDO	NINO34	NTA	AIR
Redundancy	${\mathrm{\Phi }}_{\mathrm{IIR}}$	0.0090	0.0194	0.0162	0.0188	0.0069	0.0194	0.0188	0.0083	0.0034	0.0185	0.0188	0.0204	0.0165
	${\mathrm{\Phi }}_{\mathrm{RSIR}}$	0.0514	0.0194	0.0492	0.0485	0.0213	0.0194	0.0485	0.0499	0.0591	0.0581	0.0485	0.0194	0.0636
	${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$	0.0178	0.0194	0.0237	0.0237	0.0188	0.0194	0.0237	0.0205	0.0277	0.0288	0.0237	0.0194	0.0198
Synergy	${\mathrm{\Phi }}_{\mathrm{IIR}}$	−0.0029	−0.0023	−0.0029	−0.0021	−0.0007	−0.0012	−0.0033	−0.0037	−0.0021	−0.0022	−0.0037	−0.0023	−0.0037
	${\mathrm{\Phi }}_{\mathrm{RSIR}}$	−0.0022	−0.0021	−0.0022	−0.0021	−0.0007	−0.0012	−0.0024	−0.0025	−0.0021	−0.0022	−0.0025	−0.0021	−0.0025
	${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$	−0.0022	−0.0021	−0.0022	−0.0021	−0.0007	−0.0012	−0.0024	−0.0025	−0.0021	−0.0022	−0.0025	−0.0021	−0.0025

, it is clear that redundant interactions dominate the system behaviors over synergistic ones. This trend holds across all measures considered—${\mathrm{\Phi }}_{\mathrm{IIR}}$, ${\mathrm{\Phi }}_{\mathrm{RSIR}}$, and ${\mathrm{\Phi }}_{\Delta \mathrm{OIR}}$—where synergy values remained consistently close to zero. Given the higher density of connections in the climate network [30], this result aligns with the results of the simulation studies as discussed in Section 3.2, showing that the number of pairwise connections influences the presence of HOIs, particularly redundancy.

5. Conclusions

This study introduces a novel sequential approach based on the measures IIR, RSIR, and $\Delta \mathrm{OIR}$ to detect and quantify HOIs in complex, high-dimensional networks. The proposed method iteratively identifies multiplets that interact redundantly or synergistically with a predefined target process. By doing so, it circumvents the computational explosion associated with exhaustively considering all possible combinations, as seen in the approaches presented in [12,20,33], which scale exponentially with the number of network nodes. This makes the method particularly well-suited for analyzing large-scale systems, such as biological and climate networks, where understanding HOIs is crucial for uncovering emergent behaviors and collective dynamics.

The simulations demonstrate that the proposed method effectively detects and quantifies redundancy and synergy in complex networks. In a five-node simulation, it accurately identified both effects, aligning with theoretical expectations. In random networks, higher connection density increased high-order interactions, with redundancy becoming dominant. The application to a climate network, the method revealed strong redundancy—especially between SOI and NINO34—and identified a synergistic triplet [SOI, TSA, AIR], suggesting an atmospheric teleconnection pattern: $[\mathrm{AIR},\mathrm{TSA}]\to \mathrm{SOI}$.

One important point to emphasize is that the linear estimation approach used to compute information-theoretic measures captures only linear interactions, which can limit its applicability to real-world networks—such as the climate networks analyzed in this work—that often exhibit nonlinear dynamics. Therefore, future work should focus on implementing this framework using model-free estimation methods, such as the nearest-neighbor approach [57], which can capture both linear and nonlinear dependencies more effectively. Despite this limitation, the proposed sequential framework demonstrates strong capability in efficiently handling complex networks with many interacting processes at relatively low computational cost. This advantage opens new avenues for exploring intricate biological systems and their interactions. In particular, the methodology shows great promise for studying physiological networks—such as cardiovascular and respiratory interactions [9,34]—as well as brain networks [58] and brain–heart interactions [59,60,61].