# Compensated Transfer Entropy as a Tool for Reliably Estimating Information Transfer in Physiological Time Series

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methods

#### 2.1. Transfer Entropy

**Z**= {Z

^{(k)}}

_{k = 1,...,M-2}. We develop our framework under the assumption of stationarity, which allows to perform estimations replacing ensemble averages with time averages (for non-stationary formulations see, e.g., [10] and references therein). Accordingly, we denote x, y and

**z**as the stationary stochastic processes describing the state visited by the systems X, Y and

**Z**over time, and x

_{n}, y

_{n}and

**z**

_{n}as the stochastic variables obtained sampling the processes at the time n. Moreover, let x

_{t}

_{:n}represent the vector variable describing all the states visited by X from time t up to time n (assuming n as the present time and setting the origin of time at t = 1, x

_{1:n-1}represents the whole past history of the process x). Then, the transfer entropy (TE) from X to Y conditioned to

**Z**is defined as:

**a**) is the probability associated with the vector variable

**a**, and p(b|

**a**) = p(

**a**,b)/p(

**a**) is the probability of the scalar variable b conditioned to

**a**. The conditional probabilities used in (1) can be interpreted as transition probabilities, in the sense that they describe the dynamics of the transition of the destination system from its past states to its present state, accounting for the past of the other processes. Utilization of the transition probabilities as defined in (1) makes the resulting measure able to quantify the extent to which the transition of the destination system Y into its present state is affected by the past states visited by the source system X. Specifically, the TE quantifies the information provided by the past states of X about the present state of Y that is not already provided by the past of Y or any other system included in

**Z**. The formulation presented in (1) is an extension of the original TE measure proposed for bivariate systems [1] to the case of multiple interacting processes. The multivariate (conditional) TE formulation, also denoted as partial TE [25], rules out the information shared between X and Y that could be possibly triggered by their common interaction with

**Z**. As such, this formulation fulfills for multivariate systems the correspondence between TE and the concept of Granger causality [19], that refers to the exclusive consideration of direct effects between two processes after resolving the conditional effects of the other observed processes. Note that the conditional formulation has been shown essential for taking under control the effects of common confounders in experimental contexts such as cardiovascular variability analysis [24] or neural signal analysis [26]. In the following, we will indicate Granger causal effects from the system X to the system Y with the notation X→Y (or x

_{1:n-1}→y

_{n}if we refer to the corresponding processes).

_{n}|y

_{1:n-1},

**z**

_{1:n-1}) = H(y

_{1:n},

**z**

_{1:n-1}) − H(y

_{1:n-1},

**z**

_{1:n-1}) and H(y

_{n}|x

_{1:n-1},y

_{1:n-1},

**z**

_{1:n-1}) = H(x

_{1:n-1},y

_{1:n},

**z**

_{1:n-1}) − H(x

_{1:n-1},y

_{1:n-1},

**z**

_{1:n-1}), where the entropy of any vector variable

**a**is defined as H(

**a**) = −∑ p(

**a**)·log p(

**a**) and is usually measured in bits when the base of the logarithm is 2 or in nats when the base is e (as in the present study).

#### 2.2. Compensated Transfer Entropy

_{n}→y

_{n}). Instantaneous effects are the practical evidence of the concept of instantaneous causality, which is a known issue in causal analysis [16,19]. In practice, instantaneous causality between two time series may either have a proper causal meaning, when the time resolution of the measurements is lower than the time scale of the lagged causal influences between the underlying processes, or be void of such causal meaning, in the case of common driving effects occurring when an unmeasured process simultaneously affects the two processes under analysis [17]. In either case, instantaneous causality has an impact on the estimation of the TE: if it is causally meaningful, the analysis misses the zero-lag effect x

_{n}→y

_{n}, if not, the analysis includes potential spurious effects taking the form x

_{1:n-1}→x

_{n}→y

_{n}; these misleading detections may impair respectively the sensitivity and the specificity of TE estimation.

_{n}, is incorporated in the second CE term used for TE computation:

_{n}plays a similar role as x

_{1:n-1}), so that the present state of the source system is taken as causally relevant to account for instantaneous causality in TE computation. If, on the contrary, instantaneous effects are deemed as non causally meaningful, the zero-lag term is incorporated both in the first and in the second CE terms used for TE computation:

_{n}plays a similar role as y

_{1:n-1}and

**z**

_{1:n}), so that the present state of the source system is compensated to remove instantaneous causality from TE computation. The compensation performed in (5) is alternative to the test of time-shifted data recently proposed to detect instantaneous mixing between coupled processes [4]. Note that in both compensations in (4) and (5) instantaneous effects possibly occurring from any scalar element of

**Z**towards Y are conditioned out considering the present term

**z**

_{n}, in addition to the past terms

**z**

_{1:n-1}, in the two CE computations; this is done to avoid that indirect effects x

_{1:n-1}→

**z**

_{n}→y

_{n}were misinterpreted as the presence of predictive information transfer from the system X to the system Y. Note that, in the absence of instantaneous causality among the observed processes, the two cTE measures defined in (4) and (5) reduce to the traditional TE.

#### 2.3. Estimation Approach

**Z**}. The most commonly followed approach is to perform uniform time delay embedding, whereby each scalar process is mapped into trajectories described by delayed coordinates uniformly spaced in time [29]. In this way the past history of the source process, x

_{1:n-1}, is approximated with the d-dimensional delay vector [x

_{n-u-(d-1)τ}, ..., x

_{n-u-τ}, x

_{n-u}], with τ and u representing the so-called embedding time and prediction time. This procedure suffers from many disadvantages: first, univariate embedding whereby coordinate selection is performed separately for each process does not guarantee optimality of the reconstruction for the multivariate state space [30]; second, selection of the embedding parameters d, τ and u is not straightforward, as many competing criteria exist which are all heuristic and somewhat mutually exclusive [31]; third, the inclusion of irrelevant coordinates consequent to the use of an uniform embedding exposes the reconstruction procedure to the so called “curse of dimensionality”, a concept related to the sparsity of the available data within state spaces of increasing volume [32]. All these problems become more cumbersome when the available realizations are of short length, as commonly happens in physiological time series analysis due to lack of data or stationarity requirements. To counteract these problems, we describe in the following a TE/cTE estimation strategy based on the utilization of a non-uniform embedding procedure combined with a corrected CE estimator [15].

_{n}|y

_{1:n-1},

**z**

_{1:n}) will be the set Ω

_{1}= {y

_{n}

_{-1},...,y

_{n}

_{-L},

**z**

_{n},

**z**

_{n}

_{-1},...,

**z**

_{n}

_{-L}}, and the candidate set for the estimation of H(y

_{n}|x

_{1:n}, y

_{1:n-1},

**z**

_{1:n}) in (4) will be the set Ω

_{2}= {Ω

_{1},x

_{n},x

_{n}

_{-1},...,x

_{n}

_{-L}} (L is the number of time lagged terms to be tested for each scalar process). Given the generic candidate set Ω, the procedure for estimating the CE H(y

_{n}|Ω) starts with an empty embedding vector V

_{0}= [·], and proceeds as follows: (i) at each step k ≥ 1, form the candidate vector [s,V

_{k}

_{-1}], where s is an element of Ω not already included in V

_{k}

_{-1}, and compute the CE of the destination process Y given the considered candidate vector, H(y

_{n}|[s,V

_{k}

_{-1}]); (ii) repeat step (i) for all possible candidates, and then retain the candidate for which the estimated CE is minimum, i.e., set V

_{k}= [s′,V

_{k}

_{-1}] where s′ = arg min

_{s}H(y

_{n}|[s,V

_{k}

_{-1}]); (iii) terminate the procedure when irrelevant terms begin to be selected, i.e. when the decrease of CE is no longer significant; according to the estimation procedure detailed below, this corresponds to stop the iterations at the step k′ such that H(y

_{n}|V

_{k′}) ≥ H(y

_{n}|V

_{k′}

_{-1}), and set V

_{K}= V

_{k′}

_{-1}as embedding vector. With this procedure, only the components that effectively contribute to resolving the uncertainty of the target process (in terms of CE reduction) are included into the embedding vector, while the irrelevant components are left out. This feature, together with the termination criterion which prevents the selection of new terms when they do not bring further resolution of uncertainty for the destination process, help escaping the curse of dimensionality for multivariate CE estimation. Moreover the procedure avoids the nontrivial task of setting the embedding parameters (the only parameter is the number L of candidates to be tested for each process, which can be as high as allowed by the affordable computational times). It is worth noting that the proposed sequential procedure for candidate selection takes into account one term at a time, somehow disregarding joint effects that more candidates may have on CE reduction. As a consequence, the sequential instead of exhaustive strategy does not guarantee convergence to the absolute minimum of CE, and thus does not assure a semipositive value for the TE/cTE measures estimated according to (3), (4) and (5). However a sequential approach is often necessary in practical analysis, since exhaustive exploration of all possible combinations of candidate terms would become computationally intractable still at low embedding dimensions.

_{n}conditioned to the vector variable V

_{k}, seen as the difference of two Shannon entropies: H(y

_{n}|V

_{k}) = H(y

_{n},V

_{k}) − H(V

_{k}). A major problem in estimating CE is the bias towards zero which affects the estimates as the dimension of the reconstructed state space grows higher [33,34]. Since the bias increases progressively with the embedding dimension, its occurrence also prevents from being able to reveal the inclusion of irrelevant terms into the embedding vector by looking at the estimated CE; in other words, since the estimated CE decreases progressively as a result of the bias rather than of the inclusion of relevant terms, the iterations of the sequential procedure for nonuniform embedding cannot be properly stopped. To deal with this important problem, we propose to compensate the CE bias adding a corrective term as proposed by Porta et al. [28,34], in order to achieve a minimum in the estimated CE which serves as stopping criterion for the embedding procedure. The idea is based on the consideration that, for time series of limited length, the CE estimation bias is due to the isolation of the points in the k-dimensional state space identified by the vectors V

_{k}; such an isolation becomes more and more severe as the dimension k increases. Since isolated points tend to give the same contribution to the two entropy terms forming CE (i.e., p(V

_{k}) ≈ p(y

_{n},V

_{k}) if V

_{k}is an isolated point), their contribution to the CE estimate will be null; therefore, the CE estimate decreases progressively towards zero at increasing the embedding dimension [i.e., when k is high compared to the series length, H(V

_{k}) ≈ H(y

_{n},V

_{k}) and thus H(y

_{n}|V

_{k}) ≈ 0], even for completely unpredictable processes for which conditioning should not decrease the information carried. This misleading indication of predictability in the analysis of short time series is counteracted introducing a corrective term for the CE. The correction is meant at quantifying the fraction of isolated points V

_{k}in the k-dimensional state space, denoted as n(V

_{k}), and on substituting their null contribution with the maximal information amount carried by a white noise with the same marginal distribution of the observed process y

_{n}[i.e., with H(y

_{n})]. The resulting final estimate is obtained adding the corrective term n(V

_{k})H(y

_{n}) to the estimated CE H(y

_{n}|V

_{k}). In the present study, practical implementation of the correction is performed in the context of entropy estimation through uniform quantization [15,28,34]. Briefly, each time series is coarse grained spreading its dynamics over Q quantization levels, so that the state space containing the vectors V

_{k}is partitioned in Q

^{k}disjoint hypercubes. As all points falling within the same hypercube are considered indistinguishable to each other, the Shannon entropy is estimated approximating the probabilities with the frequency of visitation of the hypercubes. Partitioning in disjoint hypercubes helps also in quantifying the fraction of isolated points n(V

_{k}), which is taken simply as the fraction of points found only once inside the hypercubes.

## 3. Validation

^{K}≈ N for series of length N (with K the embedding dimension) [15,28,34].

#### 3.1. Physiologically Meaningful Instantaneous Causality

_{n}, v

_{n}and w

_{n}are independent white noises with zero mean and variance σ

^{2}

_{u}= 5, σ

^{2}

_{v}= 1, and σ

^{2}

_{w}= 1. According to (6), the processes X and Y are represented as second order autoregressive processes described by two complex-conjugate poles with modulus ρ

_{x}

_{,y}and phases φ

_{x}

_{,y}= ±2πf

_{x}

_{,y}; setting modulus and central frequency of the poles as ρ

_{x}= 0.95, ρ

_{y}= 0.92, f

_{x}= 0.3, f

_{y}= 0.1, the parameters quantifying the dependence of x

_{n}and y

_{n}on their own past in (6) are a

_{1}= 2ρ

_{x}cosφ

_{x}= 0.5871, a

_{2}= −ρ

^{2}

_{x}= −0.9025, b

_{1}= 2ρ

_{y}cosφ

_{y}= 1.4886, a

_{2}= −ρ

^{2}

_{y}= −0.8464. The other parameters, all set with a magnitude c = 0.5, identify causal effects between pairs of processes; the imposed effects are mixed instantaneous and lagged from X to Y, exclusively instantaneous from Y to Z, and exclusively lagged from X to Z. With this setting, self-dependencies and causal effects are consistent with rhythms and interactions commonly observed in cardiovascular and cardiorespiratory variability, showing an autonomous oscillation at the frequency of the Maier waves (f

_{y}~ 0.1 Hz) for Y, which is transmitted to Z mimicking feedback effects from arterial pressure to heart period, and an oscillation at a typical respiratory frequency (f

_{x}~ 0.3 Hz) for X, which is transmitted to both Y and Z mimicking respiratory-related effects on arterial pressure and heart period (a realization of the three processes is shown in Figure 1a).

_{1}= {x

_{n}

_{-1},...,x

_{n}

_{-10}, z

_{n}

_{-1},...,z

_{n}

_{-10}} − selects progressively the past terms of X with lags 5, 2, and 3, terminating at the third step with the embedding vector V

_{3}= [x

_{n}

_{-5}, x

_{n}

_{-2}, x

_{n}

_{-3}]. The second repetition of the procedure, although starting with the enlarged set of candidates Ω

_{2}= {Ω

_{1},y

_{n}

_{-1},...,y

_{n}

_{-10},} which includes also past terms from the source system Y, selects exactly the same candidates leading again to the embedding vector V

_{3}= [x

_{n}

_{-5}, x

_{n}

_{-2}, x

_{n}

_{-3}] and yielding no reduction in the estimated CE minimum, so that we have TE

_{Y}

_{→X|Z}= cTE′

_{Y}

_{→X|Z}= 0.

**Figure 1.**Example of transfer entropy analysis performed for the first simulation.

**(a)**realization of the three processes generated according to (6).

**(b)**TE estimation between pairs of processes based on nonuniform embedding; each panel depicts the CE estimated for the destination process through application of the non-uniform embedding procedure without considering the source process (black circles), or considering the source process (red triangles), in the definition of the set of candidates; the terms selected at each step k of the sequential embedding are indicated within the plots, while filled symbols denote each detected CE minimum.

**(c)**Same of (b) for estimation of the compensated TE (cTE′).

_{n}

_{-1}, z

_{n}

_{-5}, x

_{n}

_{-3}], while the second embedding selects at the second and third steps some past terms from the source process Y (i.e., the terms y

_{n}

_{-5}and y

_{n}

_{-1}), so that the selected embedding vector changes to [x

_{n}

_{-1}, y

_{n}

_{-5}, y

_{n}

_{-1}] and this results in a reduction of the CE minimum with respect to the first embedding and in the detection of a nonzero information transfer (TE

_{Y→Z|X}> 0).

_{n}

_{-5}, y

_{n}

_{-4}, z

_{n}

_{-2}], failing to include any term from the source system X and thus returning TE

_{X}

_{→Y|Z}= 0 (Figure 1b, upper left panel); on the contrary the compensated TE captures the information transfer thanks to the fact that the zero-lag term x

_{n}is in the set of candidates for the second embedding, and is selected determining a reduction in the estimated CE that ultimately leads to cTE′

_{X}

_{→X|Z}> 0 [Figure 1c, upper left panel].

**Figure 2.**Results of transfer entropy analysis for the first simulation.

**(a)**Distribution over 100 realizations of (6) (expressed as 5th percentile, median and 95th percentile) of the information transfer estimated between each pair of processes using the traditional TE (white) and the compensated TE (black).

**(b)**Percentage of realizations for which the information transfer estimated using TE (white) and compensated TE (black) was detected as statistically significant according to the test based on time-shifted surrogates.

#### 3.2. Non-Physiological Instantaneous Causality

_{1}= 3.86 and R

_{2}= 4 to obtain a chaotic behavior for the two logistic maps describing the autonomous dynamics of X and Y; the parameters C and ε in (7) and (8) set respectively the strength of coupling from X to Y and the amount of instantaneous mixing between the two processes.

_{X}

_{→Y}and cTE′′

_{X}

_{→Y}the second repetition of the conditioning procedure (red) selects a term from the input process (i.e., x

_{n}

_{-1}) determining a decrease in the estimated CE minimum and thus the detection of a positive information transfer; on the contrary, the analysis performed from Y to X does not select any term from the source process in the second repetition of the conditioning procedure, thus leading to unvaried CE and hence to null values of the information transfer (TE

_{Y}

_{→X}= cTE′′

_{Y}

_{→X}= 0). The identical behavior of TE and cTE is explained by noting that, in this case with absence of instantaneous signal mixing, zero-lag effects are not present, and indeed the zero-lag term is not selected (although tested) during the embedding procedures for cTE. On the contrary, in the case of Figure 3b where the instantaneous mixing is not trivial, the two repetitions of the embedding procedure for cTE both select the zero lag-term (x

_{n}in the analysis from X to Y and y

_{n}in the analysis from Y to X); as a consequence, the cTE correctly reveals the absence of information transfer from X to Y and from Y to X, while the TE seems to indicate a false positive detection of information transfer over both directions because of the CE reduction determined by inclusion of a term from the input process during the second conditioning.

**Figure 3.**Example of transfer entropy analysis performed for the second simulation.

**(a)**Presence of coupling and absence of instantaneous mixing (C = 0.2, ε = 0)

**(b)**Absence of coupling and presence of instantaneous mixing (C = 0, ε = 0.2). Panels depict a realization of the two processes X and Y generated according to (7) and (8), together with the estimation of TE and cTE′′ over the two directions of interaction based on nonuniform embedding and conditional entropy (CE, see caption of Figure 1 for details).

_{X}

_{→Y}and TE

_{Y}

_{→X}are statistically significant with ε>0 even though X and Y are uncoupled over both the directions of interaction, and in Figure 4c where TE

_{Y}

_{→X}is statistically significant with ε = 0.2 even though no coupling was imposed from Y to X (in total, false positive detections using the TE were five out of six negative cases with presence of instantaneous mixing). Unlike the traditional TE, the cTE does not take false positive values in the presence of signal cross-talk, as the detected information transfer is not statistically significant over both directions in the case of uncoupled systems of Figure 4b, and is statistically significant from X to Y but not from Y to X in the case of unidirectionally coupled systems of Figure 4c. Thus, in this simulation where instantaneous causality is due to common driving effects, utilization of cTE′′ in place of the traditional TE measure yields a better specificity in the detection of predictive information transfer.

**Figure 4.**Results of transfer entropy analysis for the second simulation, showing the median values over 50 realizations of (7) and (8) of the TE (first panel row) and the compensated TE (second panel row) computed along the two directions of interactions (X→Y, circles; Y→X, triangles)

**(a)**at varying the parameter C with parameter ε = 0;

**(b)**at varying ε with C = 0 (b); and

**(c)**varying ε with C = 0.2. Filled symbols denote statistically significant values of TE or cTE′′ assessed by means of the permutation test.

## 4. Application Examples

#### 4.1. Cardiovascular and Cardiorespiratory Variability

_{n}, systolic pressure, y

_{n,}, and respiratory flow, x

_{n}, respectively as the sequences of the temporal distances between consecutive heartbeats detected from the electrocardiogram, the local maxima of the arterial pressure signal (acquired through the Finapres device) measured inside each detected heart period, and the values of the airflow signal (acquired from the nose through a differential pressure transducer) sampled at the onset of each detected heart period. The measurement convention is illustrated in Figure 5. The experimental protocol consisted in signal acquisition, after subject stabilization in the resting supine position, for 15 min with spontaneous breathing, followed by further 15 min with the subject inhaling and exhaling in time with a metronome acting at 15 cycles/min (paced breathing at 0.25 Hz). Two artifact-free windows of N = 300 samples, measured synchronously for the M = 3 series during spontaneous breathing and during paced breathing, were considered for the analysis. Weak stationarity of each series was checked by means of a test checking the stability of the mean and variance over the analysis window [38]. The analyzed series are shown in Figure 6.

**Figure 5.**Measurement of heart period (series z), systolic arterial pressure (series y) and respiratory flow (series x) variability series from the electrocardiogram, arterial blood pressure and nasal flow signals.

_{n}, precedes in time the occurrence of the present systolic pressure value, y

_{n}, which in turn precedes in time the end of the present heart period, z

_{n}(see Figure 5). Therefore, cTE analysis was performed for this application using the compensation proposed in (4). The statistical significance of each estimated TE and cTE′ was assessed using time shifted surrogates. The results of the analysis for the spontaneous breathing and paced breathing conditions are depicted in Figure 6a,b, respectively. Utilization of the traditional TE led to detect as statistically significant the information transfer measured from respiration to heart period during spontaneous breathing (TE

_{X}

_{→Z}in Figure 6a, and from respiration to systolic pressure during paced breathing (TE

_{X}

_{→Y}in Figure 6b. The same analysis performed accounting for instantaneous causality effects led to detect a higher number of statistically significant interactions, specifically from respiration to heart period and from systolic pressure to heart period during both conditions (cTE′

_{X}

_{→Z}and cTE′

_{Y}

_{→Z}in Figure 6a,b), and also from respiration to systolic pressure during paced breathing (cTE′

_{X}

_{→Y}in Figure 6b).

**Figure 6.**Transfer entropy analysis in cardiovascular and cardiorespiratory variability performed.

**(a)**during spontaneous breathing and

**(b)**during paced breathing. Plots depict the analyzed time series of respiratory flow (x

_{n}, system X), systolic arterial pressure (y

_{n}, system Y) and heart period (z

_{n}, system Z) together with the corresponding TE (circles) and compensated TE (triangles) estimated between each pair of series. The gray symbols indicate the values of TE/cTE obtained over 40 pairs of time-shifted surrogates; filled symbols denote statistically significant TE or cTE′.

_{Y}

_{→Z}but not for TE

_{Y}

_{→Z}in both conditions, could suggest a major role played by fast vagal effects −whereby the systolic pressure affects heart period within the same heartbeat—in the functioning of the baroreflex mechanism.

#### 4.2. Magnetoencephalography

**Figure 7.**Transfer entropy analysis in magnetoencephalography performed before (left) and during (right) presentation of the combined visuo-tactile stimuli.

**(a)**Representative MEG signals acquired from the somatosensory cortex (x

_{n}, system X) and the visual cortex (y

_{n}, system Y) for one of the experiment trials (n ranges from 1 to 293 samples before and during simulation).

**(b)**Median over the 60 trials of TE (circles) and compensated TE (triangles) estimated for the two directions of interaction between X and Y before and during stimulation; gray symbols indicate the values of TE/cTE′′ obtained over 100 trial permutations; filled symbols denote statistically significant TE or cTE′′.

## 5. Discussion

_{n}→y

_{n}, but by TE as well reflecting an indirect effect x

_{1:n-1}→x

_{n}→y

_{n}, (provided that X has an internal memory structure). Therefore, the higher sensitivity observed for the cTE in this case should be explained in practical terms (i.e., as an easier estimation of a direct than an indirect effect). Moreover, when instantaneous effects are causally meaningful, including them in TE computation as done in (4) might yield to a detection of information transfer not only over the direction of the actual causal effects, but also over the opposite direction. On the other hand, when instantaneous effects are not causally meaningful the full removal of zero-lag effects performed by (5) may be conservative when real causal effects taking place within the same sample are present besides the spurious effects to be removed. Another point regarding theoretical values of the index cTE′′ is that conditioning to the zero-lag term as done in (5) may cause, in particular circumstances involving unobserved variables (e.g., due to latent confounders or resulting from inappropriate sampling), spurious detections of predictive information transfer reflecting an effect known as “selection bias” or “conditioning on a collider” [45]. Nevertheless it is likely that, in most practical situations in which real short data sequences are considered and significance tests are applied, the null hypothesis of absence of information transfer cannot be rejected solely as a consequence of spurious effects deriving from selection bias. Further studies should be aimed at assessing the real capability of these spurious effects to produce detectable predictive information transfer in practical estimation contexts.

## References

- Schreiber, T. Measuring information transfer. Phys. Rev. Lett.
**2000**, 2000, 461–464. [Google Scholar] [CrossRef] - Barnett, L.; Barrett, A.B.; Seth, A.K. Granger causality and transfer entropy are equivalent for Gaussian variables. Phys. Rev. Lett.
**2009**, 103, 238701. [Google Scholar] [CrossRef] [PubMed] - Wibral, M.; Rahm, B.; Rieder, M.; Lindner, M.; Vicente, R.; Kaiser, J. Transfer entropy in magnetoencephalographic data: Quantifying information flow in cortical and cerebellar networks. Progr. Biophys. Mol. Biol.
**2011**, 105, 80–97. [Google Scholar] [CrossRef] [PubMed] - Vicente, R.; Wibral, M.; Lindner, M.; Pipa, G. Transfer entropy-a model-free measure of effective connectivity for the neurosciences. J. Comp. Neurosci.
**2011**, 30, 45–67. [Google Scholar] [CrossRef] [PubMed] - Vakorin, V.A.; Kovacevic, N.; McIntosh, A.R. Exploring transient transfer entropy based on a group-wise ICA decomposition of EEG data. Neuroimage
**2010**, 49, 1593–1600. [Google Scholar] [CrossRef] [PubMed] - Gourevitch, B.; Eggermont, J.J. Evaluating information transfer between auditory cortical neurons. J. Neurophysiol.
**2007**, 97, 2533–2543. [Google Scholar] [CrossRef] [PubMed] - Faes, L.; Nollo, G.; Porta, A. Information domain approach to the investigation of cardio-vascular, cardio-pulmonary, and vasculo-pulmonary causal couplings. Front. Physiol.
**2011**, 2, 1–13. [Google Scholar] [CrossRef] [PubMed] - Faes, L.; Nollo, G.; Porta, A. Non-uniform multivariate embedding to assess the information transfer in cardiovascular and cardiorespiratory variability series. Comput. Biol. Med.
**2012**, 42, 290–297. [Google Scholar] [CrossRef] [PubMed] - Vejmelka, M.; Palus, M. Inferring the directionality of coupling with conditional mutual information. Phys. Rev. E
**2008**, 77, 026214. [Google Scholar] [CrossRef] - Chicharro, D.; Ledberg, A. Framework to study dynamic dependencies in networks of interacting processes. Phys. Rev. E
**2012**, 86, 041901. [Google Scholar] [CrossRef] - Chicharro, D.; Ledberg, A. When two become one: The limits of causality analysis of brain dynamics. PLoS One
**2012**. [Google Scholar] [CrossRef] [PubMed] - Lizier, J.T.; Prokopenko, M. Differentiating information transfer and causal effect. Eur. Phys. J. B
**2010**, 73, 605–615. [Google Scholar] [CrossRef] - Hlavackova-Schindler, K.; Palus, M.; Vejmelka, M.; Bhattacharya, J. Causality detection based on information-theoretic approaches in time series analysis. Phys. Rep.
**2007**, 441, 1–46. [Google Scholar] [CrossRef] - Lee, J.; Nemati, S.; Silva, I.; Edwards, B.A.; Butler, J.P.; Malhotra, A. Transfer Entropy Estimation and Directional Coupling Change Detection in Biomedical Time Series. Biomed. Eng.
**2012**. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Faes, L.; Nollo, G.; Porta, A. Information-based detection of nonlinear Granger causality in multivariate processes via a nonuniform embedding technique. Phys. Rev. E
**2011**, 83, 051112. [Google Scholar] [CrossRef] - Lutkepohl, H. New Introduction to Multiple Time Series Analysis; Springer-Verlag: Heidelberg, Germany, 2005. [Google Scholar]
- Faes, L.; Erla, S.; Porta, A.; Nollo, G. A framework for assessing frequency domain causality in physiological time series with instantaneous effects. Philos. Transact. A
**2013**, in press. [Google Scholar] [CrossRef] [PubMed] - Faes, L.; Nollo, G. Extended causal modelling to assess Partial Directed Coherence in multiple time series with significant instantaneous interactions. Biol. Cybern.
**2010**, 103, 387–400. [Google Scholar] [CrossRef] [PubMed] - Granger, C.W.J. Investigating causal relations by econometric models and cross-spectral methods. Econometrica
**1969**, 37, 424–438. [Google Scholar] [CrossRef] - Geweke, J. Measurement of linear dependence and feedback between multiple time series. J. Am. Stat. Assoc.
**1982**, 77, 304–313. [Google Scholar] [CrossRef] - Guo, S.X.; Seth, A.K.; Kendrick, K.M.; Zhou, C.; Feng, J.F. Partial Granger causality—Eliminating exogenous inputs and latent variables. J. Neurosci. Methods
**2008**, 172, 79–93. [Google Scholar] [CrossRef] [PubMed] - Barrett, A.B.; Barnett, L.; Seth, A.K. Multivariate Granger causality and generalized variance. Phys. Rev. E
**2010**, 81, 041907. [Google Scholar] [CrossRef] - Hyvarinen, A.; Zhang, K.; Shimizu, S.; Hoyer, P.O. Estimation of a Structural Vector Autoregression Model Using Non-Gaussianity. J. Machine Learn. Res.
**2010**, 11, 1709–1731. [Google Scholar] - Porta, A.; Bassani, T.; Bari, V.; Pinna, G.D.; Maestri, R.; Guzzetti, S. Accounting for Respiration is Necessary to Reliably Infer Granger Causality From Cardiovascular Variability Series. IEEE Trans. Biomed. Eng.
**2012**, 59, 832–841. [Google Scholar] [CrossRef] [PubMed] - Vakorin, V.A.; Krakovska, O.A.; McIntosh, A.R. Confounding effects of indirect connections on causality estimation. J. Neurosci. Methods
**2009**, 184, 152–160. [Google Scholar] [CrossRef] [PubMed] - Chen, Y.; Bressler, S.L.; Ding, M. Frequency decomposition of conditional Granger causality and application to multivariate neural field potential data. J. Neurosci. Methods
**2006**, 150, 228–237. [Google Scholar] [CrossRef] [PubMed] - Kraskov, A.; Stogbauer, H.; Grassberger, P. Estimating mutual information. Phys. Rev. E
**2004**, 69, 066138. [Google Scholar] [CrossRef] - Porta, A.; Baselli, G.; Lombardi, F.; Montano, N.; Malliani, A.; Cerutti, S. Conditional entropy approach for the evaluation of the coupling strength. Biol. Cybern.
**1999**, 81, 119–129. [Google Scholar] [CrossRef] [PubMed] - Takens, F. Detecting strange attractors in fluid turbulence. In Dynamical Systems and Turbulence; Rand, D., Young, S.L., Eds.; Springer-Verlag: Berlin, Germany, 1981; pp. 336–381. [Google Scholar]
- Vlachos, I.; Kugiumtzis, D. Nonuniform state-space reconstruction and coupling detection. Phys. Rev. E
**2010**, 82, 016207. [Google Scholar] [CrossRef] - Small, M. Applied nonlinear time series analysis: Applications in physics, physiology and finance; World Scientific Publishing: Singapore, 2005. [Google Scholar]
- Runge, J.; Heitzig, J.; Petoukhov, V.; Kurths, J. Escaping the Curse of Dimensionality in Estimating Multivariate Transfer Entropy. Phys. Rev. Lett.
**2012**, 108, 258701. [Google Scholar] [CrossRef] [PubMed] - Pincus, S.M. Approximate Entropy As A Measure of System-Complexity. Proc. Nat. Acad. Sci. USA
**1991**, 88, 2297–2301. [Google Scholar] [CrossRef] [PubMed] - Porta, A.; Baselli, G.; Liberati, D.; Montano, N.; Cogliati, C.; Gnecchi-Ruscone, T.; Malliani, A.; Cerutti, S. Measuring regularity by means of a corrected conditional entropy in sympathetic outflow. Biol. Cybern.
**1998**, 78, 71–78. [Google Scholar] [CrossRef] [PubMed] - Bollen, K.A. Structural equations with latent variables; John Wiley & Sons: NY, USA, 1989. [Google Scholar]
- Yu, G.H.; Huang, C.C. A distribution free plotting position. Stoch. Env. Res. Risk Ass.
**2001**, 15, 462–476. [Google Scholar] [CrossRef] - Erla, S.; Faes, L.; Nollo, G.; Arfeller, C.; Braun, C.; Papadelis, C. Multivariate EEG spectral analysis elicits the functional link between motor and visual cortex during integrative sensorimotor tasks. Biomed. Signal Process. Contr.
**2011**, 7, 221–227. [Google Scholar] [CrossRef] - Magagnin, V.; Bassani, T.; Bari, V.; Turiel, M.; Maestri, R.; Pinna, G.D.; Porta, A. Non-stationarities significantly distort short-term spectral, symbolic and entropy heart rate variability indices. Physiol Meas.
**2011**, 32, 1775–1786. [Google Scholar] [CrossRef] [PubMed] - Cohen, M.A.; Taylor, J.A. Short-term cardiovascular oscillations in man: measuring and modelling the physiologies. J. Physiol
**2002**, 542, 669–683. [Google Scholar] [CrossRef] [PubMed] - Hirsch, J.A.; Bishop, B. Respiratory sinus arrhythmia in humans: how breathing pattern modulates heart rate. Am. J. Physiol.
**1981**, 241, H620–H629. [Google Scholar] [PubMed] - Toska, K.; Eriksen, M. Respiration-synchronous fluctuations in stroke volume, heart rate and arterial pressure in humans. J. Physiol
**1993**, 472, 501–512. [Google Scholar] [CrossRef] [PubMed] - Erla, S.; Papadelis, C.; Faes, L.; Braun, C.; Nollo, G. Studying brain visuo-tactile integration through cross-spectral analysis of human MEG recordings. In Medicon 2010, IFMBE Proceedings; Bamidis, P.D., Pallikarakis, N., Eds.; Springer: Berlin, Germany, 2010; pp. 73–76. [Google Scholar]
- Marzetti, L.; del Gratta, C.; Nolte, G. Understanding brain connectivity from EEG data by identifying systems composed of interacting sources. Neuroimage
**2008**, 42, 87–98. [Google Scholar] [CrossRef] [PubMed] - Bauer, M. Multisensory integration: A functional role for inter-area synchronization? Curr. Biol.
**2008**, 18, R709–R710. [Google Scholar] [CrossRef] [PubMed] - Cole, S.R.; Platt, R.W.; Schisterman, E.F.; Chu, H.; Westreich, D.; Richardson, D.; Poole, C. Illustrating bias due to conditioning on a collider. Int. J. Epidemiol.
**2010**, 36, 417–420. [Google Scholar] [CrossRef] [PubMed] - Marinazzo, D.; Pellicoro, M.; Stramaglia, S. Causal information approach to partial conditioning in multivariate data sets. Comput. Math. Methods Med.
**2012**, 2012, 303601. [Google Scholar] [CrossRef] [PubMed]

© 2013 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Faes, L.; Nollo, G.; Porta, A.
Compensated Transfer Entropy as a Tool for Reliably Estimating Information Transfer in Physiological Time Series. *Entropy* **2013**, *15*, 198-219.
https://doi.org/10.3390/e15010198

**AMA Style**

Faes L, Nollo G, Porta A.
Compensated Transfer Entropy as a Tool for Reliably Estimating Information Transfer in Physiological Time Series. *Entropy*. 2013; 15(1):198-219.
https://doi.org/10.3390/e15010198

**Chicago/Turabian Style**

Faes, Luca, Giandomenico Nollo, and Alberto Porta.
2013. "Compensated Transfer Entropy as a Tool for Reliably Estimating Information Transfer in Physiological Time Series" *Entropy* 15, no. 1: 198-219.
https://doi.org/10.3390/e15010198