Transfer Entropy

Statistical relationships among the variables of a complex system reveal a lot about its physical behavior[...].


Methods and Theory
The first section begins with the presentation of a recipe to estimate the information flow in dynamical systems [1]. In their work, Gencaga et al. propose a Bayesian approach to estimate TE and apply a set of methods together as an accuracy cross-check to provide a reliable mathematical tool for any given dataset. The work of Zhu et al. [2] proposes a k-Nearest Neighbor approach to estimate TE and demonstrates its effectiveness as an extension of the KSG MI estimator.
The methodological section continues with the analytical derivations of the TE expressions for a class of non-Gaussian distributions. Here, Jafari-Mamaghani and Tyrcha [3] provide the expressions of TE in the cases of multivariate exponential, logistic, Pareto (Type I-IV), and Burr distributions. Next, Nichols et al. elaborate on the linearized TE for continuous and coupled second-order systems and they derive an analytical expression for time-delayed transfer entropy (TDTE) [4]. They conclude with an alternative interpretation of TE, which can be viewed as a measure of the ability of a given system component to predict the dynamics of another. Coupling between random processes is also explored by Hahs and Pethel [5], where the TE is computed over multiple time lags for multivariate Gaussian autoregressive processes. In two examples, they demonstrate the change in TE as a response of variations in the correlation and coupling coefficient parameters. The case of coupling dynamics with time-varying dependencies is investigated by Gómez-Herrero et al. [6] if access to an ensemble of independent repetitions of time series is available. They estimate combinations of entropies and detect time-varying information flow between dynamical systems using the ensemble members.
The relation between Granger causality and directed information theory is discussed next in the review paper of Amblard and Michel [7], in which they focus on conditional independence and causal influences between stochastic processes. In addition to the link between directed information and hypothesis testing, instantaneous dependencies are emphasized to be different than dependencies on past values.
The next two papers demonstrate two new interpretations of TE. First, motivated by the relativistic effects on the observation of information dynamics, Lizier and Mahoney bring a new explanation of a local framework for information dynamics [8]. Second, Prokopenko et al. present a thermodynamic interpretation of TE near equilibrium and emphasize the nuance between TE and causality [9]. The methodological section ends with the comparisons of Papana et al. where they study direct causality measures in multivariate time series by simulations. The authors compare measures such as the conditional Granger causality index, partial Granger causality index, partial directed coherence, partial TE, partial symbolic TE, and partial MI on mixed embedding. Simulations include stochastic and chaotic dynamical systems with different embedding dimensions and time series lengths [10].

Applications
In this section, we present six contributions on the applications of TE. In the first paper, Faes et al. introduce a tool for reliably estimating information transfer in physiological time series using compensated TE [11]. This tool provides a set of solutions to the problems arising from the high dimensionality and small sample size, which are frequently encountered in entropy estimations of cardiovascular and neurological time series. Next, Materassi et al. elucidate a different normalized TE and use it to detect the verse of energy flux transfer in a synthetic model of fluid turbulence, namely the Gledzer-Ohkitana-Yamada shell model [12]. They emphasize the superior performance compared to those of the traditional methods. Applications continue with a paper on network inference by Ai [13]. The author addresses a TE-based framework to quantify the relationships among topological measures and provides a general approach to infer a drive-response structure in a complex network. This work is followed by two financial applications. The first contribution is authored by Li et al., in which a TE-based method is developed to determine the interbank exposure matrix between banks and the stability of the Chinese banking system is evaluated by simulating the risk contagion process [14]. In the second application, Sandoval Jr. uses the stocks of the 197 largest companies in the world and explores their relationships using TE [15]. This Special Issue ends with the presentation of the theory and applications of the Liang-Kleeman information flow [16]. Here, Liang points out the importance of information flow as a potential measure of the cause and effect relation between dynamical events and presents applications on the Baker transformation, Henon map, truncated Burgers-Hopf system, and Langevin equation.
This Special Issue demonstrates the importance of information-theoretic quantities in the analysis of the statistical dependencies between the variables of a complex system. Unlike correlation and MI, TE is shown to be effective for the detection of directional interactions, which are closely related to cause and effect relationships. The examples demonstrate the difficulties in estimating information-theoretic quantities from data and present approaches to overcome these problems.
In this Special Issue, we have collected 16 outstanding papers by the experts in the field. I would like to express our special thanks to each researcher and anonymous referee for their invaluable contributions. I would also like to thank the Editor-in-Chief, Prof. Kevin H. Knuth, for his encouragement during the organization of this Special Issue. My grateful thanks are also extended to the members of the editorial board and the editorial assistants of the Entropy Journal for their support. Last, but not least, I would like to thank MDPI Books for giving me the opportunity to publish this Special Issue.
We are excited to present this Special Issue as a reference for the theory and applications of transfer entropy and we hope that this publication contributes to novelties in all disciplines of research and development.