Special Issue "Information Transfer, Entropy Production, Irreversibility and Time Series Analysis"

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Statistical Physics".

Deadline for manuscript submissions: closed (31 January 2021) | Viewed by 15901

Special Issue Editor

Dr. Milan Paluš
E-Mail Website
Guest Editor
Institute of Computer Science, Czech Academy of Sciences, Pod vodarenskou vezi 2, 182 07 Praha 8, Czech Republic
Interests: complex systems; nonlinear dynamics; theory of deterministic chaos; synchronization; causality

Special Issue Information

Dear Colleagues,

Time series record the time evolution of systems or processes in nature or society and are sources of important information about system states, transitions between them, interactions among system components, or about physical mechanisms underlying observed phenomena. Time series analysis is an established scientific discipline, traditionally rooted in the theory of stochastic processes. There are also new avenues in time series analysis inspired by nonlinear dynamical systems, the theory of deterministic chaos and statistical physics. Different approaches, however, are becoming unified in using notions of information, entropy, entropy rates or production and information transfer. Ideas and tools from information theory have become an important part of time series analysis and related interdisciplinary research in an increasing number of scientific disciplines.

The focus of this Special Issue is the theoretical development as well as interesting applications of methods for estimating information transfer, causality, entropy production and irreversibility from time series recorded in complex systems. We anticipate theoretical developments accompanied with explanatory examples using either simulated or experimental data and interdisciplinary applications that uncover new phenomena or shed new light on known events in different scientific fields. Especially welcome are contributions studying interactions of the topic issues, e.g., causality and information transfer, causality and irreversibility, irreversibility and entropy production.

Dr. Milan Paluš
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1800 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • time series
  • complex systems
  • entropy rate
  • entropy production
  • irreversibility
  • information transfer
  • causality
  • nonlinear dynamics
  • multiscale processes

Published Papers (11 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Article
Causality in Reversed Time Series: Reversed or Conserved?
Entropy 2021, 23(8), 1067; https://doi.org/10.3390/e23081067 - 17 Aug 2021
Viewed by 717
Abstract
The inference of causal relations between observable phenomena is paramount across scientific disciplines; however, the means for such enterprise without experimental manipulation are limited. A commonly applied principle is that of the cause preceding and predicting the effect, taking into account other circumstances. [...] Read more.
The inference of causal relations between observable phenomena is paramount across scientific disciplines; however, the means for such enterprise without experimental manipulation are limited. A commonly applied principle is that of the cause preceding and predicting the effect, taking into account other circumstances. Intuitively, when the temporal order of events is reverted, one would expect the cause and effect to apparently switch roles. This was previously demonstrated in bivariate linear systems and used in design of improved causal inference scores, while such behaviour in linear systems has been put in contrast with nonlinear chaotic systems where the inferred causal direction appears unchanged under time reversal. The presented work explores the conditions under which the causal reversal happens—either perfectly, approximately, or not at all—using theoretical analysis, low-dimensional examples, and network simulations, focusing on the simplified yet illustrative linear vector autoregressive process of order one. We start with a theoretical analysis that demonstrates that a perfect coupling reversal under time reversal occurs only under very specific conditions, followed up by constructing low-dimensional examples where indeed the dominant causal direction is even conserved rather than reversed. Finally, simulations of random as well as realistically motivated network coupling patterns from brain and climate show that level of coupling reversal and conservation can be well predicted by asymmetry and anormality indices introduced based on the theoretical analysis of the problem. The consequences for causal inference are discussed. Full article
Show Figures

Figure 1

Article
Cross-Scale Causality and Information Transfer in Simulated Epileptic Seizures
Entropy 2021, 23(5), 526; https://doi.org/10.3390/e23050526 - 25 Apr 2021
Viewed by 693
Abstract
An information-theoretic approach for detecting causality and information transfer was applied to phases and amplitudes of oscillatory components related to different time scales and obtained using the wavelet transform from a time series generated by the Epileptor model. Three main time scales and [...] Read more.
An information-theoretic approach for detecting causality and information transfer was applied to phases and amplitudes of oscillatory components related to different time scales and obtained using the wavelet transform from a time series generated by the Epileptor model. Three main time scales and their causal interactions were identified in the simulated epileptic seizures, in agreement with the interactions of the model variables. An approach consisting of wavelet transform, conditional mutual information estimation, and surrogate data testing applied to a single time series generated by the model was demonstrated to be successful in the identification of all directional (causal) interactions between the three different time scales described in the model. Thus, the methodology was prepared for the identification of causal cross-frequency phase–phase and phase–amplitude interactions in experimental and clinical neural data. Full article
Show Figures

Figure 1

Article
Granger Causality on forward and Reversed Time Series
Entropy 2021, 23(4), 409; https://doi.org/10.3390/e23040409 - 30 Mar 2021
Cited by 4 | Viewed by 839
Abstract
In this study, the information flow time arrow is investigated for stochastic data defined by vector autoregressive models. The time series are analyzed forward and backward by different Granger causality detection methods. Besides the normal distribution, which is usually required for the validity [...] Read more.
In this study, the information flow time arrow is investigated for stochastic data defined by vector autoregressive models. The time series are analyzed forward and backward by different Granger causality detection methods. Besides the normal distribution, which is usually required for the validity of Granger causality analysis, several other distributions of predictive errors are considered. A clear effect of a change in the order of cause and effect on the time-reversed series of unidirectionally connected variables was detected with standard Granger causality test (GC), when the product of the connection strength and the ratio of the predictive errors of the driver and the recipient was below a certain level, otherwise bidirectional causal connection was detected. On the other hand, opposite causal link was detected unconditionally by the methods based on the time reversal testing, but they were not able to detect correct bidirectional connection. The usefulness of the backward analysis is manifested in cases where falsely detected unidirectional connections can be rejected by applying the result obtained after the time reversal, and in cases of uncorrelated causally independent variables, where the absence of a causal link detected by GC on the original series should be confirmed on the time-reversed series. Full article
Show Figures

Figure 1

Article
Causality and Information Transfer Between the Solar Wind and the Magnetosphere–Ionosphere System
Entropy 2021, 23(4), 390; https://doi.org/10.3390/e23040390 - 25 Mar 2021
Cited by 6 | Viewed by 1036
Abstract
An information-theoretic approach for detecting causality and information transfer is used to identify interactions of solar activity and interplanetary medium conditions with the Earth’s magnetosphere–ionosphere systems. A causal information transfer from the solar wind parameters to geomagnetic indices is detected. The vertical component [...] Read more.
An information-theoretic approach for detecting causality and information transfer is used to identify interactions of solar activity and interplanetary medium conditions with the Earth’s magnetosphere–ionosphere systems. A causal information transfer from the solar wind parameters to geomagnetic indices is detected. The vertical component of the interplanetary magnetic field (Bz) influences the auroral electrojet (AE) index with an information transfer delay of 10 min and the geomagnetic disturbances at mid-latitudes measured by the symmetric field in the H component (SYM-H) index with a delay of about 30 min. Using a properly conditioned causality measure, no causal link between AE and SYM-H, or between magnetospheric substorms and magnetic storms can be detected. The observed causal relations can be described as linear time-delayed information transfer. Full article
Show Figures

Figure 1

Article
Time-Reversibility, Causality and Compression-Complexity
Entropy 2021, 23(3), 327; https://doi.org/10.3390/e23030327 - 10 Mar 2021
Cited by 3 | Viewed by 933
Abstract
Detection of the temporal reversibility of a given process is an interesting time series analysis scheme that enables the useful characterisation of processes and offers an insight into the underlying processes generating the time series. Reversibility detection measures have been widely employed in [...] Read more.
Detection of the temporal reversibility of a given process is an interesting time series analysis scheme that enables the useful characterisation of processes and offers an insight into the underlying processes generating the time series. Reversibility detection measures have been widely employed in the study of ecological, epidemiological and physiological time series. Further, the time reversal of given data provides a promising tool for analysis of causality measures as well as studying the causal properties of processes. In this work, the recently proposed Compression-Complexity Causality (CCC) measure (by the authors) is shown to be free of the assumption that the "cause precedes the effect", making it a promising tool for causal analysis of reversible processes. CCC is a data-driven interventional measure of causality (second rung on the Ladder of Causation) that is based on Effort-to-Compress (ETC), a well-established robust method to characterize the complexity of time series for analysis and classification. For the detection of the temporal reversibility of processes, we propose a novel measure called the Compressive Potential based Asymmetry Measure. This asymmetry measure compares the probability of the occurrence of patterns at different scales between the forward-time and time-reversed process using ETC. We test the performance of the measure on a number of simulated processes and demonstrate its effectiveness in determining the asymmetry of real-world time series of sunspot numbers, digits of the transcedental number π and heart interbeat interval variability. Full article
Show Figures

Figure 1

Article
Coupling between Blood Pressure and Subarachnoid Space Width Oscillations during Slow Breathing
Entropy 2021, 23(1), 113; https://doi.org/10.3390/e23010113 - 15 Jan 2021
Cited by 2 | Viewed by 1147
Abstract
The precise mechanisms connecting the cardiovascular system and the cerebrospinal fluid (CSF) are not well understood in detail. This paper investigates the couplings between the cardiac and respiratory components, as extracted from blood pressure (BP) signals and oscillations of the subarachnoid space width [...] Read more.
The precise mechanisms connecting the cardiovascular system and the cerebrospinal fluid (CSF) are not well understood in detail. This paper investigates the couplings between the cardiac and respiratory components, as extracted from blood pressure (BP) signals and oscillations of the subarachnoid space width (SAS), collected during slow ventilation and ventilation against inspiration resistance. The experiment was performed on a group of 20 healthy volunteers (12 females and 8 males; BMI =22.1±3.2 kg/m2; age 25.3±7.9 years). We analysed the recorded signals with a wavelet transform. For the first time, a method based on dynamical Bayesian inference was used to detect the effective phase connectivity and the underlying coupling functions between the SAS and BP signals. There are several new findings. Slow breathing with or without resistance increases the strength of the coupling between the respiratory and cardiac components of both measured signals. We also observed increases in the strength of the coupling between the respiratory component of the BP and the cardiac component of the SAS and vice versa. Slow breathing synchronises the SAS oscillations, between the brain hemispheres. It also diminishes the similarity of the coupling between all analysed pairs of oscillators, while inspiratory resistance partially reverses this phenomenon. BP–SAS and SAS–BP interactions may reflect changes in the overall biomechanical characteristics of the brain. Full article
Show Figures

Figure 1

Article
Measuring Information Coupling between the Solar Wind and the Magnetosphere–Ionosphere System
Entropy 2020, 22(3), 276; https://doi.org/10.3390/e22030276 - 28 Feb 2020
Cited by 11 | Viewed by 1996
Abstract
The interaction between the solar wind and the Earth’s magnetosphere–ionosphere system is very complex, being essentially the result of the interplay between an external driver, the solar wind, and internal processes to the magnetosphere–ionosphere system. In this framework, modelling the Earth’s magnetosphere–ionosphere response [...] Read more.
The interaction between the solar wind and the Earth’s magnetosphere–ionosphere system is very complex, being essentially the result of the interplay between an external driver, the solar wind, and internal processes to the magnetosphere–ionosphere system. In this framework, modelling the Earth’s magnetosphere–ionosphere response to the changes of the solar wind conditions requires a correct identification of the causality relations between the different parameters/quantities used to monitor this coupling. Nowadays, in the framework of complex dynamical systems, both linear statistical tools and Granger causality models drastically fail to detect causal relationships between time series. Conversely, information theory-based concepts can provide powerful model-free statistical quantities capable of disentangling the complex nature of the causal relationships. In this work, we discuss how to deal with the problem of measuring causal information in the solar wind–magnetosphere–ionosphere system. We show that a time delay of about 30–60 min is found between solar wind and magnetospheric and ionospheric overall dynamics as monitored by geomagnetic indices, with a great information transfer observed between the z component of the interplanetary magnetic field and geomagnetic indices, while a lower transfer is found when other solar wind parameters are considered. This suggests that the best candidate for modelling the geomagnetic response to solar wind changes is the interplanetary magnetic field component B z . A discussion of the relevance of our results in the framework of Space Weather is also provided. Full article
Show Figures

Figure 1

Article
Detecting Causality in Multivariate Time Series via Non-Uniform Embedding
Entropy 2019, 21(12), 1233; https://doi.org/10.3390/e21121233 - 16 Dec 2019
Cited by 16 | Viewed by 1854
Abstract
Causal analysis based on non-uniform embedding schemes is an important way to detect the underlying interactions between dynamic systems. However, there are still some obstacles to estimating high-dimensional conditional mutual information and forming optimal mixed embedding vector in traditional non-uniform embedding schemes. In [...] Read more.
Causal analysis based on non-uniform embedding schemes is an important way to detect the underlying interactions between dynamic systems. However, there are still some obstacles to estimating high-dimensional conditional mutual information and forming optimal mixed embedding vector in traditional non-uniform embedding schemes. In this study, we present a new non-uniform embedding method framed in information theory to detect causality for multivariate time series, named LM-PMIME, which integrates the low-dimensional approximation of conditional mutual information and the mixed search strategy for the construction of the mixed embedding vector. We apply the proposed method to simulations of linear stochastic, nonlinear stochastic, and chaotic systems, demonstrating its superiority over partial conditional mutual information from mixed embedding (PMIME) method. Moreover, the proposed method works well for multivariate time series with weak coupling strengths, especially for chaotic systems. In the actual application, we show its applicability to epilepsy multichannel electrocorticographic recordings. Full article
Show Figures

Figure 1

Article
Residual Predictive Information Flow in the Tight Coupling Limit: Analytic Insights from a Minimalistic Model
Entropy 2019, 21(10), 1010; https://doi.org/10.3390/e21101010 - 17 Oct 2019
Cited by 1 | Viewed by 1929
Abstract
In a coupled system, predictive information flows from the causing to the caused variable. The amount of transferred predictive information can be quantified through the use of transfer entropy or, for Gaussian variables, equivalently via Granger causality. It is natural to expect and [...] Read more.
In a coupled system, predictive information flows from the causing to the caused variable. The amount of transferred predictive information can be quantified through the use of transfer entropy or, for Gaussian variables, equivalently via Granger causality. It is natural to expect and has been repeatedly observed that a tight coupling does not permit to reconstruct a causal connection between causing and caused variables. Here, we show that for a model of interacting social groups, carried from the master equation to the Fokker–Planck level, a residual predictive information flow can remain for a pair of uni-directionally coupled variables even in the limit of infinite coupling strength. We trace this phenomenon back to the question of how the synchronizing force and the noise strength scale with the coupling strength. A simplified model description allows us to derive analytic expressions that fully elucidate the interplay between deterministic and stochastic model parts. Full article
Show Figures

Figure 1

Article
Correlation Dimension Detects Causal Links in Coupled Dynamical Systems
Entropy 2019, 21(9), 818; https://doi.org/10.3390/e21090818 - 21 Aug 2019
Cited by 4 | Viewed by 1794
Abstract
It is becoming increasingly clear that causal analysis of dynamical systems requires different approaches than, for example, causal analysis of interconnected autoregressive processes. In this study, a correlation dimension estimated in reconstructed state spaces is used to detect causality. If deterministic dynamics plays [...] Read more.
It is becoming increasingly clear that causal analysis of dynamical systems requires different approaches than, for example, causal analysis of interconnected autoregressive processes. In this study, a correlation dimension estimated in reconstructed state spaces is used to detect causality. If deterministic dynamics plays a dominant role in data then the method based on the correlation dimension can serve as a fast and reliable way to reveal causal relationships between and within the systems. This study demonstrates that the method, unlike most other causal approaches, detects causality well, even for very weak links. It can also identify cases of uncoupled systems that are causally affected by a hidden common driver. Full article
Show Figures

Figure 1

Article
Information Thermodynamics for Time Series of Signal-Response Models
Entropy 2019, 21(2), 177; https://doi.org/10.3390/e21020177 - 14 Feb 2019
Cited by 6 | Viewed by 2132
Abstract
The entropy production in stochastic dynamical systems is linked to the structure of their causal representation in terms of Bayesian networks. Such a connection was formalized for bipartite (or multipartite) systems with an integral fluctuation theorem in [Phys. Rev. Lett. 111, 180603 (2013)]. [...] Read more.
The entropy production in stochastic dynamical systems is linked to the structure of their causal representation in terms of Bayesian networks. Such a connection was formalized for bipartite (or multipartite) systems with an integral fluctuation theorem in [Phys. Rev. Lett. 111, 180603 (2013)]. Here we introduce the information thermodynamics for time series, that are non-bipartite in general, and we show that the link between irreversibility and information can only result from an incomplete causal representation. In particular, we consider a backward transfer entropy lower bound to the conditional time series irreversibility that is induced by the absence of feedback in signal-response models. We study such a relation in a linear signal-response model providing analytical solutions, and in a nonlinear biological model of receptor-ligand systems where the time series irreversibility measures the signaling efficiency. Full article
Show Figures

Figure 1

Back to TopTop