Open AccessThis article is
- freely available
On Thermodynamic Interpretation of Transfer Entropy
CSIRO Information and Communications Technologies Centre, PO Box 76, Epping, NSW 1710, Australia
School of Physics A28, University of Sydney, NSW 2006, Australia
CSIRO Materials Science and Engineering, Bradfield Road, West Lindfield, NSW 2070, Australia
* Author to whom correspondence should be addressed.
Received: 16 November 2012; in revised form: 16 January 2013 / Accepted: 28 January 2013 / Published: 1 February 2013
Abstract: We propose a thermodynamic interpretation of transfer entropy near equilibrium, using a specialised Boltzmann’s principle. The approach relates conditional probabilities to the probabilities of the corresponding state transitions. This in turn characterises transfer entropy as a difference of two entropy rates: the rate for a resultant transition and another rate for a possibly irreversible transition within the system affected by an additional source. We then show that this difference, the local transfer entropy, is proportional to the external entropy production, possibly due to irreversibility. Near equilibrium, transfer entropy is also interpreted as the difference in equilibrium stabilities with respect to two scenarios: a default case and the case with an additional source. Finally, we demonstrated that such a thermodynamic treatment is not applicable to information flow, a measure of causal effect.
Keywords: transfer entropy; information transfer; entropy production; irreversibility; Kullback–Leibler divergence; thermodynamic equilibrium; Boltzmann’s principle; causal effect
Citations to this Article
Cite This Article
MDPI and ACS Style
Prokopenko, M.; Lizier, J.T.; Price, D.C. On Thermodynamic Interpretation of Transfer Entropy. Entropy 2013, 15, 524-543.
Prokopenko M, Lizier JT, Price DC. On Thermodynamic Interpretation of Transfer Entropy. Entropy. 2013; 15(2):524-543.
Prokopenko, Mikhail; Lizier, Joseph T.; Price, Don C. 2013. "On Thermodynamic Interpretation of Transfer Entropy." Entropy 15, no. 2: 524-543.