Next Article in Journal
Learning Coefficient of Vandermonde Matrix-Type Singularities in Model Selection
Next Article in Special Issue
Time–Energy and Time–Entropy Uncertainty Relations in Nonequilibrium Quantum Thermodynamics under Steepest-Entropy-Ascent Nonlinear Master Equations
Previous Article in Journal
Parameter Estimation with Data-Driven Nonparametric Likelihood Functions
Previous Article in Special Issue
Where Was Past Low-Entropy?
Open AccessFeature PaperArticle

The Thermodynamics of Network Coding, and an Algorithmic Refinement of the Principle of Maximum Entropy

by Hector Zenil 1,2,3,4,5,6,*, Narsis A. Kiani 1,2,3 and Jesper Tegnér 2,5,6
1
Algorithmic Dynamics Lab, Karolinska Institute, 17177 Stockholm, Sweden
2
Unit of Computational Medicine, Center for Molecular Medicine, Department of Medicine Solna, Karolinska Institute, 17177 Stockholm, Sweden
3
Algorithmic Nature Group, Laboratory of Scientific Research (LABORES) for the Natural and Digital Sciences, 75006 Paris, France
4
Oxford Immune Algorithmics, Oxford University Innovation, Reading RG1 7TT, UK
5
Biological and Environmental Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
6
Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology (KAUST), Thuwal 23955, Saudi Arabia
*
Author to whom correspondence should be addressed.
An animated video explaining some of the methods is available at: https://www.youtube.com/watch?v=BEaXyDS_1Cw.
Entropy 2019, 21(6), 560; https://doi.org/10.3390/e21060560
Received: 9 April 2019 / Revised: 17 May 2019 / Accepted: 20 May 2019 / Published: 3 June 2019
(This article belongs to the Special Issue Entropy Production and Its Applications: From Cosmology to Biology)
The principle of maximum entropy (Maxent) is often used to obtain prior probability distributions as a method to obtain a Gibbs measure under some restriction giving the probability that a system will be in a certain state compared to the rest of the elements in the distribution. Because classical entropy-based Maxent collapses cases confounding all distinct degrees of randomness and pseudo-randomness, here we take into consideration the generative mechanism of the systems considered in the ensemble to separate objects that may comply with the principle under some restriction and whose entropy is maximal but may be generated recursively from those that are actually algorithmically random offering a refinement to classical Maxent. We take advantage of a causal algorithmic calculus to derive a thermodynamic-like result based on how difficult it is to reprogram a computer code. Using the distinction between computable and algorithmic randomness, we quantify the cost in information loss associated with reprogramming. To illustrate this, we apply the algorithmic refinement to Maxent on graphs and introduce a Maximal Algorithmic Randomness Preferential Attachment (MARPA) Algorithm, a generalisation over previous approaches. We discuss practical implications of evaluation of network randomness. Our analysis provides insight in that the reprogrammability asymmetry appears to originate from a non-monotonic relationship to algorithmic probability. Our analysis motivates further analysis of the origin and consequences of the aforementioned asymmetries, reprogrammability, and computation. View Full-Text
Keywords: second law of thermodynamics; reprogrammability; algorithmic complexity; generative mechanisms; deterministic systems; algorithmic randomness; principle of maximum entropy; Maxent second law of thermodynamics; reprogrammability; algorithmic complexity; generative mechanisms; deterministic systems; algorithmic randomness; principle of maximum entropy; Maxent
Show Figures

Figure 1

MDPI and ACS Style

Zenil, H.; Kiani, N.A.; Tegnér, J. The Thermodynamics of Network Coding, and an Algorithmic Refinement of the Principle of Maximum Entropy. Entropy 2019, 21, 560.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop