Entropy 2011, 13(3), 595-611; doi:10.3390/e13030595

Entropy Measures vs. Kolmogorov Complexity

Received: 14 January 2011; in revised form: 25 February 2011 / Accepted: 26 February 2011 / Published: 3 March 2011
(This article belongs to the Special Issue Kolmogorov Complexity)
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract: Kolmogorov complexity and Shannon entropy are conceptually different measures. However, for any recursive probability distribution, the expected value of Kolmogorov complexity equals its Shannon entropy, up to a constant. We study if a similar relationship holds for R´enyi and Tsallis entropies of order α, showing that it only holds for α = 1. Regarding a time-bounded analogue relationship, we show that, for some distributions we have a similar result. We prove that, for universal time-bounded distribution mt(x), Tsallis and Rényi entropies converge if and only if α is greater than 1. We also establish the uniform continuity of these entropies.
Keywords: Kolmogorov complexity; Shannon entropy; Rényi entropy; Tsallis entropy
PDF Full-text Download PDF Full-Text [172 KB, uploaded 3 March 2011 10:28 CET]

Export to BibTeX |

MDPI and ACS Style

Teixeira, A.; Matos, A.; Souto, A.; Antunes, L. Entropy Measures vs. Kolmogorov Complexity. Entropy 2011, 13, 595-611.

AMA Style

Teixeira A, Matos A, Souto A, Antunes L. Entropy Measures vs. Kolmogorov Complexity. Entropy. 2011; 13(3):595-611.

Chicago/Turabian Style

Teixeira, Andreia; Matos, Armando; Souto, André; Antunes, Luís. 2011. "Entropy Measures vs. Kolmogorov Complexity." Entropy 13, no. 3: 595-611.

Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert