• Abstract
• Full-Text PDF [172 KB]

• Article Statistics
• Google Scholar
• Order Reprints

• Cite this article
• Export to BibTeX
• Export to EndNote

• [+] on DOAJ
• Teixeira, A
• Matos, A
• Souto, A
• Antunes, L
• [+] on Google Scholar
• Teixeira, A
• Matos, A
• Souto, A
• Antunes, L
• [+] on PubMed
• Teixeira, A
• Matos, A
• Souto, A
• Antunes, L

•  CiteULike
•  Facebook
•  Mendeley
•  Twitter

This article is

• freely available
• re-usable

Entropy 2011, 13(3), 595-611; doi:10.3390/e13030595

Article

Entropy Measures vs. Kolmogorov Complexity

Andreia Teixeira ^1,2,* , Armando Matos ^1,3 , André Souto ^1,2  and Luís Antunes ^1,2

¹ Computer Science Department, Faculty of Sciences, University of Porto, Rua Campo Alegre 1021/1055, 4169-007 Porto, Portugal ² Instituto de Telecomunicações, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal ³ Laboratório de Inteligência Artificial e Ciência de Computadores, Rua Campo Alegre 1021/1055, 4169-007 Porto, Portugal

* Author to whom correspondence should be addressed.

Received: 14 January 2011; in revised form: 25 February 2011 / Accepted: 26 February 2011 / Published: 3 March 2011

(This article belongs to the Special Issue Kolmogorov Complexity)

Download PDF Full-Text [172 KB, uploaded 3 March 2011 10:28 CET]

Abstract: Kolmogorov complexity and Shannon entropy are conceptually different measures. However, for any recursive probability distribution, the expected value of Kolmogorov complexity equals its Shannon entropy, up to a constant. We study if a similar relationship holds for R´enyi and Tsallis entropies of order α, showing that it only holds for α = 1. Regarding a time-bounded analogue relationship, we show that, for some distributions we have a similar result. We prove that, for universal time-bounded distribution m^t(x), Tsallis and Rényi entropies converge if and only if α is greater than 1. We also establish the uniform continuity of these entropies.

Keywords: Kolmogorov complexity; Shannon entropy; Rényi entropy; Tsallis entropy

Article Statistics

Cite This Article