Next Article in Journal
Information Theory and Dynamical System Predictability
Next Article in Special Issue
Quantum Kolmogorov Complexity and Information-Disturbance Theorem
Previous Article in Journal
Towards an Evolutionary Model of Animal-Associated Microbiomes
Article Menu

Export Article

Open AccessArticle
Entropy 2011, 13(3), 595-611; doi:10.3390/e13030595

Entropy Measures vs. Kolmogorov Complexity

Computer Science Department, Faculty of Sciences, University of Porto, Rua Campo Alegre 1021/1055, 4169-007 Porto, Portugal
Instituto de Telecomunicações, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal
Laboratório de Inteligência Artificial e Ciência de Computadores, Rua Campo Alegre 1021/1055, 4169-007 Porto, Portugal
Author to whom correspondence should be addressed.
Received: 14 January 2011 / Revised: 25 February 2011 / Accepted: 26 February 2011 / Published: 3 March 2011
(This article belongs to the Special Issue Kolmogorov Complexity)
View Full-Text   |   Download PDF [172 KB, uploaded 24 February 2015]


Kolmogorov complexity and Shannon entropy are conceptually different measures. However, for any recursive probability distribution, the expected value of Kolmogorov complexity equals its Shannon entropy, up to a constant. We study if a similar relationship holds for R´enyi and Tsallis entropies of order α, showing that it only holds for α = 1. Regarding a time-bounded analogue relationship, we show that, for some distributions we have a similar result. We prove that, for universal time-bounded distribution mt(x), Tsallis and Rényi entropies converge if and only if α is greater than 1. We also establish the uniform continuity of these entropies.
Keywords: Kolmogorov complexity; Shannon entropy; Rényi entropy; Tsallis entropy Kolmogorov complexity; Shannon entropy; Rényi entropy; Tsallis entropy
This is an open access article distributed under the Creative Commons Attribution License (CC BY 3.0).

Scifeed alert for new publications

Never miss any articles matching your research from any publisher
  • Get alerts for new papers matching your research
  • Find out the new papers from selected authors
  • Updated daily for 49'000+ journals and 6000+ publishers
  • Define your Scifeed now

SciFeed Share & Cite This Article

MDPI and ACS Style

Teixeira, A.; Matos, A.; Souto, A.; Antunes, L. Entropy Measures vs. Kolmogorov Complexity. Entropy 2011, 13, 595-611.

Show more citation formats Show less citations formats

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top