Next Article in Journal
Thermodynamics of Superdiffusion Generated by Lévy–Wiener Fluctuating Forces
Previous Article in Journal
“The Heisenberg Method”: Geometry, Algebra, and Probability in Quantum Theory
Article Menu
Issue 9 (September) cover image

Export Article

Open AccessArticle
Entropy 2018, 20(9), 657; https://doi.org/10.3390/e20090657

Low Complexity Estimation Method of Rényi Entropy for Ergodic Sources

Department of Information and Communication Engineering, Chosun University, 309 Pilmoondae-ro Dong-gu, Gwangju 61452, Korea
This paper is an extended version of my paper published in the 2014 International Symposium on Information Theory and Its Applications, Melbourne, VIC, Australia, 26–29 October 2014.
Received: 11 July 2018 / Revised: 28 August 2018 / Accepted: 30 August 2018 / Published: 31 August 2018
(This article belongs to the Section Information Theory)
Full-Text   |   PDF [1184 KB, uploaded 3 September 2018]   |  

Abstract

Since the entropy is a popular randomness measure, there are many studies for the estimation of entropies for given random samples. In this paper, we propose an estimation method of the Rényi entropy of order α . Since the Rényi entropy of order α is a generalized entropy measure including the Shannon entropy as a special case, the proposed estimation method for Rényi entropy can detect any significant deviation of an ergodic stationary random source’s output. It is shown that the expected test value of the proposed scheme is equivalent to the Rényi entropy of order α . After deriving a general representation of parameters of the proposed estimator, we discuss on the particular orders of Rényi entropy such as α 1 , α = 1 / 2 , and α = 2 . Because the Rényi entropy of order 2 is the most popular one, we present an iterative estimation method for the application with stringent resource restrictions. View Full-Text
Keywords: entropy estimation; Shannon entropy; Rényi entropy; quadratic entropy; random number generation; nearest neighbor distance; security entropy estimation; Shannon entropy; Rényi entropy; quadratic entropy; random number generation; nearest neighbor distance; security
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Kim, Y.-S. Low Complexity Estimation Method of Rényi Entropy for Ergodic Sources. Entropy 2018, 20, 657.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top