Next Article in Journal
Positive Sofic Entropy Implies Finite Stabilizer
Previous Article in Journal
Three Strategies for the Design of Advanced High-Entropy Alloys
Article Menu

Export Article

Open AccessArticle
Entropy 2016, 18(7), 262;

Greedy Algorithms for Optimal Distribution Approximation

Institute for Communications Engineering, Technical University of Munich, Munich 80290, Germany
Author to whom correspondence should be addressed.
Academic Editor: Raúl Alcaraz Martínez
Received: 14 June 2016 / Revised: 1 July 2016 / Accepted: 11 July 2016 / Published: 18 July 2016
(This article belongs to the Section Information Theory)
View Full-Text   |   Download PDF [278 KB, uploaded 18 July 2016]   |  


The approximation of a discrete probability distribution t by an M-type distribution p is considered. The approximation error is measured by the informational divergence D ( t p ) , which is an appropriate measure, e.g., in the context of data compression. Properties of the optimal approximation are derived and bounds on the approximation error are presented, which are asymptotically tight. A greedy algorithm is proposed that solves this M-type approximation problem optimally. Finally, it is shown that different instantiations of this algorithm minimize the informational divergence D ( p t ) or the variational distance p t 1 . View Full-Text
Keywords: distribution approximation; finite precision; informational divergence; greedy algorithm distribution approximation; finite precision; informational divergence; greedy algorithm

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).

Share & Cite This Article

MDPI and ACS Style

Geiger, B.C.; Böcherer, G. Greedy Algorithms for Optimal Distribution Approximation. Entropy 2016, 18, 262.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top