Next Article in Journal
Application of Sample Entropy Based LMD-TFPF De-Noising Algorithm for the Gear Transmission System
Previous Article in Journal
Symplectic Entropy as a Novel Measure for Complex Systems
Article Menu
Issue 11 (November) cover image

Export Article

Open AccessArticle
Entropy 2016, 18(11), 410;

Information-Theoretic Analysis of Memoryless Deterministic Systems

Institute for Communications Engineering, Technical University of Munich, Munich 80290, Germany
Signal Processing and Speech Communication Laboratory, Graz University of Technology, Graz 8010, Austria
Author to whom correspondence should be addressed.
Academic Editors: Raúl Alcaraz Martínez and Kevin H. Knuth
Received: 18 August 2016 / Revised: 27 October 2016 / Accepted: 14 November 2016 / Published: 17 November 2016
(This article belongs to the Section Information Theory)
View Full-Text   |   Download PDF [369 KB, uploaded 17 November 2016]   |  


The information loss in deterministic, memoryless systems is investigated by evaluating the conditional entropy of the input random variable given the output random variable. It is shown that for a large class of systems the information loss is finite, even if the input has a continuous distribution. For systems with infinite information loss, a relative measure is defined and shown to be related to Rényi information dimension. As deterministic signal processing can only destroy information, it is important to know how this information loss affects the solution of inverse problems. Hence, we connect the probability of perfectly reconstructing the input to the information lost in the system via Fano-type bounds. The theoretical results are illustrated by example systems commonly used in discrete-time, nonlinear signal processing and communications. View Full-Text
Keywords: information processing; signal processing; system theory; Fano’s inequality; information loss; Rényi information dimension information processing; signal processing; system theory; Fano’s inequality; information loss; Rényi information dimension

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).

Share & Cite This Article

MDPI and ACS Style

Geiger, B.C.; Kubin, G. Information-Theoretic Analysis of Memoryless Deterministic Systems. Entropy 2016, 18, 410.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top