On Relations Between the Relative Entropy and χ2-Divergence, Generalizations and Applications
1
Independent Researcher, Tokyo 206–0003, Japan
2
Faculty of Electrical Engineering, Technion—Israel Institute of Technology, Technion City, Haifa 3200003, Israel
*
Author to whom correspondence should be addressed.
Entropy 2020, 22(5), 563; https://doi.org/10.3390/e22050563
Received: 22 April 2020
/
Revised: 12 May 2020
/
Accepted: 17 May 2020
/
Published: 18 May 2020
(This article belongs to the Special Issue Divergence Measures: Mathematical Foundations and Applications in Information-Theoretic and Statistical Problems)
The relative entropy and the chi-squared divergence are fundamental divergence measures in information theory and statistics. This paper is focused on a study of integral relations between the two divergences, the implications of these relations, their information-theoretic applications, and some generalizations pertaining to the rich class of f-divergences. Applications that are studied in this paper refer to lossless compression, the method of types and large deviations, strong data–processing inequalities, bounds on contraction coefficients and maximal correlation, and the convergence rate to stationarity of a type of discrete-time Markov chains.
View Full-Text
Keywords:
relative entropy; chi-squared divergence; f-divergences; method of types; large deviations; strong data–processing inequalities; information contraction; maximal correlation; Markov chains
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited
MDPI and ACS Style
Nishiyama, T.; Sason, I. On Relations Between the Relative Entropy and χ2-Divergence, Generalizations and Applications. Entropy 2020, 22, 563. https://doi.org/10.3390/e22050563
AMA Style
Nishiyama T, Sason I. On Relations Between the Relative Entropy and χ2-Divergence, Generalizations and Applications. Entropy. 2020; 22(5):563. https://doi.org/10.3390/e22050563
Chicago/Turabian StyleNishiyama, Tomohiro, and Igal Sason. 2020. "On Relations Between the Relative Entropy and χ2-Divergence, Generalizations and Applications" Entropy 22, no. 5: 563. https://doi.org/10.3390/e22050563
Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
