# Projection to Mixture Families and Rate-Distortion Bounds with Power Distortion Measures

## Abstract

**:**

## 1. Introduction

## 2. Rate-Distortion Function and Shannon Lower Bound

#### 2.1. Rate-Distortion Function

#### 2.2. Shannon Lower Bound

**Lemma**

**1.**

**Proof.**

#### 2.3. Probability Density Achieving Tight SLB for All D

**Theorem**

**1.**

**Proof.**

## 3. Generalized Gaussian Source and Power Distortion Measure

#### 3.1. $\beta $-th Power Distortion Measure

#### 3.2. Tightness of the SLB

**Corollary**

**1.**

**Corollary**

**2.**

## 4. Rate-Distortion Bounds for Mismatching Pairs

**Lemma**

**2.**

**Proof.**

**Theorem**

**2.**

**Theorem**

**3.**

**Proof.**

**Corollary**

**3.**

**Corollary**

**4.**

**Example**

**1.**

**Example**

**2.**

**Corollary**

**5.**

**Proof.**

## 5. Distortion-Rate Bounds for $\mathit{\u03f5}$-Insensitive Loss

**Theorem**

**4.**

## 6. Conclusions

## Acknowledgments

## Conflicts of Interest

## References

- Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J.
**1948**, 27, 623–656. [Google Scholar] [CrossRef] - Berger, T. Rate Distortion Theory: A Mathematical Basis for Data Compression; Prentice-Hall: Englewood Cliffs, NJ, USA, 1971. [Google Scholar]
- Buzo, A.; Kuhlmann, F.; Rivera, C. Rate-distortion bounds for quotient-based distortions with application to Itakura-Saito distortion measures. IEEE Trans. Inf. Theory
**1986**, 32, 141–147. [Google Scholar] [CrossRef] - Gerrish, A.; Schultheiss, P. Information rates of non-Gaussian processes. IEEE Trans. Inf. Theory
**1964**, 10, 265–271. [Google Scholar] [CrossRef] - Kostina, V. Data compression with low distortion and finite blocklength. IEEE Trans. Inf. Theory
**2017**, in press. [Google Scholar] [CrossRef] - Tan, H.H.; Yao, K. Evaluation of rate-distortion functions for a class of independent identically distributed sources under an absolute magnitude criterion. IEEE Trans. Inf. Theory
**1975**, 21, 59–64. [Google Scholar] [CrossRef] - Yao, K.; Tan, H.H. Absolute error rate-distortion functions for sources with constrained magnitudes. IEEE Trans. Inf. Theory
**1978**, 24, 499–503. [Google Scholar] - Rose, K. A mapping approach to rate-distortion computation and analysis. IEEE Trans. Inf. Theory
**1994**, 40, 1939–1952. [Google Scholar] [CrossRef] - Watanabe, K.; Ikeda, S. Rate-Distortion functions for gamma-type sources under absolute-log distortion measure. IEEE Trans. Inf. Theory
**2016**, 62, 5496–5502. [Google Scholar] [CrossRef] - Amari, S.; Nagaoka, H. Methods Information Geometry; Oxford University Press: Oxford, UK, 2000. [Google Scholar]
- Watanabe, K. Constant-width rate-distortion bounds for power distortion measures. In Proceedings of the 2016 IEEE Information Theory Workshop (ITW), Cambridge, UK, 11–14 September 2016; pp. 106–110. [Google Scholar]
- Fraysse, A.; Pesquet-Popescu, B.; Pesquet, J.C. Rate-distortion results for generalized Gaussian distributions. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Las Vegas, NV, USA, 31 March–4 April 2008; pp. 3753–3756. [Google Scholar]
- Fraysse, A.; Pesquet-Popescu, B.; Pesquet, J.C. On the uniform quantization of a class of sparse sources. IEEE Trans. Inf. Theory
**2009**, 55, 3243–3263. [Google Scholar] [CrossRef] - Steinwart, I.; Christmann, A. Support Vector Machines; Springer: Berlin/Heidelberg, Germany, 2008. [Google Scholar]
- Chu, W.; Keerthi, S.S.; Ong, C.J. Bayesian support vector regression using a unified loss function. IEEE Trans. Neural Netw.
**2004**, 15, 29–44. [Google Scholar] [CrossRef] [PubMed] - Watanabe, K. Rate-distortion bounds for ε-insensitive distortion measures. IEICE Trans. Fundam.
**2016**, E99-A, 370–377. [Google Scholar] [CrossRef] - Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley Interscience: New York, NY, USA, 1991. [Google Scholar]
- Gray, R.M. Source Coding Theory; Kluwer Academic Publishers: Dordrecht, The Netherlands, 1990. [Google Scholar]
- Gray, R.M. Entropy and Information Theory, 2nd ed.; Springer: Berlin/Heidelberg, Germany, 2011. [Google Scholar]
- Nishiyama, Y.; Fukumizu, K. Characteristic kernels and infinitely divisible distributions. J. Mach. Learn. Res.
**2016**, 17, 1–28. [Google Scholar] - Linder, T.; Zamir, R. On the asymptotic tightness of the Shannon lower bound. IEEE Trans. Inf. Theory
**1994**, 40, 2026–2031. [Google Scholar] [CrossRef] - Koch, T. The Shannon lower bound is asymptotically tight. IEEE Trans. Inf. Theory
**2016**, 62, 6155–6161. [Google Scholar] [CrossRef]

© 2017 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Watanabe, K.
Projection to Mixture Families and Rate-Distortion Bounds with Power Distortion Measures . *Entropy* **2017**, *19*, 262.
https://doi.org/10.3390/e19060262

**AMA Style**

Watanabe K.
Projection to Mixture Families and Rate-Distortion Bounds with Power Distortion Measures . *Entropy*. 2017; 19(6):262.
https://doi.org/10.3390/e19060262

**Chicago/Turabian Style**

Watanabe, Kazuho.
2017. "Projection to Mixture Families and Rate-Distortion Bounds with Power Distortion Measures " *Entropy* 19, no. 6: 262.
https://doi.org/10.3390/e19060262