# Entropy-Constrained Scalar Quantization with a Lossy-Compressed Bit

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. ECSQ and the Uniform Quantizer

- C1
- ${p}_{X}(x)\mathrm{log}{p}_{X}(x)$ is integrable, ensuring that the differential entropy $h(X)$ is well-defined and finite; and
- C2
- the integer part of the source X has finite entropy; i.e.,$$H(\lfloor X\rfloor )<\infty $$

## 3. Uniform Quantization with a Lossy-Compressed Bit

#### 3.1. Compression with a Lossy-Compressed Bit

#### 3.2. Reconstruction Values and Squared Distortion with a Lossy-Compressed Bit

**Lemma**

**1.**

**Proof.**

#### 3.3. Asymptotic Gap to the Shannon Lower Bound

**Lemma**

**2.**

**Proof.**

**Corollary**

**1.**

## 4. Lb-ECSQ in the High Distortion Regime

#### 4.1. Two-Level Quantization of a Uniform Source

**Lemma**

**3.**

**Proof.**

#### 4.2. Two-Level Quantization of a Gaussian Source

**Lemma**

**4.**

**Proof.**

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Gish, H.; Pierce, J. Asymptotically efficient quantizing. IEEE Trans. Inf. Theory
**1968**, 14, 676–683. [Google Scholar] - Goblick, T.; Holsinger, J. Analog source digitization: A comparison of theory and practice (Corresp.). IEEE Trans. Inf. Theory
**1967**, 13, 323–326. [Google Scholar] - Farvardin, N.; Modestino, J. Optimum quantizer performance for a class of non-Gaussian memoryless sources. IEEE Trans. Inf. Theory
**1984**, 30, 485–497. [Google Scholar] - Sullivan, G. Efficient scalar quantization of exponential and Laplacian random variables. IEEE Trans. Inf. Theory
**1996**, 42, 1365–1374. [Google Scholar] - Noll, P.; Zelinski, R. Bounds on Quantizer Performance in the Low Bit-Rate Region. IEEE Trans. Inf. Theory
**1978**, 26, 300–304. [Google Scholar] - Gyorgy, A.; Linder, T. Optimal entropy-constrained scalar quantization of a uniform source. IEEE Trans. Inf. Theory
**2000**, 46, 2704–2711. [Google Scholar] - Linder, T.; Zeger, K. Asymptotic entropy-constrained performance of tessellating and universal randomized lattice quantization. IEEE Trans. Inf. Theory
**1994**, 40, 575–579. [Google Scholar] - Koch, T.; Vazquez-Vilar, G. Rate-Distortion Bounds for High-Resolution Vector Quantization via Gibbs’s Inequality. arxiv
**2015**. [Google Scholar] - Chou, P.A.; Lookabaugh, T.; Gray, R.M. Entropy-constrained vector quantization. IEEE Trans. Acoust. Speech Signal Process.
**1989**, 37, 31–42. [Google Scholar] - Farvardin, N.; Vaishampayan, V. Optimal quantizer design for noisy channels: An approach to combined source-channel coding. IEEE Trans. Inf. Theory
**1987**, 33, 827–838. [Google Scholar] - Spilker, J.J. Digital Communications by Satellite; Prentice-Hall: Upper Saddle River, NJ, USA, 1977. [Google Scholar]
- Kurtenbach, A.J.; Wintz, P.A. Quantizing for noisy channels. IEEE Trans. Inf. Theory
**1969**, 17, 291–302. [Google Scholar] - Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J.
**1948**, 27, 379–423. [Google Scholar] - Koch, T. The shannon lower bound is asymptotically tight for sources with finite renyi information dimension. IEEE Trans. Inf. Theory
**2015**. [Google Scholar] [CrossRef] - Billingsley, P. Convergence of Probability Measures; John Wiley: New York, NY, USA, 1968. [Google Scholar]
- Rényi, A. Probability Theory; North-Holland Series in Applied Mathematics and Mechanics; Elsevier: Budapest, Hungary, 1970. [Google Scholar]
- Linder, T.; Zamir, R. On the asymptotic tightness of the Shannon lower bound. IEEE Trans. Inf. Theory
**1994**, 40, 2026–2031. [Google Scholar] - Aref, V.; Macris, N.; Vuffray, M. Approaching the rate-distortion limit with spatial coupling, belief propagation, and decimation. IEEE Trans. Inf. Theory
**2015**, 61, 3954–3979. [Google Scholar] - Calderbank, A.R.; Fishburn, P.C.; Rabinovich, A. Covering properties of convolutional codes and associated lattices. IEEE Trans. Inf. Theory
**1995**, 41, 732–746. [Google Scholar] - Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley Series in Telecommunications and Signal Processing; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2006. [Google Scholar]
- Marco, D.; Neuhoff, D. Low-resolution scalar quantization for Gaussian sources and squared error. IEEE Trans. Inf. Theory
**2006**, 52, 1689–1697. [Google Scholar]

**Figure 1.**Binary Source Channel model of the joint probability distribution between a Bernoulli source $b(X)$ with prior probability ${P}_{b}$ and its reconstruction $\widehat{b}(X)$ after lossy compression at Hamming distortion ${D}_{H}$, assuming that the Bernoulli rate distortion function is achieved.

**Figure 3.**Entropy-constrained scalar quantization (ECSQ) and lossy-bit ECSQ (Lb-ECSQ) rate distortion function for a 1-bit quantizer and uniform source, $X\sim U\in [-\delta /2,\delta /2]$ for $\delta =\sqrt{12}$.

**Figure 4.**For $X\sim \mathcal{N}(0,{\sigma}^{2})$, we plot the gap between the ECSQ and Lb-ECSQ rate distortion function for a 1-bit quantizer and the rate distortion function for the source; i.e., $R(D)=0.5{\mathrm{log}}_{2}({\sigma}^{2}/D)$.

© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Pradier, M.F.; Olmos, P.M.; Perez-Cruz, F. Entropy-Constrained Scalar Quantization with a Lossy-Compressed Bit. *Entropy* **2016**, *18*, 449.
https://doi.org/10.3390/e18120449

**AMA Style**

Pradier MF, Olmos PM, Perez-Cruz F. Entropy-Constrained Scalar Quantization with a Lossy-Compressed Bit. *Entropy*. 2016; 18(12):449.
https://doi.org/10.3390/e18120449

**Chicago/Turabian Style**

Pradier, Melanie F., Pablo M. Olmos, and Fernando Perez-Cruz. 2016. "Entropy-Constrained Scalar Quantization with a Lossy-Compressed Bit" *Entropy* 18, no. 12: 449.
https://doi.org/10.3390/e18120449