Next Article in Journal
Feeding Back the Output or Sharing the State: Which Is Better for the State-Dependent Wiretap Channel?
Previous Article in Journal
The Bogdanov–Takens Normal Form: A Minimal Model for Single Neuron Dynamics
Open AccessArticle

Distributed Vector Quantization Based on Kullback-Leibler Divergence

College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China
Author to whom correspondence should be addressed.
Academic Editor: Raúl Alcaraz Martínez
Entropy 2015, 17(12), 7875-7887;
Received: 22 June 2015 / Revised: 14 October 2015 / Accepted: 23 November 2015 / Published: 30 November 2015
(This article belongs to the Section Information Theory, Probability and Statistics)
The goal of vector quantization is to use a few reproduction vectors to represent original vectors/data while maintaining the necessary fidelity of the data. Distributed signal processing has received much attention in recent years, since in many applications data are dispersedly collected/stored in distributed nodes over networks, but centralizing all these data to one processing center is sometimes impractical. In this paper, we develop a distributed vector quantization (VQ) algorithm based on Kullback-Leibler (K-L) divergence. We start from the centralized case and propose to minimize the K-L divergence between the distribution of global original data and the distribution of global reproduction vectors, and then obtain an online iterative solution to this optimization problem based on the Robbins-Monro stochastic approximation. Afterwards, we extend the solution to apply to distributed cases by introducing diffusion cooperation among nodes. Numerical simulations show that the performances of the distributed K-L–based VQ algorithm are very close to the corresponding centralized algorithm. Besides, both the centralized and distributed K-L–based VQ show more robustness to outliers than the (centralized) Linde-Buzo-Gray (LBG) algorithm and the (centralized) self-organization map (SOM) algorithm. View Full-Text
Keywords: distributed signal processing; Kullback-Leibler divergence; sensor network; vector quantization distributed signal processing; Kullback-Leibler divergence; sensor network; vector quantization
Show Figures

Figure 1

MDPI and ACS Style

Shen, P.; Li, C.; Luo, Y. Distributed Vector Quantization Based on Kullback-Leibler Divergence. Entropy 2015, 17, 7875-7887.

Show more citation formats Show less citations formats

Article Access Map by Country/Region

Search more from Scilit
Back to TopTop