Next Article in Journal
Sociophysics Analysis of Multi-Group Conflicts
Previous Article in Journal
On the Determination of Kappa Distribution Functions from Space Plasma Observations
Previous Article in Special Issue
Cross-Entropy Method for Content Placement and User Association in Cache-Enabled Coordinated Ultra-Dense Networks
Open AccessArticle

Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embedding

by Yiğit Uğur 1,2,* and Abdellatif Zaidi 1,2,*
1
Laboratoire d’informatique Gaspard-Monge, Université Paris-Est, 77454 Champs-sur-Marne, France
2
Mathematical and Algorithmic Sciences Lab, Paris Research Center, Huawei Technologies, 92100 Boulogne-Billancourt, France
*
Authors to whom correspondence should be addressed.
Entropy 2020, 22(2), 213; https://doi.org/10.3390/e22020213
Received: 3 December 2019 / Revised: 4 February 2020 / Accepted: 9 February 2020 / Published: 13 February 2020
(This article belongs to the Special Issue Information Theory for Data Communications and Processing)
In this paper, we develop an unsupervised generative clustering framework that combines the variational information bottleneck and the Gaussian mixture model. Specifically, in our approach, we use the variational information bottleneck method and model the latent space as a mixture of Gaussians. We derive a bound on the cost function of our model that generalizes the Evidence Lower Bound (ELBO) and provide a variational inference type algorithm that allows computing it. In the algorithm, the coders’ mappings are parametrized using neural networks, and the bound is approximated by Markov sampling and optimized with stochastic gradient descent. Numerical results on real datasets are provided to support the efficiency of our method.
Keywords: clustering; unsupervised learning; Gaussian mixture model; information bottleneck clustering; unsupervised learning; Gaussian mixture model; information bottleneck
MDPI and ACS Style

Uğur, Y.; Zaidi, A. Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embedding. Entropy 2020, 22, 213.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop