Next Article in Journal
A Stochastic Model for Block Segmentation of Images Based on the Quadtree and the Bayes Code for It
Next Article in Special Issue
Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices
Previous Article in Journal
DiLizium: A Two-Party Lattice-Based Signature Scheme
Previous Article in Special Issue
ABCDP: Approximate Bayesian Computation with Differential Privacy
 
 
Article

Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation

1
Artificial Intelligence Group, Technische Universität Berlin, 10623 Berlin, Germany
2
Amazon Web Services, 10969 Berlin, Germany
3
Centre for Systems Modelling and Quantitative Biomedicine, University of Birmingham, Birmingham B15 2TT, UK
*
Author to whom correspondence should be addressed.
Academic Editor: Pierre Alquier
Entropy 2021, 23(8), 990; https://doi.org/10.3390/e23080990
Received: 22 June 2021 / Revised: 15 July 2021 / Accepted: 21 July 2021 / Published: 30 July 2021
(This article belongs to the Special Issue Approximate Bayesian Inference)
Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectations are known in closed-form or readily computed by quadrature. In this paper, we view the Gaussian variational approximation problem through the lens of gradient flows. We introduce a flexible and efficient algorithm based on a linear flow leading to a particle-based approximation. We prove that, with a sufficient number of particles, our algorithm converges linearly to the exact solution for Gaussian targets, and a low-rank approximation otherwise. In addition to the theoretical analysis, we show, on a set of synthetic and real-world high-dimensional problems, that our algorithm outperforms existing methods with Gaussian targets while performing on a par with non-Gaussian targets. View Full-Text
Keywords: variational inference; Gaussian; particle flow; variable flow variational inference; Gaussian; particle flow; variable flow
Show Figures

Figure 1

MDPI and ACS Style

Galy-Fajou, T.; Perrone, V.; Opper, M. Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation. Entropy 2021, 23, 990. https://doi.org/10.3390/e23080990

AMA Style

Galy-Fajou T, Perrone V, Opper M. Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation. Entropy. 2021; 23(8):990. https://doi.org/10.3390/e23080990

Chicago/Turabian Style

Galy-Fajou, Théo, Valerio Perrone, and Manfred Opper. 2021. "Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation" Entropy 23, no. 8: 990. https://doi.org/10.3390/e23080990

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop