Next Article in Journal
Identification of the Framingham Risk Score by an Entropy-Based Rule Model for Cardiovascular Disease
Next Article in Special Issue
Probabilistic Models with Deep Neural Networks
Previous Article in Journal
Assessment of Nocturnal Autonomic Cardiac Imbalance in Positional Obstructive Sleep Apnea. A Multiscale Nonlinear Approach
Article

A Robust Solution to Variational Importance Sampling of Minimum Variance

1
Serra Húnter Fellow at Department of Mathematics and Computer Science, University of Barcelona, 08007 Barcelona, Spain
2
Artificial Intelligence Research Institute (IIIA-CSIC), 08193 Bellaterra, Spain
*
Author to whom correspondence should be addressed.
Entropy 2020, 22(12), 1405; https://doi.org/10.3390/e22121405
Received: 18 November 2020 / Revised: 10 December 2020 / Accepted: 10 December 2020 / Published: 12 December 2020
(This article belongs to the Special Issue Bayesian Inference in Probabilistic Graphical Models)
Importance sampling is a Monte Carlo method where samples are obtained from an alternative proposal distribution. This can be used to focus the sampling process in the relevant parts of space, thus reducing the variance. Selecting the proposal that leads to the minimum variance can be formulated as an optimization problem and solved, for instance, by the use of a variational approach. Variational inference selects, from a given family, the distribution which minimizes the divergence to the distribution of interest. The Rényi projection of order 2 leads to the importance sampling estimator of minimum variance, but its computation is very costly. In this study with discrete distributions that factorize over probabilistic graphical models, we propose and evaluate an approximate projection method onto fully factored distributions. As a result of our evaluation it becomes apparent that a proposal distribution mixing the information projection with the approximate Rényi projection of order 2 could be interesting from a practical perspective. View Full-Text
Keywords: importance sampling; minimum variance unbiased estimator; Rényi divergence; variational inference; fully factorized family importance sampling; minimum variance unbiased estimator; Rényi divergence; variational inference; fully factorized family
Show Figures

Figure 1

MDPI and ACS Style

Hernández-González, J.; Cerquides, J. A Robust Solution to Variational Importance Sampling of Minimum Variance. Entropy 2020, 22, 1405. https://doi.org/10.3390/e22121405

AMA Style

Hernández-González J, Cerquides J. A Robust Solution to Variational Importance Sampling of Minimum Variance. Entropy. 2020; 22(12):1405. https://doi.org/10.3390/e22121405

Chicago/Turabian Style

Hernández-González, Jerónimo, and Jesús Cerquides. 2020. "A Robust Solution to Variational Importance Sampling of Minimum Variance" Entropy 22, no. 12: 1405. https://doi.org/10.3390/e22121405

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop