Next Article in Journal
Radiomics Analysis on Contrast-Enhanced Spectral Mammography Images for Breast Cancer Diagnosis: A Pilot Study
Next Article in Special Issue
TI-Stan: Model Comparison Using Thermodynamic Integration and HMC
Previous Article in Journal / Special Issue
Bayesian Maximum-A-Posteriori Approach with Global and Local Regularization to Image Reconstruction Problem in Medical Emission Tomography
Open AccessArticle

Stochastic Gradient Annealed Importance Sampling for Efficient Online Marginal Likelihood Estimation

1
Department of Physics, Stellenbosch University, Stellenbosch 7600, South Africa
2
National Institute for Theoretical Physics, Stellenbosch 7600, South Africa
3
Computer Science Division, Stellenbosch University, Stellenbosch 7600, South Africa
*
Author to whom correspondence should be addressed.
This paper is an extended version of our paper published in MaxEnt 2019.
Entropy 2019, 21(11), 1109; https://doi.org/10.3390/e21111109
Received: 30 September 2019 / Revised: 1 November 2019 / Accepted: 4 November 2019 / Published: 12 November 2019
We consider estimating the marginal likelihood in settings with independent and identically distributed (i.i.d.) data. We propose estimating the predictive distributions in a sequential factorization of the marginal likelihood in such settings by using stochastic gradient Markov Chain Monte Carlo techniques. This approach is far more efficient than traditional marginal likelihood estimation techniques such as nested sampling and annealed importance sampling due to its use of mini-batches to approximate the likelihood. Stability of the estimates is provided by an adaptive annealing schedule. The resulting stochastic gradient annealed importance sampling (SGAIS) technique, which is the key contribution of our paper, enables us to estimate the marginal likelihood of a number of models considerably faster than traditional approaches, with no noticeable loss of accuracy. An important benefit of our approach is that the marginal likelihood is calculated in an online fashion as data becomes available, allowing the estimates to be used for applications such as online weighted model combination. View Full-Text
Keywords: marginal likelihood; evidence; nested sampling; annealed importance sampling; Monte Carlo; stochastic gradients; SGHMC marginal likelihood; evidence; nested sampling; annealed importance sampling; Monte Carlo; stochastic gradients; SGHMC
Show Figures

Figure 1

MDPI and ACS Style

Cameron, S.A.; Eggers, H.C.; Kroon, S. Stochastic Gradient Annealed Importance Sampling for Efficient Online Marginal Likelihood Estimation. Entropy 2019, 21, 1109.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop