Next Article in Journal
Infrared Thermography for DIEP Flap Breast Reconstruction Part I: Measurements
Previous Article in Journal
Auditable Blockchain Randomization Tool
Open AccessProceedings

A Sequential Marginal Likelihood Approximation Using Stochastic Gradients

1
Department of Physics, Stellenbosch University, Stellenbosch 7600, South Africa
2
National Institute for Theoretical Physics, Stellenbosch 7600, South Africa
3
Computer Science Division, Stellenbosch University, Stellenbosch 7600, South Africa
*
Author to whom correspondence should be addressed.
Presented at the 39th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Garching, Germany, 30 June–5 July 2019.
Proceedings 2019, 33(1), 18; https://doi.org/10.3390/proceedings2019033018
Published: 3 December 2019
Existing algorithms like nested sampling and annealed importance sampling are able to produce accurate estimates of the marginal likelihood of a model, but tend to scale poorly to large data sets. This is because these algorithms need to recalculate the log-likelihood for each iteration by summing over the whole data set. Efficient scaling to large data sets requires that algorithms only visit small subsets (mini-batches) of data on each iteration. To this end, we estimate the marginal likelihood via a sequential decomposition into a product of predictive distributions p ( y n | y < n ) . Predictive distributions can be approximated efficiently through Bayesian updating using stochastic gradient Hamiltonian Monte Carlo, which approximates likelihood gradients using mini-batches. Since each data point typically contains little information compared to the whole data set, the convergence to each successive posterior only requires a short burn-in phase. This approach can be viewed as a special case of sequential Monte Carlo (SMC) with a single particle, but differs from typical SMC methods in that it uses stochastic gradients. We illustrate how this approach scales favourably to large data sets with some simple models.
Keywords: marginal likelihood; Monte Carlo; stochastic gradients marginal likelihood; Monte Carlo; stochastic gradients
MDPI and ACS Style

Cameron, S.A.; Eggers, H.C.; Kroon, S. A Sequential Marginal Likelihood Approximation Using Stochastic Gradients. Proceedings 2019, 33, 18.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop