Next Article in Journal
Algorithmic Probability Method Versus Kolmogorov Complexity with No-Threshold Encoding Scheme for Short Time Series: An Analysis of Day-To-Day Hourly Solar Radiation Time Series over Tropical Western Indian Ocean
Next Article in Special Issue
Online Gradient Descent for Kernel-Based Maximum Correntropy Criterion
Previous Article in Journal
Information-Domain Analysis of Cardiovascular Complexity: Night and Day Modulations of Entropy and the Effects of Hypertension
Previous Article in Special Issue
Compact Belief Rule Base Learning for Classification with Evidential Clustering
Open AccessFeature PaperArticle

MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning

by Diego Granziol 1,2,*,†, Binxin Ru 1,2,*,†, Stefan Zohren 1,2, Xiaowen Dong 1,2, Michael Osborne 1,2 and Stephen Roberts 1,2
1
Machine Learning Research Group, University of Oxford, Walton Well Rd, Oxford OX2 6ED, UK
2
Oxford-Man Institute of Quantitative Finance, Walton Well Rd, Oxford OX2 6ED, UK
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
Entropy 2019, 21(6), 551; https://doi.org/10.3390/e21060551
Received: 30 April 2019 / Revised: 25 May 2019 / Accepted: 29 May 2019 / Published: 31 May 2019
(This article belongs to the Special Issue Entropy Based Inference and Optimization in Machine Learning)
Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the proposed method, its equivalence to constrained Bayesian variational inference and demonstrate its superiority over existing approaches in two applications, namely, fast log determinant estimation and information-theoretic Bayesian optimisation. View Full-Text
Keywords: maximum entropy; log determinant estimation; Bayesian optimisation maximum entropy; log determinant estimation; Bayesian optimisation
Show Figures

Figure 1

MDPI and ACS Style

Granziol, D.; Ru, B.; Zohren, S.; Dong, X.; Osborne, M.; Roberts, S. MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning. Entropy 2019, 21, 551.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop