Next Article in Journal
Entropy-Based Greedy Algorithm for Decision Trees Using Hypotheses
Next Article in Special Issue
ABCDP: Approximate Bayesian Computation with Differential Privacy
Previous Article in Journal
Psychomotor Predictive Processing
Previous Article in Special Issue
Understanding the Variability in Graph Data Sets through Statistical Modeling on the Stiefel Manifold
Article

Variational Message Passing and Local Constraint Manipulation in Factor Graphs

1
Department of Electrical Engineering, Eindhoven University of Technology, 5600 MB Eindhoven, The Netherlands
2
GN Hearing, JF Kennedylaan 2, 5612 AB Eindhoven, The Netherlands
*
Author to whom correspondence should be addressed.
Academic Editor: Pierre Alquier
Entropy 2021, 23(7), 807; https://doi.org/10.3390/e23070807
Received: 19 May 2021 / Revised: 18 June 2021 / Accepted: 22 June 2021 / Published: 24 June 2021
(This article belongs to the Special Issue Approximate Bayesian Inference)
Accurate evaluation of Bayesian model evidence for a given data set is a fundamental problem in model development. Since evidence evaluations are usually intractable, in practice variational free energy (VFE) minimization provides an attractive alternative, as the VFE is an upper bound on negative model log-evidence (NLE). In order to improve tractability of the VFE, it is common to manipulate the constraints in the search space for the posterior distribution of the latent variables. Unfortunately, constraint manipulation may also lead to a less accurate estimate of the NLE. Thus, constraint manipulation implies an engineering trade-off between tractability and accuracy of model evidence estimation. In this paper, we develop a unifying account of constraint manipulation for variational inference in models that can be represented by a (Forney-style) factor graph, for which we identify the Bethe Free Energy as an approximation to the VFE. We derive well-known message passing algorithms from first principles, as the result of minimizing the constrained Bethe Free Energy (BFE). The proposed method supports evaluation of the BFE in factor graphs for model scoring and development of new message passing-based inference algorithms that potentially improve evidence estimation accuracy. View Full-Text
Keywords: Bayesian inference; Bethe free energy; factor graphs; message passing; variational free energy; variational inference; variational message passing Bayesian inference; Bethe free energy; factor graphs; message passing; variational free energy; variational inference; variational message passing
Show Figures

Figure 1

MDPI and ACS Style

Şenöz, İ.; van de Laar, T.; Bagaev, D.; de Vries, B. Variational Message Passing and Local Constraint Manipulation in Factor Graphs. Entropy 2021, 23, 807. https://doi.org/10.3390/e23070807

AMA Style

Şenöz İ, van de Laar T, Bagaev D, de Vries B. Variational Message Passing and Local Constraint Manipulation in Factor Graphs. Entropy. 2021; 23(7):807. https://doi.org/10.3390/e23070807

Chicago/Turabian Style

Şenöz, İsmail, Thijs van de Laar, Dmitry Bagaev, and Bert de Vries. 2021. "Variational Message Passing and Local Constraint Manipulation in Factor Graphs" Entropy 23, no. 7: 807. https://doi.org/10.3390/e23070807

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop