Next Article in Journal
The Effect of Catalogue Lead Time on Medium-Term Earthquake Forecasting with Application to New Zealand Data
Next Article in Special Issue
Approximate Bayesian Inference
Previous Article in Journal
What Can Local Transfer Entropy Tell Us about Phase-Amplitude Coupling in Electrophysiological Signals?
Article

Dynamics of Coordinate Ascent Variational Inference: A Case Study in 2D Ising Models

Department of Statistics, Texas A&M University, College Station, TX 77843, USA
*
Author to whom correspondence should be addressed.
Entropy 2020, 22(11), 1263; https://doi.org/10.3390/e22111263
Received: 3 September 2020 / Revised: 26 October 2020 / Accepted: 3 November 2020 / Published: 6 November 2020
(This article belongs to the Special Issue Approximate Bayesian Inference)
Variational algorithms have gained prominence over the past two decades as a scalable computational environment for Bayesian inference. In this article, we explore tools from the dynamical systems literature to study the convergence of coordinate ascent algorithms for mean field variational inference. Focusing on the Ising model defined on two nodes, we fully characterize the dynamics of the sequential coordinate ascent algorithm and its parallel version. We observe that in the regime where the objective function is convex, both the algorithms are stable and exhibit convergence to the unique fixed point. Our analyses reveal interesting discordances between these two versions of the algorithm in the region when the objective function is non-convex. In fact, the parallel version exhibits a periodic oscillatory behavior which is absent in the sequential version. Drawing intuition from the Markov chain Monte Carlo literature, we empirically show that a parameter expansion of the Ising model, popularly called the Edward–Sokal coupling, leads to an enlargement of the regime of convergence to the global optima. View Full-Text
Keywords: bifurcation; dynamical systems; Edward–Sokal coupling; mean-field; Kullback–Leibler divergence; variational inference bifurcation; dynamical systems; Edward–Sokal coupling; mean-field; Kullback–Leibler divergence; variational inference
Show Figures

Figure 1

MDPI and ACS Style

Plummer, S.; Pati, D.; Bhattacharya, A. Dynamics of Coordinate Ascent Variational Inference: A Case Study in 2D Ising Models. Entropy 2020, 22, 1263. https://doi.org/10.3390/e22111263

AMA Style

Plummer S, Pati D, Bhattacharya A. Dynamics of Coordinate Ascent Variational Inference: A Case Study in 2D Ising Models. Entropy. 2020; 22(11):1263. https://doi.org/10.3390/e22111263

Chicago/Turabian Style

Plummer, Sean, Debdeep Pati, and Anirban Bhattacharya. 2020. "Dynamics of Coordinate Ascent Variational Inference: A Case Study in 2D Ising Models" Entropy 22, no. 11: 1263. https://doi.org/10.3390/e22111263

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop