Next Article in Journal
Granger-Causality Inference of the Existence of Unobserved Important Components in Network Analysis
Next Article in Special Issue
Fast Compression of MCMC Output
Previous Article in Journal
A Novel Algebraic Structure of (α,β)-Complex Fuzzy Subgroups
Previous Article in Special Issue
Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation
Article

Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices

1
Communication Science Laboratories, NTT, Hikaridai, Seika-cho, “Keihanna Science City”, Kyoto 619-0237, Japan
2
Department of Computer Science, Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo 113-0033, Japan
*
Author to whom correspondence should be addressed.
Academic Editor: Pierre Alquier
Entropy 2021, 23(8), 993; https://doi.org/10.3390/e23080993
Received: 21 June 2021 / Revised: 14 July 2021 / Accepted: 27 July 2021 / Published: 30 July 2021
(This article belongs to the Special Issue Approximate Bayesian Inference)
Langevin dynamics (LD) has been extensively studied theoretically and practically as a basic sampling technique. Recently, the incorporation of non-reversible dynamics into LD is attracting attention because it accelerates the mixing speed of LD. Popular choices for non-reversible dynamics include underdamped Langevin dynamics (ULD), which uses second-order dynamics and perturbations with skew-symmetric matrices. Although ULD has been widely used in practice, the application of skew acceleration is limited although it is expected to show superior performance theoretically. Current work lacks a theoretical understanding of issues that are important to practitioners, including the selection criteria for skew-symmetric matrices, quantitative evaluations of acceleration, and the large memory cost of storing skew matrices. In this study, we theoretically and numerically clarify these problems by analyzing acceleration focusing on how the skew-symmetric matrix perturbs the Hessian matrix of potential functions. We also present a practical algorithm that accelerates the standard LD and ULD, which uses novel memory-efficient skew-symmetric matrices under parallel-chain Monte Carlo settings. View Full-Text
Keywords: Markov Chain Monte Carlo; Langevin dynamics; Hamilton Monte Carlo; non-reversible dynamics Markov Chain Monte Carlo; Langevin dynamics; Hamilton Monte Carlo; non-reversible dynamics
Show Figures

Figure 1

MDPI and ACS Style

Futami, F.; Iwata, T.; Ueda, N.; Sato, I. Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices. Entropy 2021, 23, 993. https://doi.org/10.3390/e23080993

AMA Style

Futami F, Iwata T, Ueda N, Sato I. Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices. Entropy. 2021; 23(8):993. https://doi.org/10.3390/e23080993

Chicago/Turabian Style

Futami, Futoshi, Tomoharu Iwata, Naonori Ueda, and Issei Sato. 2021. "Accelerated Diffusion-Based Sampling by the Non-Reversible Dynamics with Skew-Symmetric Matrices" Entropy 23, no. 8: 993. https://doi.org/10.3390/e23080993

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop