Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
4.9
2.1
Journals
Entropy
Special Issues
The Statistical Physics of Generative Diffusion Models
entropy-logo
Submit to Special Issue Submit Abstract to Special Issue Review for Entropy Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

The Statistical Physics of Generative Diffusion Models

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Statistical Physics".

Deadline for manuscript submissions: 15 October 2025 | Viewed by 2741

Special Issue Editor

Dr. Luca Ambrogioni

E-Mail Website
Guest Editor
Donders Institute for Brain, Cognition, and Behaviour, Radboud University, 6525 XZ Nijmegen, The Netherlands
Interests: generative models; diffusion models; deep learning; variational Inference; theoretical neuroscience

Special Issue Information

Dear Colleagues,

Generative diffusion models and related methods such as stochastic interpolants have become the state of the art in image and video generation. While these methods were inspired by the physics of out-of-equilibrium systems, recent work revealed deep connections between generative diffusion models and equilibrium statistical physics. In particular, it was shown that the generative diffusion process is punctuated by spontaneous symmetry breaking events that correspond to splits between semantic classes or visual features and are formally equivalent to mean-field critical phase transitions. These ‘speciation’ phase transitions correspond to critical windows where the generative process is maximally controllable. Another fascinating venue of research is the connection between generative diffusion models and associative memory networks such as (modern) Hopfield networks. For example, using Hopfield techniques, it has been shown that memorization in generative diffusion is the result of ‘glassy’ (i.e., disordered) phase transitions in the average free energy. The connections between spin glass sampling and generative diffusion have been further investigated using the concept of stochastic localization, which describes the (spontaneous) concentration of measure observed in generative diffusion sampling. These developments have the potential to drive a large inflow of physical theory and techniques to the study of generative machine learning models, which could lead to radical insights on the nature of learning and intelligence.

Given these fascinating developments, we are excited to launch a Special Issue aimed at connecting research in statistical physics and generative diffusion modeling. Authors are encouraged to submit their research to this Special Issue. Topics include, but are not limited to, the following:

  • Theoretical analysis of generative diffusion processes;
  • Connection between diffusion models and Hopfield networks;
  • Statistical physics analysis of flow matching processes and stochastic interpolants;
  • Theoretical analysis of prompt conditioning in generative diffusion;
  • Differential geometry of diffusion latent manifolds;
  • Acceleration of generative diffusion sampling using computational physics methods;
  • Discrete generative diffusion processes;
  • Connection between generative diffusion processes and spin glasses;
  • Spontaneous symmetry breaking in equivariant generative diffusion models;
  • Applications of generative diffusion to statistical physics problems;
  • Stochastic localization processes;
  • Statistical physics of consistency models;
  • Applications of generative diffusion to econophysics and finance.

Dr. Luca Ambrogioni
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • generative diffusion models
  • generative models
  • statistical physics
  • equilibrium thermodynamics
  • symmetry breaking
  • phase transition
  • stochastic localization
  • Hopfield networks

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue policies can be found here.

Published Papers (4 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

29 pages, 1945 KiB  
Article
Latent Abstractions in Generative Diffusion Models
by Giulio Franzese, Mattia Martini, Giulio Corallo, Paolo Papotti and Pietro Michiardi
Entropy 2025, 27(4), 371; https://doi.org/10.3390/e27040371 - 31 Mar 2025
Viewed by 258
Abstract
In this work, we study how diffusion-based generative models produce high-dimensional data, such as images, by relying on latent abstractions that guide the generative process. We introduce a novel theoretical framework extending Nonlinear Filtering (NLF), offering a new perspective on SDE-based generative models. [...] Read more.
In this work, we study how diffusion-based generative models produce high-dimensional data, such as images, by relying on latent abstractions that guide the generative process. We introduce a novel theoretical framework extending Nonlinear Filtering (NLF), offering a new perspective on SDE-based generative models. Our theory is based on a new formulation of joint (state and measurement) dynamics and an information-theoretic measure of state influence on the measurement process. We show that diffusion models can be interpreted as a system of SDE, describing a non-linear filter where unobservable latent abstractions steer the dynamics of an observable measurement process. Additionally, we present an empirical study validating our theory and supporting previous findings on the emergence of latent abstractions at different generative stages. Full article
(This article belongs to the Special Issue The Statistical Physics of Generative Diffusion Models)
►▼ Show Figures

Figure 1

25 pages, 14143 KiB  
Article
U-Turn Diffusion
by Hamidreza Behjoo and Michael Chertkov
Entropy 2025, 27(4), 343; https://doi.org/10.3390/e27040343 - 26 Mar 2025
Viewed by 223
Abstract
We investigate diffusion models generating synthetic samples from the probability distribution represented by the ground truth (GT) samples. We focus on how GT sample information is encoded in the score function (SF), computed (not simulated) from the Wiener–Ito linear forward process in the [...] Read more.
We investigate diffusion models generating synthetic samples from the probability distribution represented by the ground truth (GT) samples. We focus on how GT sample information is encoded in the score function (SF), computed (not simulated) from the Wiener–Ito linear forward process in the artificial time t∈[0→∞], and then used as a nonlinear drift in the simulated Wiener–Ito reverse process with t∈[∞→0]. We propose U-Turn diffusion, an augmentation of a pre-trained diffusion model, which shortens the forward and reverse processes to t∈[0→Tu] and t∈[Tu→0]. The U-Turn reverse process is initialized at Tu with a sample from the probability distribution of the forward process (initialized at t=0 with a GT sample) ensuring a detailed balance relation between the shortened forward and reverse processes. Our experiments on the class-conditioned SF of the ImageNet dataset and the multi-class, single SF of the CIFAR-10 dataset reveal a critical Memorization Time Tm, beyond which generated samples diverge from the GT sample used to initialize the U-Turn scheme, and a Speciation Time Ts, where for Tu>Ts>Tm, samples begin representing different classes. We further examine the role of SF nonlinearity through a Gaussian Test, comparing empirical and Gaussian-approximated U-Turn auto-correlation functions and showing that the SF becomes effectively affine for t>Ts and approximately affine for t∈[Tm,Ts]. Full article
(This article belongs to the Special Issue The Statistical Physics of Generative Diffusion Models)
►▼ Show Figures

Figure 1

17 pages, 991 KiB  
Article
The Statistical Thermodynamics of Generative Diffusion Models: Phase Transitions, Symmetry Breaking, and Critical Instability
by Luca Ambrogioni
Entropy 2025, 27(3), 291; https://doi.org/10.3390/e27030291 - 11 Mar 2025
Cited by 15 | Viewed by 648
Abstract
Generative diffusion models have achieved spectacular performance in many areas of machine learning and generative modeling. While the fundamental ideas behind these models come from non-equilibrium physics, variational inference, and stochastic calculus, in this paper we show that many aspects of these models [...] Read more.
Generative diffusion models have achieved spectacular performance in many areas of machine learning and generative modeling. While the fundamental ideas behind these models come from non-equilibrium physics, variational inference, and stochastic calculus, in this paper we show that many aspects of these models can be understood using the tools of equilibrium statistical mechanics. Using this reformulation, we show that generative diffusion models undergo second-order phase transitions corresponding to symmetry breaking phenomena. We show that these phase transitions are always in a mean-field universality class, as they are the result of a self-consistency condition in the generative dynamics. We argue that the critical instability arising from these phase transitions lies at the heart of their generative capabilities, which are characterized by a set of mean-field critical exponents. Finally, we show that the dynamic equation of the generative process can be interpreted as a stochastic adiabatic transformation that minimizes the free energy while keeping the system in thermal equilibrium. Full article
(This article belongs to the Special Issue The Statistical Physics of Generative Diffusion Models)
►▼ Show Figures

Figure 1

16 pages, 4864 KiB  
Article
Control of Overfitting with Physics
by Sergei V. Kozyrev, Ilya A. Lopatin and Alexander N. Pechen
Entropy 2024, 26(12), 1090; https://doi.org/10.3390/e26121090 - 13 Dec 2024
Cited by 1 | Viewed by 844
Abstract
While there are many works on the applications of machine learning, not so many of them are trying to understand the theoretical justifications to explain their efficiency. In this work, overfitting control (or generalization property) in machine learning is explained using analogies from [...] Read more.
While there are many works on the applications of machine learning, not so many of them are trying to understand the theoretical justifications to explain their efficiency. In this work, overfitting control (or generalization property) in machine learning is explained using analogies from physics and biology. For stochastic gradient Langevin dynamics, we show that the Eyring formula of kinetic theory allows to control overfitting in the algorithmic stability approach—when wide minima of the risk function with low free energy correspond to low overfitting. For the generative adversarial network (GAN) model, we establish an analogy between GAN and the predator–prey model in biology. An application of this analogy allows us to explain the selection of wide likelihood maxima and ab overfitting reduction for GANs. Full article
(This article belongs to the Special Issue The Statistical Physics of Generative Diffusion Models)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-4
Back to TopTop