# Interacting Particle Solutions of Fokker–Planck Equations Through Gradient–Log–Density Estimation

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Deterministic Particle Dynamics for Fokker–Planck Equations

## 3. Variational Representation of Gradient–Log–Densities

## 4. Gradient–Log–Density Estimator

#### Estimating the Entropy Rate

## 5. Function Classes

#### 5.1. Linear Models

#### 5.2. Kernel Approaches

#### 5.3. A Sparse Kernel Approximation

## 6. A Note on Expectations

## 7. Equilibrium Dynamics

#### 7.1. Relative Entropy

#### 7.2. Relation to Stein Variational Gradient Descent

#### 7.3. Relation to Geometric Formulation of FPE Flow

## 8. Extension to General Diffusion Processes

## 9. Second Order Langevin Dynamics (Kramer’s Equation)

## 10. Simulating Accurate Fokker–Planck Solutions for Model Systems

#### 10.1. Linear Conservative System with Additive Noise

#### 10.2. Bi-Stable Nonlinear System with Additive Noise

#### 10.3. Nonlinear System Perturbed by Multiplicative Noise

#### 10.4. Performance in Higher Dimensions

#### 10.5. Second order Langevin Systems

#### 10.6. Nonconservative Chaotic System with Additive Noise (Lorenz Attractor)

## 11. Discussion and Outlook

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Appendix A. Simulated Systems

#### Appendix A.1. Two Dimensional Ornstein-Uhlenbeck Process

#### Appendix A.2. Bistable Nonlinear System

#### Appendix A.3. Multi-Dimensional Ornstein-Uhlenbeck Processes

#### Appendix A.4. Second Order Langevin Dynamics

#### Appendix A.5. Lorenz attractor

## Appendix B. Computing Central Moment Trajectories for Linear Processes

## Appendix C. Kullback–Leibler Divergence for Gaussian Distributions

## Appendix D. Wasserstein Distance

## Appendix E. Frobenious Norm

## Appendix F. Influence of Hyperparameter Values on the Performance of the Gradient–Log–Density Estimator

- -
- The hyperparameter that strongly influences the approximation accuracy is the kernel length scale l (Figure A1).
- -
- Underestimation of kernel length scale l has stronger impact on approximation accuracy than overestimation (Figure A1).
- -
- -
- For overestimation of the kernel length scale l, the regularisation parameter $\lambda $ and inducing point number M have nearly no effect on the resulting approximation error (Figure A1).
- -
- For underestimation of kernel length scale l, increasing the number of inducing points M in the estimator results in larger approximation errors (Figure A2 (upper left)).

**Figure A1.**Approximation error for increasing kernel length scale l for different regularisation parameter values $\lambda $ and inducing point number M.

**Figure A2.**Approximation error for increasing regularisation parameter value $\lambda $ for different kernel length scale l and inducing point number M.

## Appendix G. Required Number of Particles for Accurate Fokker–Planck Solutions

**Figure A3.**

**Required particle number, ${N}_{KL}^{*}$, to attain time averaged Kullback–Leibler divergence to ground truth, ${\langle \mathrm{KL}\left(\right)open="("\; close=")">{P}_{t}^{A},{P}_{t}^{N}}_{\rangle}$, for deterministic (green) and stochastic (brown) particle systems for a two dimensional Ornstein-Uhlenbeck process.**Markers indicate mean required particle number, while error bars denote one standard deviation over 20 independent realisations. Grey circles indicate required particle number for each individual realisation. The deterministic particle system consistently required at least one order of magnitude less particles compared to its stochastic counterpart. (Further parameter values: regularisation constant $\lambda =0.001$, inducing point number $M=100$, and RBF kernel length scale l estimated at every time point as two times the standard deviation of the state vector. Inducing point locations were selected randomly at each time step from a uniform distribution spanning the state space volume covered by the state vector).

## Appendix H. Algorithm for Simulating Deterministic Particle System

Algorithm A1: Gradient Log Density Estimator | |

Input: X: $N\times D$ state vectorZ: $M\times D$ inducing points vector d: dimension for gradient l: RBF Kernel length scale Output: G: $N\times 1$ vector for gradient-log-density at each position X in d dimension | |

1${K}^{xz}\u27f5K(X,Z;l)$ | // $N\times M$$\mathcal{O}\left(\right)open="("\; close=")">N\phantom{\rule{0.166667em}{0ex}}M$ |

2${K}^{zz}\u27f5K(Z,Z;l)$ | //$M\times M$$\mathcal{O}\left(\right)open="("\; close=")">{M}^{2}$ |

3$I\_{K}^{zz}\u27f5{\left(\right)}^{{K}^{zz}}-1$ | // $M\times M$$\mathcal{O}\left(\right)open="("\; close=")">{M}^{3}$ |

4$grad\_K\u27f5{\nabla}_{{X}^{\left(d\right)}}K(X,Z;l)$ | // $N\times M$$\mathcal{O}\left(\right)open="("\; close=")">N\phantom{\rule{0.166667em}{0ex}}M$ |

5$sgrad\_K\u27f5{\displaystyle \sum _{{X}_{i}}}grad\_K$ | // $1\times M$ |

6$G\u27f5{K}^{xz}\phantom{\rule{0.166667em}{0ex}}{\left(\right)}^{\lambda}\phantom{\rule{0.166667em}{0ex}}{K}^{xz}+{10}^{-3}\phantom{\rule{0.166667em}{0ex}}I\phantom{\rule{0.166667em}{0ex}}I\_{K}^{zz}\phantom{\rule{0.166667em}{0ex}}sgrad\_{K}^{\u22ba}$ | // $N\times 1$ |

//$\mathcal{O}\left(\right)open="("\; close=")">N\phantom{\rule{0.166667em}{0ex}}{M}^{2}$ |

Algorithm A2: Deterministic Particle Simulation |

## References

- Swain, P.S.; Elowitz, M.B.; Siggia, E.D. Intrinsic and extrinsic contributions to stochasticity in gene expression. Proc. Natl. Acad. Sci. USA
**2002**, 99, 12795–12800. [Google Scholar] [CrossRef] [PubMed][Green Version] - Pimentel, J.A.; Aldana, M.; Huepe, C.; Larralde, H. Intrinsic and extrinsic noise effects on phase transitions of network models with applications to swarming systems. Phys. Rev. E
**2008**, 77, 061138. [Google Scholar] [CrossRef] [PubMed][Green Version] - Hilfinger, A.; Paulsson, J. Separating intrinsic from extrinsic fluctuations in dynamic biological systems. Proc. Natl. Acad. Sci. USA
**2011**, 108, 12167–12172. [Google Scholar] [CrossRef] [PubMed][Green Version] - Elowitz, M.B.; Levine, A.J.; Siggia, E.D.; Swain, P.S. Stochastic gene expression in a single cell. Science
**2002**, 297, 1183–1186. [Google Scholar] [CrossRef][Green Version] - White, J.A.; Rubinstein, J.T.; Kay, A.R. Channel noise in neurons. Trends Neurosci.
**2000**, 23, 131–137. [Google Scholar] [CrossRef] - Volfson, D.; Marciniak, J.; Blake, W.J.; Ostroff, N.; Tsimring, L.S.; Hasty, J. Origins of extrinsic variability in eukaryotic gene expression. Nature
**2006**, 439, 861–864. [Google Scholar] [CrossRef] - Fellous, J.M.; Rudolph, M.; Destexhe, A.; Sejnowski, T.J. Synaptic background noise controls the input/output characteristics of single cells in an in vitro model of in vivo activity. Neuroscience
**2003**, 122, 811–829. [Google Scholar] [CrossRef] - Schnoerr, D.; Sanguinetti, G.; Grima, R. Approximation and inference methods for stochastic biochemical kinetics—A tutorial review. J. Phys. Math. Theor.
**2017**, 50, 093001. [Google Scholar] [CrossRef] - Van Kampen, N.G. The expansion of the master equation. Adv. Chem. Phys.
**1976**, 34, 245–309. [Google Scholar] - Suzuki, M. Passage from an initial unstable state to a final stable state. Adv. Chem. Phys.
**1981**, 46, 195–278. [Google Scholar] - Gammaitoni, L.; Hänggi, P.; Jung, P.; Marchesoni, F. Stochastic resonance. Rev. Mod. Phys.
**1998**, 70, 223. [Google Scholar] [CrossRef] - Benzi, R.; Parisi, G.; Sutera, A.; Vulpiani, A. Stochastic resonance in climatic change. Tellus
**1982**, 34, 10–16. [Google Scholar] [CrossRef] - Horsthemke, W. Noise induced transitions. In Non-Equilibrium Dynamics in Chemical Systems; Springer: Berlin/Heidelberg, Germany, 1984; pp. 150–160. [Google Scholar]
- Adorno, D.P.; Pizzolato, N.; Valenti, D.; Spagnolo, B. External noise effects in doped semiconductors operating under sub-THz signals. Rep. Math. Phys.
**2012**, 70, 171–179. [Google Scholar] [CrossRef][Green Version] - Assaf, M.; Roberts, E.; Luthey-Schulten, Z.; Goldenfeld, N. Extrinsic noise driven phenotype switching in a self-regulating gene. Phys. Rev. Lett.
**2013**, 111, 058102. [Google Scholar] [CrossRef] [PubMed][Green Version] - Balázsi, G.; van Oudenaarden, A.; Collins, J.J. Cellular decision making and biological noise: From microbes to mammals. Cell
**2011**, 144, 910–925. [Google Scholar] [CrossRef][Green Version] - Hughes, S.W.; Cope, D.W.; Toth, T.I.; Williams, S.R.; Crunelli, V. All thalamocortical neurones possess a T-type Ca2+ ‘window’current that enables the expression of bistability-mediated activities. J. Physiol.
**1999**, 517, 805–815. [Google Scholar] [CrossRef] - Rose, J.E.; Brugge, J.F.; Anderson, D.J.; Hind, J.E. Phase-locked response to low-frequency tones in single auditory nerve fibers of the squirrel monkey. J. Neurophysiol.
**1967**, 30, 769–793. [Google Scholar] [CrossRef] - Nicolis, C. Solar variability and stochastic effects on climate. Sol. Phys.
**1981**, 74, 473–478. [Google Scholar] [CrossRef] - Nicolis, C. Stochastic aspects of climatic transitions–response to a periodic forcing. Tellus
**1982**, 34, 1–9. [Google Scholar] [CrossRef] - Risken, H. Fokker-Planck equation. In The Fokker-Planck Equation; Springer: Berlin/Heidelberg, Germany, 1996; pp. 63–95. [Google Scholar]
- Wang, M.C.; Uhlenbeck, G.E. On the theory of the Brownian motion II. Rev. Mod. Phys.
**1945**, 17, 323. [Google Scholar] [CrossRef] - Särkkä, S.; Solin, A. Applied Stochastic Differential Equations; Cambridge University Press: Cambridge, UK, 2019; Volume 10. [Google Scholar]
- Gillespie, D.T. The chemical Langevin equation. J. Chem. Phys.
**2000**, 113, 297–306. [Google Scholar] [CrossRef] - Melykuti, B.; Burrage, K.; Zygalakis, K.C. Fast stochastic simulation of biochemical reaction systems by alternative formulations of the chemical Langevin equation. J. Chem. Phys.
**2010**, 132, 164109. [Google Scholar] [CrossRef] [PubMed][Green Version] - Schadschneider, A.; Chowdhury, D.; Nishinari, K. Stochastic Transport in Complex Systems: From Molecules to Vehicles; Elsevier: Amsterdam, The Netherlands, 2010. [Google Scholar]
- Kumar, P.; Narayanan, S. Solution of Fokker-Planck equation by finite element and finite difference methods for nonlinear systems. Sadhana
**2006**, 31, 445–461. [Google Scholar] [CrossRef] - Brics, M.; Kaupuzs, J.; Mahnke, R. How to solve Fokker-Planck equation treating mixed eigenvalue spectrum? arXiv
**2013**, arXiv:1303.5211. [Google Scholar] [CrossRef][Green Version] - Chang, J.; Cooper, G. A practical difference scheme for Fokker-Planck equations. J. Comput. Phys.
**1970**, 6, 1–16. [Google Scholar] [CrossRef] - Pichler, L.; Masud, A.; Bergman, L.A. Numerical solution of the Fokker–Planck equation by finite difference and finite element methods—A comparative study. In Computational Methods in Stochastic Dynamics; Springer: Berlin/Heidelberg, Germany, 2013; pp. 69–85. [Google Scholar]
- Harrison, G.W. Numerical solution of the Fokker Planck equation using moving finite elements. Numer. Methods Partial Differ. Equ.
**1988**, 4, 219–232. [Google Scholar] [CrossRef] - Epperlein, E. Implicit and conservative difference scheme for the Fokker-Planck equation. J. Comput. Phys.
**1994**, 112, 291–297. [Google Scholar] [CrossRef] - Leimkuhler, B.; Reich, S. Simulating Hamiltonian Dynamics; Cambridge University Press: Cambridge, UK, 2004; Volume 14. [Google Scholar]
- Chen, N.; Majda, A.J. Efficient statistically accurate algorithms for the Fokker–Planck equation in large dimensions. J. Comput. Phys.
**2018**, 354, 242–268. [Google Scholar] [CrossRef][Green Version] - Lin, Y.; Cai, G. Probabilistic Structural Dynamics: Advanced Theory and Applications; McGraw-Hill: New York, NY, USA, 1995. [Google Scholar]
- Roberts, J.B.; Spanos, P.D. Random Vibration and Statistical Linearization; Courier Corporation: Chelmsford, MA, USA, 2003. [Google Scholar]
- Proppe, C.; Pradlwarter, H.; Schuëller, G. Equivalent linearization and Monte Carlo simulation in stochastic dynamics. Probab. Eng. Mech.
**2003**, 18, 1–15. [Google Scholar] [CrossRef] - Grigoriu, M. Stochastic Calculus: Applications in Science and Engineering; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2013. [Google Scholar]
- ØKsendal, B. Stochastic Differential Equations; Springer: Berlin/Heidelberg, Germany, 2003. [Google Scholar]
- Kroese, D.P.; Taimre, T.; Botev, Z.I. Handbook of Monte Carlo Methods; John Wiley &; Sons: Hoboken, NJ, USA, 2013; Volume 706. [Google Scholar]
- Carrillo, J.; Craig, K.; Patacchini, F. A blob method for diffusion. Calc. Var. Partial Differ. Equ.
**2019**, 58, 1–53. [Google Scholar] [CrossRef][Green Version] - Pathiraja, S.; Reich, S. Discrete gradients for computational Bayesian inference. J. Comp. Dyn.
**2019**, 6, 236–251. [Google Scholar] [CrossRef][Green Version] - Reich, S.; Weissmann, S. Fokker-Planck particle systems for Bayesian inference: Computational approaches. arXiv
**2019**, arXiv:1911.10832. [Google Scholar] - Liu, Q.; Lee, J.; Jordan, M. A kernelized Stein discrepancy for goodness-of-fit tests. In Proceedings of the International Conference on Machine Learning, New York, NY, USA, 19–24 June 2016; pp. 276–284. [Google Scholar]
- Taghvaei, A.; Mehta, P.G. Accelerated flow for probability distributions. arXiv
**2019**, arXiv:1901.03317. [Google Scholar] - Velasco, R.M.; Scherer García-Colín, L.; Uribe, F.J. Entropy production: Its role in non-equilibrium thermodynamics. Entropy
**2011**, 13, 82–116. [Google Scholar] [CrossRef] - Hyvärinen, A. Estimation of non-normalized statistical models by score matching. J. Mach. Learn. Res.
**2005**, 6, 695–709. [Google Scholar] - Li, Y.; Turner, R.E. Gradient estimators for implicit models. arXiv
**2017**, arXiv:1705.07107. [Google Scholar] - Shi, J.; Sun, S.; Zhu, J. A spectral approach to gradient estimation for implicit distributions. arXiv
**2018**, arXiv:1806.02925. [Google Scholar] - Tomé, T.; De Oliveira, M.J. Stochastic Dynamics and Irreversibility; Springer: Berlin/Heidelberg, Germany, 2015. [Google Scholar]
- Shawe-Taylor, J.; Cristianini, N. Kernel Methods for Pattern Analysis; Cambridge University Press: Cambridge, UK, 2004. [Google Scholar]
- Scholkopf, B.; Smola, A.J. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond; MIT Press: Cambridge, MA, USA, 2001. [Google Scholar]
- Sutherland, D.J.; Strathmann, H.; Arbel, M.; Gretton, A. Efficient and principled score estimation with Nyström kernel exponential families. arXiv
**2017**, arXiv:1705.08360. [Google Scholar] - Rasmussen, C.E. Gaussian Processes in Machine Learning; Summer School on Machine Learning; Springer: Berlin/Heidelberg, Germany, 2003; pp. 63–71. [Google Scholar]
- Liu, Q.; Wang, D. Stein variational gradient descent: A general purpose Bayesian inference algorithm. In Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain, 5–10 December 2016; pp. 2378–2386. [Google Scholar]
- Liu, Q. Stein variational gradient descent as gradient flow. In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; pp. 3115–3123. [Google Scholar]
- Garbuno-Inigo, A.; Nüsken, N.; Reich, S. Affine invariant interacting Langevin dynamics for Bayesian inference. arXiv
**2019**, arXiv:1912.02859. [Google Scholar] - Otto, F. The geometry of dissipative evolution equations: The porous medium equation. Commun. Partial Differ. Equ.
**2001**, 26, 101–174. [Google Scholar] [CrossRef] - Villani, C. Optimal Transport: Old and New; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2008. [Google Scholar]
- Frogner, C.; Poggio, T. Approximate inference with Wasserstein gradient flows. arXiv
**2018**, arXiv:1806.04542. [Google Scholar] - Caluya, K.; Halder, A. Gradient flow algorithms for density propagation in stochastic systems. IEEE Trans. Autom. Control
**2019**. [Google Scholar] [CrossRef][Green Version] - Batz, P.; Ruttor, A.; Opper, M. Variational estimation of the drift for stochastic differential equations from the empirical density. J. Stat. Mech. Theory Exp.
**2016**, 2016, 083404. [Google Scholar] [CrossRef][Green Version] - Milstein, G.; Tretyakov, M. Computing ergodic limits for Langevin equations. Phys. D Nonlinear Phenom.
**2007**, 229, 81–95. [Google Scholar] [CrossRef] - Saremi, S.; Mehrjou, A.; Schölkopf, B.; Hyvärinen, A. Deep energy estimator networks. arXiv
**2018**, arXiv:1805.08306. [Google Scholar] - Reich, S.; Cotter, C. Probabilistic Forecasting and Bayesian Data Assimilation; Cambridge University Press: Cambridge, UK, 2015. [Google Scholar]
- Lorenz, E.N. Deterministic nonperiodic flow. J. Atmos. Sci.
**1963**, 20, 130–141. [Google Scholar] [CrossRef][Green Version] - Bobkov, S.; Ledoux, M. One-Dimensional Empirical Measures, Order Statistics and Kantorovich Transport Distances; Amer Mathematical Society: Providence, RI, USA, 2014. [Google Scholar]

**Figure 1.**

**Accuracy of Fokker–Planck solutions for two dimensional Ornstein Uhlenbeck process.**(

**a**) Mean, ${\left(\right)}_{{\mathcal{W}}_{1}}$, and (

**c**) stationary ${\mathcal{W}}_{1}({P}_{\infty}^{A},{P}_{\infty}^{N})$, 1-Wasserstein distance, between analytic solution and deterministic(D)/stochastic(S) simulations of N particles (for different inducing point number M). (

**b**) Average temporal deviations from analytic mean ${m}_{t}$ and (

**d**) covariance matrix ${C}_{t}$ for deterministic and stochastic system for increasing particle number N. Deterministic particle simulations consistently outperformed stochastic ones in approximating the temporal evolution of the mean and covariance of the distribution for all examined particle number settings. (Further parameter values: regularisation constant $\lambda =0.001$, Euler integration time step $dt={10}^{-3}$, and RBF kernel length scale l estimated at every time point as two times the standard deviation of the state vector. Inducing point locations were selected randomly at each time step from a uniform distribution spanning the state space volume covered by the state vector).

**Figure 2.**

**Stationary and transient Fokker–Planck solutions computed with deterministic (green) and stochastic (brown) particle dynamics for a two dimensional Ornstein Uhlenbeck process.**(

**a**,

**b**) Estimated stationary PDFs arising from deterministic ($N=1000$) (green), and stochastic ($N=1000$) (brown) particle dynamics. Purple contours denote analytically calculated stationary distributions, while top and side histograms display marginal distributions for each dimension. (

**c**) Temporal evolution of marginal statistics, mean $\langle x\rangle $, standard deviation ${\sigma}_{x}$, skewness ${s}_{x}$, and kurtosis ${k}_{x}$, for analytic solution (A), and for stochastic (S) and deterministic (D) particle systems comprising $N=1000$, with initial state distribution $\mathcal{N}\left(\right)open="("\; close=")">\left(\right)open="["\; close="]">\begin{array}{c}0.5\\ 0.5\end{array}$, for $M=100$ randomly selected inducing points employed in the gradient–log–density estimation. Deterministic particle simulations deliver smooth cumulant trajectories, as opposed to highly fluctuating stochastic particle cumulants. (Further parameter values: regularisation constant $\lambda =0.001$, and RBF kernel length scale l estimated at every time point as two times the standard deviation of the state vector. Inducing point locations were selected randomly at each time step from a uniform distribution spanning the state space volume covered by the state vector).

**Figure 3.**

**Performance of deterministic (green) and stochastic (brown) N particle solutions compared to ${N}^{\infty}$ (grey) stochastic particle densities for a nonlinear bi-stable process.**(

**a**) Instances of estimated pdfs arising from (

**left**) stochastic (${N}^{\infty}$ = 26,000) (grey) and deterministic ($N=1000$) (green), and (

**right**) stochastic (${N}^{\infty}=26,000$) (grey) and stochastic ($N=1000$) (brown) particle dynamics at times (

**i**) $t=0.005$, (

**ii**) $t=0.231$, and (

**iii**) $t=1.244$. (

**b**) Temporal evolution of first four distribution cumulants, mean $\langle x\rangle $, standard deviation ${\sigma}_{x}$, skewness ${s}_{x}$, and kurtosis ${k}_{x}$, for stochastic (${S}^{\infty}$ and S) and deterministic (D) systems comprising ${N}^{\infty}$ = 26,000, $N=1000$, with initial state distribution $\mathcal{N}(0,0.{05}^{2})$, by employing $M=150$ inducing points in the gradient–log–density estimation. (

**c**) Mean, ${\left(\right)}_{{\mathcal{W}}_{1}}$, and (

**d**) stationary, ${\mathcal{W}}_{1}({P}_{\infty}^{A},{P}_{\infty}^{N})$, 1-Wasserstein distance, between ${N}^{\infty}$ = 26,000 stochastic, and deterministic (D)/stochastic (S) simulations of N particles (for different inducing point number M). (Further parameter values: regularisation constant $\lambda =0.001$, Euler integration time step $dt={10}^{-3}$, and RBF kernel length scale $l=0.5$. Inducing point locations were selected randomly at each time step from a uniform distribution spanning the state space volume covered by the state vector).

**Figure 4.**

**Accuracy of Fokker–Planck solutions for a nonlinear system perturbed with state-dependent noise.**(

**a**) Instances of $N=1000$ particle distributions resulting from deterministic (green) and (

**b**) stochastic (brown) simulations against stochastic particle distributions comprising ${N}^{\infty}$ = 35,000 particles (grey) for (

**i**) $t=0.1$, (

**ii**) $t=3.2$, and (

**iii**) $t=4.4$. Insets provide a closer view of details of distribution for visual clarity. Distributions resulting from deterministic particle simulations closer agree with underlying distribution for all three instances. (

**c**) Temporal evolution of first four cumulants for the three particle systems (grey: ${S}_{\infty}$—stochastic with ${N}^{\infty}$ = 35000 particles, brown: S - stochastic with N = 1000 particles, and green: D—deterministic with N = 1000 particles). Deterministically evolved distributions result in smooth cumulant trajectories. (

**d**,

**e**) Temporal average and (

**f**,

**g**) stationary 1-Wasserstein distance between distributions mediated through stochastic simulations of ${N}^{\infty}$ = 35000, and through deterministic (green) and stochastic (brown) simulations of N particles against particle number N and inducing point number M. Shaded regions and error bars denote one standard deviation among 20 independent repetitions. Different green hues designate different inducing point number M employed in the gradient–log–density estimation. (Further parameter values: regularisation constant $\lambda =0.001$, Euler integration time step $dt={10}^{-3}$, and RBF kernel length scale $l=0.25$. Inducing points were arranged on a regular grid spanning the instantaneous state space volume captured by the state vector).

**Figure 5.**

**Accuracy of Fokker-Planck solutions for multi-dimensional Ornstein–Uhlenbeck processes for $M=100$ inducing points.**Comparison of deterministic particle Fokker–Planck solutions with stochastic particle systems and analytic solutions for multi-dimensional Ornstein–Uhlenbeck process of D = {2, 3, 4, 5} dimensions. (

**a**) Time-averaged and (

**d**) stationary Kullback–Leibler (KL) divergence between simulated particle solutions (green: deterministic, brown: stochastic) and analytic solutions for different dimensions. Deterministic particle simulations outperform stochastic particle solutions even for increasing system dimensionality. (

**b**) Time averaged and (

**e**) stationary error between analytic, ${m}_{t}$, and sample mean, ${\widehat{m}}_{t}$, for increasing particle number. (

**c**) Time averaged and (

**f**) stationary discrepancy between simulated, ${\widehat{C}}_{t}$, and analytic covariances, ${C}_{t}$, as captured by the Frobenius norm of the relevant covariance matrices difference. The accuracy of the estimated covariance decreases for increasing dimensionality. (Further parameter values: regularisation constant $\lambda =0.001$, Euler integration time step $dt={10}^{-3}$, and adaptive RBF kernel length scale l calculated at every time step as two times the standard deviation of the state vector. Inducing point locations were selected randomly at each time step from a uniform distribution spanning the state space volume covered by the state vector).

**Figure 6.**

**Accuracy of Fokker-Planck solutions for multi-dimensional Ornstein–Uhlenbeck processes for $M=200$ inducing points.**Comparison of deterministic particle Fokker–Planck solutions with stochastic particle systems and analytic solutions for multi-dimensional Ornstein–Uhlenbeck process of D = {2,3,4,5} dimensions. (

**a**) Time-averaged and (

**d**) stationary Kullback–Leibler (KL) divergence between simulated particle solutions (green: deterministic, brown: stochastic) and analytic solutions for different dimensions. Deterministic particle simulations outperform stochastic particle solutions even for increasing system dimensionality. (

**b**) Time averaged and (

**e**) stationary error between analytic, ${m}_{t}$, and sample mean, ${\widehat{m}}_{t}$, for increasing particle number. (

**c**) Time averaged and (

**f**) stationary discrepancy between simulated, ${\widehat{C}}_{t}$, and analytic covariances, ${C}_{t}$, as captured by the Frobenius norm of the relevant covariance matrices difference. The accuracy of the estimated covariance decreases for increasing dimensionality. (Further parameter values: regularisation constant $\lambda =0.001$, Euler integration time step $dt={10}^{-3}$, and adaptive RBF kernel length scale l calculated at every time step as two times the standard deviation of the state vector. Inducing point locations were selected randomly at each time step from a uniform distribution spanning the state space volume covered by the state vector).

**Figure 7.**

**Energy preservation for second order Langevin dynamics in a quadratic potential.**Comparison of deterministic particle Fokker–Planck solutions with stochastic particle systems for a harmonic oscillator (

**a**) First four cumulant temporal evolution for deterministic (green) and stochastic (brown) system. (

**b**) Stationary joint and marginal distributions for deterministic (green) and stochastic (brown) systems. Purple lines denote analytically derived stationary distributions. (

**c,d**) State space trajectory of a single particle for deterministic (green) and stochastic (brown) system. Color gradients denote time. (

**e**) Temporal evolution of individual particle energy ${E}_{t}^{\left(i\right)}$ for deterministic system for 5 particles. (

**f**) Difference between velocity and gradient–log–density term for individual particles. After the system reaches stationary state the particle velocity and GLD term cancel out. (

**g**) Ensemble average kinetic energy through time resorts to $\frac{{\sigma}^{2}}{2\phantom{\rule{0.166667em}{0ex}}\gamma}$ (grey dashed line) after equilibrium is reached. (Further parameter values: regularisation constant $\lambda =0.001$, integration time step $dt=2\times {10}^{-3}$, and adaptive RBF kernel length scale l calculated at every time step as two times the standard deviation of the state vector. Number of inducing points $M=300$. Inducing point locations were selected randomly at each time step from a uniform distribution spanning the state space volume covered by the state vector).

**Figure 8.**

**Energy preservation for second order Langevin dynamics in a double well potential.**Comparison of deterministic particle Fokker–Planck solutions with stochastic particle systems for a bistable process. (

**a**,

**b**) Joint and marginal distributions of system states mediated by $N=8000$ particles evolved with our framework (green) and with direct stochastic simulations comprising ${N}^{\infty}$ = 20,000 (grey) and $N=8000$ (brown) particles at (

**a**) $t=0.6$, and (

**b**) $t=10$. Purple lines denote the analytically derived stationary density. (

**c**) First four cumulant temporal evolution for deterministic (green) and stochastic (brown) system. (

**d**) State space trajectory of a single particle for deterministic and (

**e**) stochastic system. Color gradients denote time. (

**f**) Temporal evolution of individual particle energy ${E}_{t}^{\left(i\right)}$ for deterministic system for 5 particles. (

**g**) Temporal evolution of distribution of particle energies ${E}_{t}^{\left(i\right)}$ for deterministic (green) and stochastic (brown) system. (

**h**) Difference between velocity and gradient log density term for individual particles. (

**i**) Ensemble average kinetic energy through time resorts to $\frac{{\sigma}^{2}}{2\phantom{\rule{0.166667em}{0ex}}\gamma}$ (grey dashed line) after equilibrium is reached. (Further parameter values: regularisation constant $\lambda =0.001$, integration time step $dt=2\phantom{\rule{3.33333pt}{0ex}}\times \phantom{\rule{3.33333pt}{0ex}}{10}^{-3}$ and adaptive RBF kernel length scale l calculated at every time step as two times the standard deviation of the state vector. Number of inducing points $M=300$. Inducing point locations were selected randomly at each time step from a uniform distribution spanning the state space volume covered by the state vector).

**Figure 9.**

**Deterministic (green) and stochastic (brown) Fokker–Planck particle solutions for a three dimensional Lorenz attractor system in the chaotic regime perturbed by additive Gaussian noise.**(

**a**) Joint and marginal distributions of system states mediated by $N=4000$ particles evolved with our framework (green) and with direct stochastic simulations comprising N = 150,000 (grey) and $N=4000$ (brown) particles at $t=0.4$. (

**b**) Cumulant trajectories for the three particle systems. Cumulants derived from deterministic particle simulations (green) closer match cumulant evolution of the underlying distribution (grey) compared to stochastic simulations (brown). (Further parameter values: regularisation constant $\lambda =0.001$, Euler integration time step $dt={10}^{-3}$, adaptive RBF kernel length scale l calculated at every time step as two times the standard deviation of the state vector. Number of inducing points: $M=200$. Inducing point locations were selected randomly at each time step from a uniform distribution spanning the state space volume covered by the state vector).

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Maoutsa, D.; Reich, S.; Opper, M.
Interacting Particle Solutions of Fokker–Planck Equations Through Gradient–Log–Density Estimation. *Entropy* **2020**, *22*, 802.
https://doi.org/10.3390/e22080802

**AMA Style**

Maoutsa D, Reich S, Opper M.
Interacting Particle Solutions of Fokker–Planck Equations Through Gradient–Log–Density Estimation. *Entropy*. 2020; 22(8):802.
https://doi.org/10.3390/e22080802

**Chicago/Turabian Style**

Maoutsa, Dimitra, Sebastian Reich, and Manfred Opper.
2020. "Interacting Particle Solutions of Fokker–Planck Equations Through Gradient–Log–Density Estimation" *Entropy* 22, no. 8: 802.
https://doi.org/10.3390/e22080802