Next Article in Journal
Multicomponent and High Entropy Alloys
Next Article in Special Issue
Simultaneous State and Parameter Estimation Using Maximum Relative Entropy with Nonhomogenous Differential Equation Constraints
Previous Article in Journal
Some New Results on the Multiple-AccessWiretap Channel
Previous Article in Special Issue
Identifying Chaotic FitzHugh–Nagumo Neurons Using Compressive Sensing

Information Anatomy of Stochastic Equilibria

Department of Physics, University of California at Berkeley, Berkeley, CA 94720, USA
Complexity Sciences Center, Department of Physics, University of California at Davis, One Shields Avenue, Davis, CA 95616, USA
Authors to whom correspondence should be addressed.
Entropy 2014, 16(9), 4713-4748;
Received: 17 March 2014 / Revised: 3 August 2014 / Accepted: 19 August 2014 / Published: 25 August 2014
(This article belongs to the Special Issue Information in Dynamical Systems and Complex Systems)
A stochastic nonlinear dynamical system generates information, as measured by its entropy rate. Some—the ephemeral information—is dissipated and some—the bound information—is actively stored and so affects future behavior. We derive analytic expressions for the ephemeral and bound information in the limit of infinitesimal time discretization for two classical systems that exhibit dynamical equilibria: first-order Langevin equations (i) where the drift is the gradient of an analytic potential function and the diffusion matrix is invertible and (ii) with a linear drift term (Ornstein–Uhlenbeck), but a noninvertible diffusion matrix. In both cases, the bound information is sensitive to the drift and diffusion, while the ephemeral information is sensitive only to the diffusion matrix and not to the drift. Notably, this information anatomy changes discontinuously as any of the diffusion coefficients vanishes, indicating that it is very sensitive to the noise structure. We then calculate the information anatomy of the stochastic cusp catastrophe and of particles diffusing in a heat bath in the overdamped limit, both examples of stochastic gradient descent on a potential landscape. Finally, we use our methods to calculate and compare approximations for the time-local predictive information for adaptive agents. View Full-Text
Keywords: Langevin equation; entropy rate; ephemeral information; bound information;time-local predictive information Langevin equation; entropy rate; ephemeral information; bound information;time-local predictive information
Show Figures

Graphical abstract

MDPI and ACS Style

Marzen, S.; Crutchfield, J.P. Information Anatomy of Stochastic Equilibria. Entropy 2014, 16, 4713-4748.

AMA Style

Marzen S, Crutchfield JP. Information Anatomy of Stochastic Equilibria. Entropy. 2014; 16(9):4713-4748.

Chicago/Turabian Style

Marzen, Sarah, and James P. Crutchfield 2014. "Information Anatomy of Stochastic Equilibria" Entropy 16, no. 9: 4713-4748.

Find Other Styles

Article Access Map by Country/Region

Only visits after 24 November 2015 are recorded.
Back to TopTop