Entropy 2014, 16(7), 3605-3634; doi:10.3390/e16073605
Article

Entropy Evolution and Uncertainty Estimation with Dynamical Systems

1,2email
Received: 4 May 2014; in revised form: 11 June 2014 / Accepted: 19 June 2014 / Published: 30 June 2014
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract: This paper presents a comprehensive introduction and systematic derivation of the evolutionary equations for absolute entropy H and relative entropy D, some of which exist sporadically in the literature in different forms under different subjects, within the framework of dynamical systems. In general, both H and D are dissipated, and the dissipation bears a form reminiscent of the Fisher information; in the absence of stochasticity, dH/dt is connected to the rate of phase space expansion, and D stays invariant, i.e., the separation of two probability density functions is always conserved. These formulas are validated with linear systems, and put to application with the Lorenz system and a large-dimensional stochastic quasi-geostrophic flow problem. In the Lorenz case, H falls at a constant rate with time, implying that H will eventually become negative, a situation beyond the capability of the commonly used computational technique like coarse-graining and bin counting. For the stochastic flow problem, it is first reduced to a computationally tractable low-dimensional system, using a reduced model approach, and then handled through ensemble prediction. Both the Lorenz system and the stochastic flow system are examples of self-organization in the light of uncertainty reduction. The latter particularly shows that, sometimes stochasticity may actually enhance the self-organization process.
Keywords: uncertainty estimation; entropy; second law of thermodynamics; Lorenz system; ensemble prediction; quasi-geostrophic flow; shear instability; Fisher information matrix; reduced model approach; self-organization
PDF Full-text Download PDF Full-Text [530 KB, uploaded 30 June 2014 10:58 CEST]

Export to BibTeX |
EndNote


MDPI and ACS Style

Liang, X.S. Entropy Evolution and Uncertainty Estimation with Dynamical Systems. Entropy 2014, 16, 3605-3634.

AMA Style

Liang XS. Entropy Evolution and Uncertainty Estimation with Dynamical Systems. Entropy. 2014; 16(7):3605-3634.

Chicago/Turabian Style

Liang, X. S. 2014. "Entropy Evolution and Uncertainty Estimation with Dynamical Systems." Entropy 16, no. 7: 3605-3634.

Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert