Entropy Evolution and Uncertainty Estimation with Dynamical Systems
AbstractThis paper presents a comprehensive introduction and systematic derivation of the evolutionary equations for absolute entropy H and relative entropy D, some of which exist sporadically in the literature in different forms under different subjects, within the framework of dynamical systems. In general, both H and D are dissipated, and the dissipation bears a form reminiscent of the Fisher information; in the absence of stochasticity, dH/dt is connected to the rate of phase space expansion, and D stays invariant, i.e., the separation of two probability density functions is always conserved. These formulas are validated with linear systems, and put to application with the Lorenz system and a large-dimensional stochastic quasi-geostrophic flow problem. In the Lorenz case, H falls at a constant rate with time, implying that H will eventually become negative, a situation beyond the capability of the commonly used computational technique like coarse-graining and bin counting. For the stochastic flow problem, it is first reduced to a computationally tractable low-dimensional system, using a reduced model approach, and then handled through ensemble prediction. Both the Lorenz system and the stochastic flow system are examples of self-organization in the light of uncertainty reduction. The latter particularly shows that, sometimes stochasticity may actually enhance the self-organization process. View Full-Text
Share & Cite This Article
Liang, X.S. Entropy Evolution and Uncertainty Estimation with Dynamical Systems. Entropy 2014, 16, 3605-3634.
Liang XS. Entropy Evolution and Uncertainty Estimation with Dynamical Systems. Entropy. 2014; 16(7):3605-3634.Chicago/Turabian Style
Liang, X. S. 2014. "Entropy Evolution and Uncertainty Estimation with Dynamical Systems." Entropy 16, no. 7: 3605-3634.