Entropy

Editorial

Jump to: Research

3 pages, 111 KiB

Open AccessEditorial

Concepts of Entropy and Their Applications

by Phil Broadbridge and Anthony J. Guttmann

Entropy 2009, 11(1), 59-61; https://doi.org/10.3390/e11010059 - 12 Feb 2009

Cited by 4 | Viewed by 6685

Abstract

Since a connection was made in the 19th Century between increase of entropy and earlier expressions of the Second Law of Thermodynamics, the topic has continued to fascinate engineers, physicists, chemists, computer scientists, mathematicians and philosophers. The topic of entropy is very much [...] Read more.

Since a connection was made in the 19th Century between increase of entropy and earlier expressions of the Second Law of Thermodynamics, the topic has continued to fascinate engineers, physicists, chemists, computer scientists, mathematicians and philosophers. The topic of entropy is very much alive, as witnessed by the highly cited proceedings of a lively conference on the subject, held in Dresden Germany in 2000 [1]. Our intention in running a theme program seven years after the Dresden conference was to stimulate connections between entropy theory and broader applications. The papers in this special issue arose from a meeting of the AMSI-MASCOS Theme Program, Concepts of Entropy and their Applications, which took place in Melbourne Australia, November 26- December 12, 2007. [...] Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

Research

Jump to: Editorial

13 pages, 755 KiB

Open AccessArticle

Thermal Contact

by Gary P. Morriss, Tony Chung and Christopher Angstmann

Entropy 2008, 10(4), 786-798; https://doi.org/10.3390/e10040786 - 22 Dec 2008

Cited by 7 | Viewed by 10544

Abstract

The concepts of temperature and entropy as applied in equilibrium thermodynamics do not easily generalize to nonequilibrium systems and there are transient systems where thermodynamics cannot apply. However, it is possible that nonequilibrium steady states may have a thermodynamics description. We explore the [...] Read more.

The concepts of temperature and entropy as applied in equilibrium thermodynamics do not easily generalize to nonequilibrium systems and there are transient systems where thermodynamics cannot apply. However, it is possible that nonequilibrium steady states may have a thermodynamics description. We explore the consequences of a particular microscopic thermostat-reservoir contact needed to both stabilize and measure the temperature of a system. One particular mechanical connection mechanism is considered in detail and a contact resistance is observed in the numerical simulations. We propose a microscopic mechanism to explain this effect for both equilibrium and nonequilibrium systems. These results emphasize the difficulty in identifying a microscopic expression for the thermodynamic temperature. It is evident that the kinetic temperature is not necessarily equal to the thermodynamic temperature, especially when used to define the local temperature. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

11 pages, 471 KiB

Open AccessArticle

Non-linear Information Inequalities

by Terence Chan and Alex Grant

Entropy 2008, 10(4), 765-775; https://doi.org/10.3390/e10040765 - 22 Dec 2008

Cited by 9 | Viewed by 6643

Abstract

We construct non-linear information inequalities from Mat´uˇs’ infinite series of linear information inequalities. Each single non-linear inequality is sufficiently strong to prove that the closure of the set of all entropy functions is not polyhedral for four or more random variables, a fact [...] Read more.

We construct non-linear information inequalities from Mat´uˇs’ infinite series of linear information inequalities. Each single non-linear inequality is sufficiently strong to prove that the closure of the set of all entropy functions is not polyhedral for four or more random variables, a fact that was already established using the series of linear inequalities. To the best of our knowledge, they are the first non-trivial examples of non-linear information inequalities. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

8 pages, 158 KiB

Open AccessArticle

Speed-gradient Entropy Principle for Nonstationary Processes

by Alexander Fradkov

Entropy 2008, 10(4), 757-764; https://doi.org/10.3390/e10040757 - 8 Dec 2008

Cited by 21 | Viewed by 7618

Abstract

The speed-gradient variational principle (SG-principle) for nonstationary far from equilibrium systems is formulated and illustrated by examples. The SG-model of transient (relaxation) dynamics for systems of a finite number of particles based on maximum entropy principle is derived. It has the form dX(t)/dt [...] Read more.

The speed-gradient variational principle (SG-principle) for nonstationary far from equilibrium systems is formulated and illustrated by examples. The SG-model of transient (relaxation) dynamics for systems of a finite number of particles based on maximum entropy principle is derived. It has the form dX(t)/dt = AlnX(t); where X(t) is the vector of the cell populations, A is a symmetric matrix with two zero eigenvalues corresponding to mass and energy conservation laws. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

12 pages, 220 KiB

Open AccessArticle

An Assessment of Hermite Function Based Approximations of Mutual Information Applied to Independent Component Analysis

by Julian Sorensen

Entropy 2008, 10(4), 745-756; https://doi.org/10.3390/e10040745 - 4 Dec 2008

Cited by 5 | Viewed by 10716

Abstract

At the heart of many ICA techniques is a nonparametric estimate of an information measure, usually via nonparametric density estimation, for example, kernel density estimation. While not as popular as kernel density estimators, orthogonal functions can be used for nonparametric density estimation (via [...] Read more.

At the heart of many ICA techniques is a nonparametric estimate of an information measure, usually via nonparametric density estimation, for example, kernel density estimation. While not as popular as kernel density estimators, orthogonal functions can be used for nonparametric density estimation (via a truncated series expansion whose coefficients are calculated from the observed data). While such estimators do not necessarily yield a valid density, which kernel density estimators do, they are faster to calculate than kernel density estimators, in particular for a modified version of Renyi's entropy of order 2. In this paper, we compare the performance of ICA using Hermite series based estimates of Shannon's and Renyi's mutual information, to that of Gaussian kernel based estimates. The comparisons also include ICA using the RADICAL estimate of Shannon's entropy and a FastICA estimate of neg-entropy. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

14 pages, 1471 KiB

Open AccessArticle

Intercept Capacity: Unknown Unitary Transformation

by John Kitchen, Bill Moran and Stephen D. Howard

Entropy 2008, 10(4), 722-735; https://doi.org/10.3390/e10040722 - 20 Nov 2008

Cited by 1 | Viewed by 9211

Abstract

We consider the problem of intercepting communications signals between Multiple-Input Multiple-Output (MIMO) communication systems. To correctly detect a transmitted message it is necessary to know the gain matrix that represents the channel between the transmitter and the receiver. However, even if the receiver [...] Read more.

We consider the problem of intercepting communications signals between Multiple-Input Multiple-Output (MIMO) communication systems. To correctly detect a transmitted message it is necessary to know the gain matrix that represents the channel between the transmitter and the receiver. However, even if the receiver has knowledge of the message symbol set, it may not be possible to estimate the channel matrix. Blind Source Separation (BSS) techniques, such as Independent Component Analysis (ICA) can go some way to extracting independent signals from individual transmission antennae but these may have been preprocessed in a manner unknown to the receiver. In this paper we consider the situation where a communications interception system has prior knowledge of the message symbol set, the channel matrix between the transmission system and the interception system and is able to resolve the transmissionss from independent antennae. The question then becomes: what is the mutual information available to the interceptor when an unknown unitary transformation matrix is employed by the transmitter. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

38 pages, 1752 KiB

Open AccessArticle

Comparison of Statistical Dynamical, Square Root and Ensemble Kalman Filters

by Terence J. O’Kane and Jorgen S. Frederiksen

Entropy 2008, 10(4), 684-721; https://doi.org/10.3390/e10040684 - 20 Nov 2008

Cited by 26 | Viewed by 10442

Abstract

We present a statistical dynamical Kalman filter and compare its performance to deterministic ensemble square root and stochastic ensemble Kalman filters for error covariance modeling with applications to data assimilation. Our studies compare assimilation and error growth in barotropic flows during a period [...] Read more.

We present a statistical dynamical Kalman filter and compare its performance to deterministic ensemble square root and stochastic ensemble Kalman filters for error covariance modeling with applications to data assimilation. Our studies compare assimilation and error growth in barotropic flows during a period in 1979 in which several large scale atmospheric blocking regime transitions occurred in the Northern Hemisphere. We examine the role of sampling error and its effect on estimating the flow dependent growing error structures and the associated effects on the respective Kalman gains. We also introduce a Shannon entropy reduction measure and relate it to the spectra of the Kalman gain. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

49 pages, 6899 KiB

Open AccessArticle

Entropy, Closures and Subgrid Modeling

by Jorgen S. Frederiksen and Terence J. O’Kane

Entropy 2008, 10(4), 635-683; https://doi.org/10.3390/e10040635 - 17 Nov 2008

Cited by 41 | Viewed by 10668

Abstract

Maximum entropy states or statistical mechanical equilibrium solutions have played an important role in the development of a fundamental understanding of turbulence and its role in geophysical flows. In modern general circulation models of the earth’s atmosphere and oceans most parameterizations of the [...] Read more.

Maximum entropy states or statistical mechanical equilibrium solutions have played an important role in the development of a fundamental understanding of turbulence and its role in geophysical flows. In modern general circulation models of the earth’s atmosphere and oceans most parameterizations of the subgrid-scale energy and enstrophy transfers are based on ad hoc methods or ideas developed from equilibrium statistical mechanics or entropy production hypotheses. In this paper we review recent developments in nonequilibrium statistical dynamical closure theory, its application to subgrid-scale modeling of eddy-eddy, eddy-mean field and eddy-topographic interactions and the relationship to minimum enstrophy, maximum entropy and entropy production arguments. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

14 pages, 424 KiB

Open AccessArticle

Relaxed Plasma Equilibria and Entropy-Related Plasma Self-Organization Principles

by Robert L. Dewar, Matthew J. Hole, Mathew McGann, Ruth Mills and Stuart R. Hudson

Entropy 2008, 10(4), 621-634; https://doi.org/10.3390/e10040621 - 14 Nov 2008

Cited by 40 | Viewed by 12109

Abstract

The concept of plasma relaxation as a constrained energy minimization is reviewed. Recent work by the authors on generalizing this approach to partially relaxed threedimensional plasma systems in a way consistent with chaos theory is discussed, with a view to clarifying the thermodynamic [...] Read more.

The concept of plasma relaxation as a constrained energy minimization is reviewed. Recent work by the authors on generalizing this approach to partially relaxed threedimensional plasma systems in a way consistent with chaos theory is discussed, with a view to clarifying the thermodynamic aspects of the variational approach used. Other entropy-related approaches to finding long-time steady states of turbulent or chaotic plasma systems are also briefly reviewed. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

23 pages, 621 KiB

Open AccessArticle

On the Entropy and Letter Frequencies of Powerfree Words

by Uwe Grimm and Manuela Heuer

Entropy 2008, 10(4), 590-612; https://doi.org/10.3390/e10040590 - 12 Nov 2008

Cited by 2 | Viewed by 10258

Abstract

We review the recent progress in the investigation of powerfree words, with particular emphasis on binary cubefree and ternary squarefree words. Besides various bounds on the entropy, we provide bounds on letter frequencies and consider their empirical distribution obtained by an enumeration of [...] Read more.

We review the recent progress in the investigation of powerfree words, with particular emphasis on binary cubefree and ternary squarefree words. Besides various bounds on the entropy, we provide bounds on letter frequencies and consider their empirical distribution obtained by an enumeration of binary cubefree words up to length 80. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Graphical abstract

20 pages, 324 KiB

Open AccessArticle

Assessing the Information Content in Environmental Modelling: A Carbon Cycle Perspective

by Ian G. Enting

Entropy 2008, 10(4), 556-575; https://doi.org/10.3390/e10040556 - 3 Nov 2008

Cited by 7 | Viewed by 6730

Abstract

A model represents the way in which information about the world is captured in a form that can be manipulated for application to new situations. However, quantification of `model error' presents formidable challenges. Various inverse problems in carbon cycle modelling are presented as [...] Read more.

A model represents the way in which information about the world is captured in a form that can be manipulated for application to new situations. However, quantification of `model error' presents formidable challenges. Various inverse problems in carbon cycle modelling are presented as illustrations of the issues. A `maximum-entropy' representation of carbon cycle response is used to explore techniques for non-parametric estimation of carbon cycle uncertainty. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Graphical abstract

14 pages, 180 KiB

Open AccessArticle

Entropy and Uncertainty

by Derek W. Robinson

Entropy 2008, 10(4), 493-506; https://doi.org/10.3390/e10040493 - 16 Oct 2008

Cited by 71 | Viewed by 13263

Abstract

We give a survey of the basic statistical ideas underlying the definition of entropy in information theory and their connections with the entropy in the theory of dynamical systems and in statistical mechanics. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

16 pages, 1434 KiB

Open AccessArticle

Extended Thermodynamics: a Theory of Symmetric Hyperbolic Field Equations

by Ingo Müller

Entropy 2008, 10(4), 477-492; https://doi.org/10.3390/e10040477 - 15 Oct 2008

Cited by 15 | Viewed by 13640

Abstract

Extended thermodynamics is based on a set of equations of balance which are supplemented by local and instantaneous constitutive equations so that the field equations are quasi-linear first order differential equations. If the constitutive functions are subject to the requirements of the entropy [...] Read more.

Extended thermodynamics is based on a set of equations of balance which are supplemented by local and instantaneous constitutive equations so that the field equations are quasi-linear first order differential equations. If the constitutive functions are subject to the requirements of the entropy principle, one may write them in symmetric hyperbolic form by a suitable choice of fields. The kinetic theory of gases, or the moment theories based on the Boltzmann equation provide an explicit example for extended thermodynamics. The theory proves its usefulness and practicality in the successful treatment of light scattering in rarefied gases. This presentation is based upon the book [1] of which the author of this paper is a co-author. For more details about the motivation and exploitation of the basic principles the interested reader is referred to that reference. It would seem that extended thermodynamics is worthy of the attention of mathematicians. It may offer them a non-trivial field of study concerning hyperbolic equations, if ever they get tired of the Burgers equation. Physicists may prefer to appreciate the success of extended thermodynamics in light scattering and to work on the open problems concerning the modification of the Navier-Stokes-Fourier theory in rarefied gases as predicted by extended thermodynamics of 13, 14, and more moments. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

15 pages, 1629 KiB

Open AccessArticle

Entropy and Energy, – a Universal Competition

by Ingo Müller

Entropy 2008, 10(4), 462-476; https://doi.org/10.3390/e10040462 - 15 Oct 2008

Cited by 23 | Viewed by 11612

Abstract

When a body approaches equilibrium, energy tends to a minimum and entropy tends to a maximum. Often, or usually, the two tendencies favour different configurations of the body. Thus energy is deterministic in the sense that it favours fixed positions for the atoms, [...] Read more.

When a body approaches equilibrium, energy tends to a minimum and entropy tends to a maximum. Often, or usually, the two tendencies favour different configurations of the body. Thus energy is deterministic in the sense that it favours fixed positions for the atoms, while entropy randomizes the positions. Both may exert considerable forces in the attempt to reach their objectives. Therefore they have to compromise; indeed, under most circumstances it is the available free energy which achieves a minimum. For low temperatures that free energy is energy itself, while for high temperatures it is determined by entropy. Several examples are provided for the roles of energy and entropy as competitors: – Planetary atmospheres; – osmosis; – phase transitions in gases and liquids and in shape memory alloys, and – chemical reactions, viz. the Haber Bosch synthesis of ammonia and photosynthesis. Some historical remarks are strewn through the text to make the reader appreciate the difficulties encountered by the pioneers in understanding the subtlety of the concept of entropy, and in convincing others of the validity and relevance of their arguments. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

11 pages, 158 KiB

Open AccessArticle

The Second Entropy: A Variational Principle for Time-dependent Systems

by Phil Attard

Entropy 2008, 10(3), 380-390; https://doi.org/10.3390/e10030380 - 25 Sep 2008

Cited by 13 | Viewed by 8933

Abstract

The fundamental optimization principle for non-equilibrium thermodynamics is given. The second entropy is introduced as the quantity that is maximised to determine the optimum state of a non-equilibrium system. In contrast, the principles of maximum or minimum dissipation, which have previously been proposed [...] Read more.

The fundamental optimization principle for non-equilibrium thermodynamics is given. The second entropy is introduced as the quantity that is maximised to determine the optimum state of a non-equilibrium system. In contrast, the principles of maximum or minimum dissipation, which have previously been proposed by Onsager, Prigogine, and others as the variational principle for such systems, are shown to be incapable of fulfilling that rôle. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

15 pages, 201 KiB

Open AccessArticle

Entropy Diagnostics for Fourth Order Partial Differential Equations in Conservation Form

by Phil Broadbridge

Entropy 2008, 10(3), 365-379; https://doi.org/10.3390/e10030365 - 25 Sep 2008

Cited by 14 | Viewed by 10494

Abstract

The entropy evolution behaviour of a partial differential equation (PDE) in conservation form, may be readily discerned from the sign of the local source term of Shannon information density. This can be easily used as a diagnostic tool to predict smoothing and non-smoothing [...] Read more.

The entropy evolution behaviour of a partial differential equation (PDE) in conservation form, may be readily discerned from the sign of the local source term of Shannon information density. This can be easily used as a diagnostic tool to predict smoothing and non-smoothing properties, as well as positivity of solutions with conserved mass. The familiar fourth order diffusion equations arising in applications do not have increasing Shannon entropy. However, we obtain a new class of nonlinear fourth order diffusion equations that do indeed have this property. These equations also exhibit smoothing properties and they maintain positivity. The counter-intuitive behaviour of fourth order diffusion, observed to occur or not occur on an apparently ad hoc basis, can be predicted from an easily calculated entropy production rate. This is uniquely defined only after a technical definition of the irreducible source term of a reaction diffusion equation. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

15 pages, 224 KiB

Open AccessArticle

The Entropy Principle from Continuum Mechanics to Hyperbolic Systems of Balance Laws: The Modern Theory of Extended Thermodynamics

by Tommaso Ruggeri

Entropy 2008, 10(3), 319-333; https://doi.org/10.3390/e10030319 - 24 Sep 2008

Cited by 9 | Viewed by 9155

Abstract

We discuss the different roles of the entropy principle in modern thermodynamics. We start with the approach of rational thermodynamics in which the entropy principle becomes a selection rule for physical constitutive equations. Then we discuss the entropy principle for selecting admissible discontinuous [...] Read more.

We discuss the different roles of the entropy principle in modern thermodynamics. We start with the approach of rational thermodynamics in which the entropy principle becomes a selection rule for physical constitutive equations. Then we discuss the entropy principle for selecting admissible discontinuous weak solutions and to symmetrize general systems of hyperbolic balance laws. A particular attention is given on the local and global well-posedness of the relative Cauchy problem for smooth solutions. Examples are given in the case of extended thermodynamics for rarefied gases and in the case of a multi-temperature mixture of fluids. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

10 pages, 164 KiB

Open AccessArticle

Prudent Self-Avoiding Walks

by John C. Dethridge and Anthony J. Guttmann

Entropy 2008, 10(3), 309-318; https://doi.org/10.3390/e10030309 - 24 Sep 2008

Cited by 7 | Viewed by 7856

Abstract

We have produced extended series for prudent self-avoiding walks on the square lattice. These are subsets of self-avoiding walks. We conjecture the exact growth constant and critical exponent for the walks, and show that the (anisotropic) generating function is almost certainly not differentiably-finite. [...] Read more.

We have produced extended series for prudent self-avoiding walks on the square lattice. These are subsets of self-avoiding walks. We conjecture the exact growth constant and critical exponent for the walks, and show that the (anisotropic) generating function is almost certainly not differentiably-finite. Full article

(This article belongs to the Special Issue Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007

Share This Special Issue

Special Issue Editors

Keywords

Benefits of Publishing in a Special Issue

Related Special Issues

Published Papers (18 papers)

Editorial

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI