E-Mail Alert

Add your e-mail address to receive forthcoming issues of this journal:

Journal Browser

Journal Browser

Special Issue "Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007"

Quicklinks

A special issue of Entropy (ISSN 1099-4300).

Deadline for manuscript submissions: closed (23 May 2008)

Special Issue Editors

Guest Editor
Prof. Dr. Douglas A. Gray

The University of Adelaide, SA 5005
Guest Editor
Prof. Dr. Philip Broadbridge (Website)

Department of Mathematics and Statistics, La Trobe University Melbourne, VIC 3086, Australia
Fax: +1 3 9479 3060
Interests: concepts of entropy and their applications
Guest Editor
Prof. Dr. Tony Guttmann (Website)

The University of Melbourne, Parkville, VIC 3010

Keywords

  • Clausius entropy
  • Gibbs entropy
  • Boltzmann entropy
  • von Neumann entropy
  • Tsallis entropy
  • entropy and information theory
  • black hole entropy

Published Papers (18 papers)

View options order results:
result details:
Displaying articles 1-18
Export citation of selected articles as:

Editorial

Jump to: Research

Open AccessEditorial Concepts of Entropy and Their Applications
Entropy 2009, 11(1), 59-61; doi:10.3390/e11010059
Received: 8 February 2009 / Accepted: 9 February 2009 / Published: 12 February 2009
Cited by 1 | PDF Full-text (111 KB)
Abstract
Since a connection was made in the 19th Century between increase of entropy and earlier expressions of the Second Law of Thermodynamics, the topic has continued to fascinate engineers, physicists, chemists, computer scientists, mathematicians and philosophers. The topic of entropy is very [...] Read more.
Since a connection was made in the 19th Century between increase of entropy and earlier expressions of the Second Law of Thermodynamics, the topic has continued to fascinate engineers, physicists, chemists, computer scientists, mathematicians and philosophers. The topic of entropy is very much alive, as witnessed by the highly cited proceedings of a lively conference on the subject, held in Dresden Germany in 2000 [1]. Our intention in running a theme program seven years after the Dresden conference was to stimulate connections between entropy theory and broader applications. The papers in this special issue arose from a meeting of the AMSI-MASCOS Theme Program, Concepts of Entropy and their Applications, which took place in Melbourne Australia, November 26- December 12, 2007. [...] Full article

Research

Jump to: Editorial

Open AccessArticle Non-linear Information Inequalities
Entropy 2008, 10(4), 765-775; doi:10.3390/e10040765
Received: 24 May 2008 / Accepted: 9 December 2008 / Published: 22 December 2008
Cited by 6 | PDF Full-text (471 KB) | HTML Full-text | XML Full-text
Abstract
We construct non-linear information inequalities from Mat´uˇs’ infinite series of linear information inequalities. Each single non-linear inequality is sufficiently strong to prove that the closure of the set of all entropy functions is not polyhedral for four or more random variables, a [...] Read more.
We construct non-linear information inequalities from Mat´uˇs’ infinite series of linear information inequalities. Each single non-linear inequality is sufficiently strong to prove that the closure of the set of all entropy functions is not polyhedral for four or more random variables, a fact that was already established using the series of linear inequalities. To the best of our knowledge, they are the first non-trivial examples of non-linear information inequalities. Full article
Open AccessArticle Thermal Contact
Entropy 2008, 10(4), 786-798; doi:10.3390/e10040786
Received: 23 May 2008 / Accepted: 11 December 2008 / Published: 22 December 2008
Cited by 4 | PDF Full-text (755 KB) | HTML Full-text | XML Full-text
Abstract
The concepts of temperature and entropy as applied in equilibrium thermodynamics do not easily generalize to nonequilibrium systems and there are transient systems where thermodynamics cannot apply. However, it is possible that nonequilibrium steady states may have a thermodynamics description. We explore [...] Read more.
The concepts of temperature and entropy as applied in equilibrium thermodynamics do not easily generalize to nonequilibrium systems and there are transient systems where thermodynamics cannot apply. However, it is possible that nonequilibrium steady states may have a thermodynamics description. We explore the consequences of a particular microscopic thermostat-reservoir contact needed to both stabilize and measure the temperature of a system. One particular mechanical connection mechanism is considered in detail and a contact resistance is observed in the numerical simulations. We propose a microscopic mechanism to explain this effect for both equilibrium and nonequilibrium systems. These results emphasize the difficulty in identifying a microscopic expression for the thermodynamic temperature. It is evident that the kinetic temperature is not necessarily equal to the thermodynamic temperature, especially when used to define the local temperature. Full article
Open AccessArticle Speed-gradient Entropy Principle for Nonstationary Processes
Entropy 2008, 10(4), 757-764; doi:10.3390/e10040757
Received: 24 May 2008 / Accepted: 1 December 2008 / Published: 8 December 2008
Cited by 7 | PDF Full-text (158 KB)
Abstract
The speed-gradient variational principle (SG-principle) for nonstationary far from equilibrium systems is formulated and illustrated by examples. The SG-model of transient (relaxation) dynamics for systems of a finite number of particles based on maximum entropy principle is derived. It has the form [...] Read more.
The speed-gradient variational principle (SG-principle) for nonstationary far from equilibrium systems is formulated and illustrated by examples. The SG-model of transient (relaxation) dynamics for systems of a finite number of particles based on maximum entropy principle is derived. It has the form dX(t)/dt = AlnX(t); where X(t) is the vector of the cell populations, A is a symmetric matrix with two zero eigenvalues corresponding to mass and energy conservation laws. Full article
Open AccessArticle An Assessment of Hermite Function Based Approximations of Mutual Information Applied to Independent Component Analysis
Entropy 2008, 10(4), 745-756; doi:10.3390/e10040745
Received: 23 May 2008 / Accepted: 28 November 2008 / Published: 4 December 2008
PDF Full-text (220 KB) | HTML Full-text | XML Full-text
Abstract
At the heart of many ICA techniques is a nonparametric estimate of an information measure, usually via nonparametric density estimation, for example, kernel density estimation. While not as popular as kernel density estimators, orthogonal functions can be used for nonparametric density estimation [...] Read more.
At the heart of many ICA techniques is a nonparametric estimate of an information measure, usually via nonparametric density estimation, for example, kernel density estimation. While not as popular as kernel density estimators, orthogonal functions can be used for nonparametric density estimation (via a truncated series expansion whose coefficients are calculated from the observed data). While such estimators do not necessarily yield a valid density, which kernel density estimators do, they are faster to calculate than kernel density estimators, in particular for a modified version of Renyi's entropy of order 2. In this paper, we compare the performance of ICA using Hermite series based estimates of Shannon's and Renyi's mutual information, to that of Gaussian kernel based estimates. The comparisons also include ICA using the RADICAL estimate of Shannon's entropy and a FastICA estimate of neg-entropy. Full article
Open AccessArticle Comparison of Statistical Dynamical, Square Root and Ensemble Kalman Filters
Entropy 2008, 10(4), 684-721; doi:10.3390/e10040684
Received: 30 May 2008 / Accepted: 31 October 2008 / Published: 20 November 2008
Cited by 17 | PDF Full-text (1752 KB) | HTML Full-text | XML Full-text
Abstract
We present a statistical dynamical Kalman filter and compare its performance to deterministic ensemble square root and stochastic ensemble Kalman filters for error covariance modeling with applications to data assimilation. Our studies compare assimilation and error growth in barotropic flows during a [...] Read more.
We present a statistical dynamical Kalman filter and compare its performance to deterministic ensemble square root and stochastic ensemble Kalman filters for error covariance modeling with applications to data assimilation. Our studies compare assimilation and error growth in barotropic flows during a period in 1979 in which several large scale atmospheric blocking regime transitions occurred in the Northern Hemisphere. We examine the role of sampling error and its effect on estimating the flow dependent growing error structures and the associated effects on the respective Kalman gains. We also introduce a Shannon entropy reduction measure and relate it to the spectra of the Kalman gain. Full article
Open AccessArticle Intercept Capacity: Unknown Unitary Transformation
Entropy 2008, 10(4), 722-735; doi:10.3390/e10040722
Received: 22 May 2008 / Accepted: 10 November 2008 / Published: 20 November 2008
PDF Full-text (1471 KB) | HTML Full-text | XML Full-text
Abstract
We consider the problem of intercepting communications signals between Multiple-Input Multiple-Output (MIMO) communication systems. To correctly detect a transmitted message it is necessary to know the gain matrix that represents the channel between the transmitter and the receiver. However, even if the [...] Read more.
We consider the problem of intercepting communications signals between Multiple-Input Multiple-Output (MIMO) communication systems. To correctly detect a transmitted message it is necessary to know the gain matrix that represents the channel between the transmitter and the receiver. However, even if the receiver has knowledge of the message symbol set, it may not be possible to estimate the channel matrix. Blind Source Separation (BSS) techniques, such as Independent Component Analysis (ICA) can go some way to extracting independent signals from individual transmission antennae but these may have been preprocessed in a manner unknown to the receiver. In this paper we consider the situation where a communications interception system has prior knowledge of the message symbol set, the channel matrix between the transmission system and the interception system and is able to resolve the transmissionss from independent antennae. The question then becomes: what is the mutual information available to the interceptor when an unknown unitary transformation matrix is employed by the transmitter. Full article
Open AccessArticle Entropy, Closures and Subgrid Modeling
Entropy 2008, 10(4), 635-683; doi:10.3390/e10040635
Received: 26 May 2008 / Accepted: 19 September 2008 / Published: 17 November 2008
Cited by 27 | PDF Full-text (6899 KB) | HTML Full-text | XML Full-text
Abstract
Maximum entropy states or statistical mechanical equilibrium solutions have played an important role in the development of a fundamental understanding of turbulence and its role in geophysical flows. In modern general circulation models of the earth’s atmosphere and oceans most parameterizations of [...] Read more.
Maximum entropy states or statistical mechanical equilibrium solutions have played an important role in the development of a fundamental understanding of turbulence and its role in geophysical flows. In modern general circulation models of the earth’s atmosphere and oceans most parameterizations of the subgrid-scale energy and enstrophy transfers are based on ad hoc methods or ideas developed from equilibrium statistical mechanics or entropy production hypotheses. In this paper we review recent developments in nonequilibrium statistical dynamical closure theory, its application to subgrid-scale modeling of eddy-eddy, eddy-mean field and eddy-topographic interactions and the relationship to minimum enstrophy, maximum entropy and entropy production arguments. Full article
Open AccessArticle Relaxed Plasma Equilibria and Entropy-Related Plasma Self-Organization Principles
Entropy 2008, 10(4), 621-634; doi:10.3390/e10040621
Received: 3 June 2008 / Accepted: 1 October 2008 / Published: 14 November 2008
Cited by 21 | PDF Full-text (424 KB) | HTML Full-text | XML Full-text
Abstract
The concept of plasma relaxation as a constrained energy minimization is reviewed. Recent work by the authors on generalizing this approach to partially relaxed threedimensional plasma systems in a way consistent with chaos theory is discussed, with a view to clarifying the [...] Read more.
The concept of plasma relaxation as a constrained energy minimization is reviewed. Recent work by the authors on generalizing this approach to partially relaxed threedimensional plasma systems in a way consistent with chaos theory is discussed, with a view to clarifying the thermodynamic aspects of the variational approach used. Other entropy-related approaches to finding long-time steady states of turbulent or chaotic plasma systems are also briefly reviewed. Full article
Open AccessArticle On the Entropy and Letter Frequencies of Powerfree Words
Entropy 2008, 10(4), 590-612; doi:10.3390/e10040590
Received: 24 May 2008 / Accepted: 7 August 2008 / Published: 12 November 2008
Cited by 2 | PDF Full-text (621 KB) | HTML Full-text | XML Full-text
Abstract
We review the recent progress in the investigation of powerfree words, with particular emphasis on binary cubefree and ternary squarefree words. Besides various bounds on the entropy, we provide bounds on letter frequencies and consider their empirical distribution obtained by an enumeration [...] Read more.
We review the recent progress in the investigation of powerfree words, with particular emphasis on binary cubefree and ternary squarefree words. Besides various bounds on the entropy, we provide bounds on letter frequencies and consider their empirical distribution obtained by an enumeration of binary cubefree words up to length 80. Full article
Figures

Open AccessArticle Assessing the Information Content in Environmental Modelling: A Carbon Cycle Perspective
Entropy 2008, 10(4), 556-575; doi:10.3390/e10040556
Received: 5 May 2008 / Revised: 7 October 2008 / Accepted: 20 September 2008 / Published: 3 November 2008
Cited by 4 | PDF Full-text (324 KB) | HTML Full-text | XML Full-text
Abstract
A model represents the way in which information about the world is captured in a form that can be manipulated for application to new situations. However, quantification of `model error' presents formidable challenges. Various inverse problems in carbon cycle modelling are presented [...] Read more.
A model represents the way in which information about the world is captured in a form that can be manipulated for application to new situations. However, quantification of `model error' presents formidable challenges. Various inverse problems in carbon cycle modelling are presented as illustrations of the issues. A `maximum-entropy' representation of carbon cycle response is used to explore techniques for non-parametric estimation of carbon cycle uncertainty. Full article
Figures

Open AccessArticle Entropy and Uncertainty
Entropy 2008, 10(4), 493-506; doi:10.3390/e10040493
Received: 17 June 2008 / Accepted: 6 August 2008 / Published: 16 October 2008
Cited by 11 | PDF Full-text (180 KB) | HTML Full-text | XML Full-text
Abstract We give a survey of the basic statistical ideas underlying the definition of entropy in information theory and their connections with the entropy in the theory of dynamical systems and in statistical mechanics. Full article
Open AccessArticle Entropy and Energy, – a Universal Competition
Entropy 2008, 10(4), 462-476; doi:10.3390/e10040462
Received: 31 July 2008 / Revised: 9 September 2008 / Accepted: 22 September 2008 / Published: 15 October 2008
Cited by 9 | PDF Full-text (1629 KB) | HTML Full-text | XML Full-text
Abstract
When a body approaches equilibrium, energy tends to a minimum and entropy tends to a maximum. Often, or usually, the two tendencies favour different configurations of the body. Thus energy is deterministic in the sense that it favours fixed positions for the [...] Read more.
When a body approaches equilibrium, energy tends to a minimum and entropy tends to a maximum. Often, or usually, the two tendencies favour different configurations of the body. Thus energy is deterministic in the sense that it favours fixed positions for the atoms, while entropy randomizes the positions. Both may exert considerable forces in the attempt to reach their objectives. Therefore they have to compromise; indeed, under most circumstances it is the available free energy which achieves a minimum. For low temperatures that free energy is energy itself, while for high temperatures it is determined by entropy. Several examples are provided for the roles of energy and entropy as competitors: – Planetary atmospheres; – osmosis; – phase transitions in gases and liquids and in shape memory alloys, and – chemical reactions, viz. the Haber Bosch synthesis of ammonia and photosynthesis. Some historical remarks are strewn through the text to make the reader appreciate the difficulties encountered by the pioneers in understanding the subtlety of the concept of entropy, and in convincing others of the validity and relevance of their arguments. Full article
Open AccessArticle Extended Thermodynamics: a Theory of Symmetric Hyperbolic Field Equations
Entropy 2008, 10(4), 477-492; doi:10.3390/e10040477
Received: 31 July 2008 / Revised: 9 September 2008 / Accepted: 21 September 2008 / Published: 15 October 2008
Cited by 3 | PDF Full-text (1434 KB)
Abstract
Extended thermodynamics is based on a set of equations of balance which are supplemented by local and instantaneous constitutive equations so that the field equations are quasi-linear first order differential equations. If the constitutive functions are subject to the requirements of the [...] Read more.
Extended thermodynamics is based on a set of equations of balance which are supplemented by local and instantaneous constitutive equations so that the field equations are quasi-linear first order differential equations. If the constitutive functions are subject to the requirements of the entropy principle, one may write them in symmetric hyperbolic form by a suitable choice of fields. The kinetic theory of gases, or the moment theories based on the Boltzmann equation provide an explicit example for extended thermodynamics. The theory proves its usefulness and practicality in the successful treatment of light scattering in rarefied gases. This presentation is based upon the book [1] of which the author of this paper is a co-author. For more details about the motivation and exploitation of the basic principles the interested reader is referred to that reference. It would seem that extended thermodynamics is worthy of the attention of mathematicians. It may offer them a non-trivial field of study concerning hyperbolic equations, if ever they get tired of the Burgers equation. Physicists may prefer to appreciate the success of extended thermodynamics in light scattering and to work on the open problems concerning the modification of the Navier-Stokes-Fourier theory in rarefied gases as predicted by extended thermodynamics of 13, 14, and more moments. Full article
Open AccessArticle Entropy Diagnostics for Fourth Order Partial Differential Equations in Conservation Form
Entropy 2008, 10(3), 365-379; doi:10.3390/e10030365
Received: 3 May 2008 / Accepted: 21 September 2008 / Published: 25 September 2008
Cited by 7 | PDF Full-text (201 KB)
Abstract
The entropy evolution behaviour of a partial differential equation (PDE) in conservation form, may be readily discerned from the sign of the local source term of Shannon information density. This can be easily used as a diagnostic tool to predict smoothing and [...] Read more.
The entropy evolution behaviour of a partial differential equation (PDE) in conservation form, may be readily discerned from the sign of the local source term of Shannon information density. This can be easily used as a diagnostic tool to predict smoothing and non-smoothing properties, as well as positivity of solutions with conserved mass. The familiar fourth order diffusion equations arising in applications do not have increasing Shannon entropy. However, we obtain a new class of nonlinear fourth order diffusion equations that do indeed have this property. These equations also exhibit smoothing properties and they maintain positivity. The counter-intuitive behaviour of fourth order diffusion, observed to occur or not occur on an apparently ad hoc basis, can be predicted from an easily calculated entropy production rate. This is uniquely defined only after a technical definition of the irreducible source term of a reaction diffusion equation. Full article
Open AccessArticle The Second Entropy: A Variational Principle for Time-dependent Systems
Entropy 2008, 10(3), 380-390; doi:10.3390/e10030380
Received: 3 May 2008 / Accepted: 4 September 2008 / Published: 25 September 2008
Cited by 5 | PDF Full-text (158 KB)
Abstract
The fundamental optimization principle for non-equilibrium thermodynamics is given. The second entropy is introduced as the quantity that is maximised to determine the optimum state of a non-equilibrium system. In contrast, the principles of maximum or minimum dissipation, which have previously been [...] Read more.
The fundamental optimization principle for non-equilibrium thermodynamics is given. The second entropy is introduced as the quantity that is maximised to determine the optimum state of a non-equilibrium system. In contrast, the principles of maximum or minimum dissipation, which have previously been proposed by Onsager, Prigogine, and others as the variational principle for such systems, are shown to be incapable of fulfilling that rôle. Full article
Open AccessArticle Prudent Self-Avoiding Walks
Entropy 2008, 10(3), 309-318; doi:10.3390/e10030309
Received: 17 May 2008 / Accepted: 9 September 2008 / Published: 24 September 2008
Cited by 5 | PDF Full-text (164 KB)
Abstract
We have produced extended series for prudent self-avoiding walks on the square lattice. These are subsets of self-avoiding walks. We conjecture the exact growth constant and critical exponent for the walks, and show that the (anisotropic) generating function is almost certainly not [...] Read more.
We have produced extended series for prudent self-avoiding walks on the square lattice. These are subsets of self-avoiding walks. We conjecture the exact growth constant and critical exponent for the walks, and show that the (anisotropic) generating function is almost certainly not differentiably-finite. Full article
Open AccessArticle The Entropy Principle from Continuum Mechanics to Hyperbolic Systems of Balance Laws: The Modern Theory of Extended Thermodynamics
Entropy 2008, 10(3), 319-333; doi:10.3390/e10030319
Received: 3 May 2008 / Accepted: 4 September 2008 / Published: 24 September 2008
Cited by 4 | PDF Full-text (224 KB)
Abstract
We discuss the different roles of the entropy principle in modern thermodynamics. We start with the approach of rational thermodynamics in which the entropy principle becomes a selection rule for physical constitutive equations. Then we discuss the entropy principle for selecting admissible [...] Read more.
We discuss the different roles of the entropy principle in modern thermodynamics. We start with the approach of rational thermodynamics in which the entropy principle becomes a selection rule for physical constitutive equations. Then we discuss the entropy principle for selecting admissible discontinuous weak solutions and to symmetrize general systems of hyperbolic balance laws. A particular attention is given on the local and global well-posedness of the relative Cauchy problem for smooth solutions. Examples are given in the case of extended thermodynamics for rarefied gases and in the case of a multi-temperature mixture of fluids. Full article

Journal Contact

MDPI AG
Entropy Editorial Office
St. Alban-Anlage 66, 4052 Basel, Switzerland
entropy@mdpi.com
Tel. +41 61 683 77 34
Fax: +41 61 302 89 18
Editorial Board
Contact Details Submit to Entropy
Back to Top