http://www.mdpi.com/journal/entropy/special_issues/concepts_entropy/ Submission All papers should be durectly submitted to the Guest Editors. To be published continuously until the deadline and papers will be listed together at the special websites. Both, research articles and review articles are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editors for announcment on this website. Submitted papers should not have been published previously, nor be under consideration for publication elsewhere. All papers are refereed through a peer-review process. A guide for authors, sample copies and other relevant information for submitting papers are available on the Instructions for Authors page. Entropy is an international peer-reviewed quarterly journal published by Molecular Diversity Preservation International. Please visit the Instructions for Authors page before submitting a paper. Papers in this special issue will be published in Open Access format free of charge. http://www.mdpi.com/1099-4300/11/1/59/ Since a connection was made in the 19th Century between increase of entropy and earlier expressions of the Second Law of Thermodynamics, the topic has continued to fascinate engineers, physicists, chemists, computer scientists, mathematicians and philosophers. The topic of entropy is very much alive, as witnessed by the highly cited proceedings of a lively conference on the subject, held in Dresden Germany in 2000 [1]. Our intention in running a theme program seven years after the Dresden conference was to stimulate connections between entropy theory and broader applications. The papers in this special issue arose from a meeting of the AMSI-MASCOS Theme Program, Concepts of Entropy and their Applications, which took place in Melbourne Australia, November 26- December 12, 2007. [...] http://www.mdpi.com/1099-4300/11/1/59/ Thu, 12 Feb 2009 00:00:00 CET Entropy 2009-02-12 11 1 Editorial 59 61 1099-4300 Concepts of Entropy and Their Applications 2009-02-12 doi: 10.3390/e11010059 Phil Broadbridge Anthony J. Guttmann http://www.mdpi.com/1099-4300/10/4/786/ The concepts of temperature and entropy as applied in equilibrium thermodynamics do not easily generalize to nonequilibrium systems and there are transient systems where thermodynamics cannot apply. However, it is possible that nonequilibrium steady states may have a thermodynamics description. We explore the consequences of a particular microscopic thermostat-reservoir contact needed to both stabilize and measure the temperature of a system. One particular mechanical connection mechanism is considered in detail and a contact resistance is observed in the numerical simulations. We propose a microscopic mechanism to explain this effect for both equilibrium and nonequilibrium systems. These results emphasize the difficulty in identifying a microscopic expression for the thermodynamic temperature. It is evident that the kinetic temperature is not necessarily equal to the thermodynamic temperature, especially when used to define the local temperature. http://www.mdpi.com/1099-4300/10/4/786/ Mon, 22 Dec 2008 00:00:00 CET Entropy 2008-12-22 10 4 Article 786 798 1099-4300 Thermal Contact 2008-12-22 doi: 10.3390/e10040786 Gary P. Morriss Tony Chung Christopher Angstmann http://www.mdpi.com/1099-4300/10/4/765/ We construct non-linear information inequalities from Mat´uˇs’ infinite series of linear information inequalities. Each single non-linear inequality is sufficiently strong to prove that the closure of the set of all entropy functions is not polyhedral for four or more random variables, a fact that was already established using the series of linear inequalities. To the best of our knowledge, they are the first non-trivial examples of non-linear information inequalities. http://www.mdpi.com/1099-4300/10/4/765/ Mon, 22 Dec 2008 00:00:00 CET Entropy 2008-12-22 10 4 Article 765 775 1099-4300 Non-linear Information Inequalities 2008-12-22 doi: 10.3390/e10040765 Terence Chan Alex Grant http://www.mdpi.com/1099-4300/10/4/757/ The speed-gradient variational principle (SG-principle) for nonstationary far from equilibrium systems is formulated and illustrated by examples. The SG-model of transient (relaxation) dynamics for systems of a finite number of particles based on maximum entropy principle is derived. It has the form dX(t)/dt = AlnX(t); where X(t) is the vector of the cell populations, A is a symmetric matrix with two zero eigenvalues corresponding to mass and energy conservation laws. http://www.mdpi.com/1099-4300/10/4/757/ Mon, 08 Dec 2008 00:00:00 CET Entropy 2008-12-08 10 4 Article 757 764 1099-4300 Speed-gradient Entropy Principle for Nonstationary Processes 2008-12-08 doi: 10.3390/e10040757 Alexander Fradkov http://www.mdpi.com/1099-4300/10/4/745/ At the heart of many ICA techniques is a nonparametric estimate of an information measure, usually via nonparametric density estimation, for example, kernel density estimation. While not as popular as kernel density estimators, orthogonal functions can be used for nonparametric density estimation (via a truncated series expansion whose coefficients are calculated from the observed data). While such estimators do not necessarily yield a valid density, which kernel density estimators do, they are faster to calculate than kernel density estimators, in particular for a modified version of Renyi's entropy of order 2. In this paper, we compare the performance of ICA using Hermite series based estimates of Shannon's and Renyi's mutual information, to that of Gaussian kernel based estimates. The comparisons also include ICA using the RADICAL estimate of Shannon's entropy and a FastICA estimate of neg-entropy. http://www.mdpi.com/1099-4300/10/4/745/ Thu, 04 Dec 2008 00:00:00 CET Entropy 2008-12-04 10 4 Article 745 756 1099-4300 An Assessment of Hermite Function Based Approximations of Mutual Information Applied to Independent Component Analysis 2008-12-04 doi: 10.3390/e10040745 Julian Sorensen http://www.mdpi.com/1099-4300/10/4/722/ We consider the problem of intercepting communications signals between Multiple-Input Multiple-Output (MIMO) communication systems. To correctly detect a transmitted message it is necessary to know the gain matrix that represents the channel between the transmitter and the receiver. However, even if the receiver has knowledge of the message symbol set, it may not be possible to estimate the channel matrix. Blind Source Separation (BSS) techniques, such as Independent Component Analysis (ICA) can go some way to extracting independent signals from individual transmission antennae but these may have been preprocessed in a manner unknown to the receiver. In this paper we consider the situation where a communications interception system has prior knowledge of the message symbol set, the channel matrix between the transmission system and the interception system and is able to resolve the transmissionss from independent antennae. The question then becomes: what is the mutual information available to the interceptor when an unknown unitary transformation matrix is employed by the transmitter. http://www.mdpi.com/1099-4300/10/4/722/ Thu, 20 Nov 2008 00:00:00 CET Entropy 2008-11-20 10 4 Article 722 735 1099-4300 Intercept Capacity: Unknown Unitary Transformation 2008-11-20 doi: 10.3390/e10040722 John Kitchen Bill Moran Stephen D. Howard http://www.mdpi.com/1099-4300/10/4/684/ We present a statistical dynamical Kalman filter and compare its performance to deterministic ensemble square root and stochastic ensemble Kalman filters for error covariance modeling with applications to data assimilation. Our studies compare assimilation and error growth in barotropic flows during a period in 1979 in which several large scale atmospheric blocking regime transitions occurred in the Northern Hemisphere. We examine the role of sampling error and its effect on estimating the flow dependent growing error structures and the associated effects on the respective Kalman gains. We also introduce a Shannon entropy reduction measure and relate it to the spectra of the Kalman gain. http://www.mdpi.com/1099-4300/10/4/684/ Thu, 20 Nov 2008 00:00:00 CET Entropy 2008-11-20 10 4 Article 684 721 1099-4300 Comparison of Statistical Dynamical, Square Root and Ensemble Kalman Filters 2008-11-20 doi: 10.3390/e10040684 Terence J. O’Kane Jorgen S. Frederiksen http://www.mdpi.com/1099-4300/10/4/635/ Maximum entropy states or statistical mechanical equilibrium solutions have played an important role in the development of a fundamental understanding of turbulence and its role in geophysical flows. In modern general circulation models of the earth’s atmosphere and oceans most parameterizations of the subgrid-scale energy and enstrophy transfers are based on ad hoc methods or ideas developed from equilibrium statistical mechanics or entropy production hypotheses. In this paper we review recent developments in nonequilibrium statistical dynamical closure theory, its application to subgrid-scale modeling of eddy-eddy, eddy-mean field and eddy-topographic interactions and the relationship to minimum enstrophy, maximum entropy and entropy production arguments. http://www.mdpi.com/1099-4300/10/4/635/ Mon, 17 Nov 2008 00:00:00 CET Entropy 2008-11-17 10 4 Article 635 683 1099-4300 Entropy, Closures and Subgrid Modeling 2008-11-17 doi: 10.3390/e10040635 Jorgen S. Frederiksen Terence J. O’Kane http://www.mdpi.com/1099-4300/10/4/621/ The concept of plasma relaxation as a constrained energy minimization is reviewed. Recent work by the authors on generalizing this approach to partially relaxed threedimensional plasma systems in a way consistent with chaos theory is discussed, with a view to clarifying the thermodynamic aspects of the variational approach used. Other entropy-related approaches to finding long-time steady states of turbulent or chaotic plasma systems are also briefly reviewed. http://www.mdpi.com/1099-4300/10/4/621/ Fri, 14 Nov 2008 00:00:00 CET Entropy 2008-11-14 10 4 Article 621 634 1099-4300 Relaxed Plasma Equilibria and Entropy-Related Plasma Self-Organization Principles 2008-11-14 doi: 10.3390/e10040621 Robert L. Dewar Matthew J. Hole Mathew McGann Ruth Mills Stuart R. Hudson http://www.mdpi.com/1099-4300/10/4/590/ We review the recent progress in the investigation of powerfree words, with particular emphasis on binary cubefree and ternary squarefree words. Besides various bounds on the entropy, we provide bounds on letter frequencies and consider their empirical distribution obtained by an enumeration of binary cubefree words up to length 80. http://www.mdpi.com/1099-4300/10/4/590/ Wed, 12 Nov 2008 00:00:00 CET Entropy 2008-11-12 10 4 Article 590 612 1099-4300 On the Entropy and Letter Frequencies of Powerfree Words 2008-11-12 doi: 10.3390/e10040590 Uwe Grimm Manuela Heuer http://www.mdpi.com/1099-4300/10/4/556/ A model represents the way in which information about the world is captured in a form that can be manipulated for application to new situations. However, quantification of `model error' presents formidable challenges. Various inverse problems in carbon cycle modelling are presented as illustrations of the issues. A `maximum-entropy' representation of carbon cycle response is used to explore techniques for non-parametric estimation of carbon cycle uncertainty. http://www.mdpi.com/1099-4300/10/4/556/ Mon, 03 Nov 2008 00:00:00 CET Entropy 2008-11-03 10 4 Article 556 575 1099-4300 Assessing the Information Content in Environmental Modelling: A Carbon Cycle Perspective 2008-11-03 doi: 10.3390/e10040556 Ian G. Enting http://www.mdpi.com/1099-4300/10/4/493/ We give a survey of the basic statistical ideas underlying the definition of entropy in information theory and their connections with the entropy in the theory of dynamical systems and in statistical mechanics. http://www.mdpi.com/1099-4300/10/4/493/ Thu, 16 Oct 2008 00:00:00 CEST Entropy 2008-10-16 10 4 Article 493 506 1099-4300 Entropy and Uncertainty 2008-10-16 doi: 10.3390/e10040493 Derek W. Robinson http://www.mdpi.com/1099-4300/10/4/477/ Extended thermodynamics is based on a set of equations of balance which are supplemented by local and instantaneous constitutive equations so that the field equations are quasi-linear first order differential equations. If the constitutive functions are subject to the requirements of the entropy principle, one may write them in symmetric hyperbolic form by a suitable choice of fields. The kinetic theory of gases, or the moment theories based on the Boltzmann equation provide an explicit example for extended thermodynamics. The theory proves its usefulness and practicality in the successful treatment of light scattering in rarefied gases. This presentation is based upon the book [1] of which the author of this paper is a co-author. For more details about the motivation and exploitation of the basic principles the interested reader is referred to that reference. It would seem that extended thermodynamics is worthy of the attention of mathematicians. It may offer them a non-trivial field of study concerning hyperbolic equations, if ever they get tired of the Burgers equation. Physicists may prefer to appreciate the success of extended thermodynamics in light scattering and to work on the open problems concerning the modification of the Navier-Stokes-Fourier theory in rarefied gases as predicted by extended thermodynamics of 13, 14, and more moments. http://www.mdpi.com/1099-4300/10/4/477/ Wed, 15 Oct 2008 00:00:00 CEST Entropy 2008-10-15 10 4 Article 477 492 1099-4300 Extended Thermodynamics: a Theory of Symmetric Hyperbolic Field Equations 2008-10-15 doi: 10.3390/e10040477 Ingo Müller http://www.mdpi.com/1099-4300/10/4/462/ When a body approaches equilibrium, energy tends to a minimum and entropy tends to a maximum. Often, or usually, the two tendencies favour different configurations of the body. Thus energy is deterministic in the sense that it favours fixed positions for the atoms, while entropy randomizes the positions. Both may exert considerable forces in the attempt to reach their objectives. Therefore they have to compromise; indeed, under most circumstances it is the available free energy which achieves a minimum. For low temperatures that free energy is energy itself, while for high temperatures it is determined by entropy. Several examples are provided for the roles of energy and entropy as competitors: – Planetary atmospheres; – osmosis; – phase transitions in gases and liquids and in shape memory alloys, and – chemical reactions, viz. the Haber Bosch synthesis of ammonia and photosynthesis. Some historical remarks are strewn through the text to make the reader appreciate the difficulties encountered by the pioneers in understanding the subtlety of the concept of entropy, and in convincing others of the validity and relevance of their arguments. http://www.mdpi.com/1099-4300/10/4/462/ Wed, 15 Oct 2008 00:00:00 CEST Entropy 2008-10-15 10 4 Article 462 476 1099-4300 Entropy and Energy, – a Universal Competition 2008-10-15 doi: 10.3390/e10040462 Ingo Müller http://www.mdpi.com/1099-4300/10/3/380/ The fundamental optimization principle for non-equilibrium thermodynamics is given. The second entropy is introduced as the quantity that is maximised to determine the optimum state of a non-equilibrium system. In contrast, the principles of maximum or minimum dissipation, which have previously been proposed by Onsager, Prigogine, and others as the variational principle for such systems, are shown to be incapable of fulfilling that rôle. http://www.mdpi.com/1099-4300/10/3/380/ Thu, 25 Sep 2008 00:00:00 CEST Entropy 2008-09-25 10 3 Article 380 390 1099-4300 The Second Entropy: A Variational Principle for Time-dependent Systems 2008-09-25 doi: 10.3390/e10030380 Phil Attard http://www.mdpi.com/1099-4300/10/3/365/ The entropy evolution behaviour of a partial differential equation (PDE) in conservation form, may be readily discerned from the sign of the local source term of Shannon information density. This can be easily used as a diagnostic tool to predict smoothing and non-smoothing properties, as well as positivity of solutions with conserved mass. The familiar fourth order diffusion equations arising in applications do not have increasing Shannon entropy. However, we obtain a new class of nonlinear fourth order diffusion equations that do indeed have this property. These equations also exhibit smoothing properties and they maintain positivity. The counter-intuitive behaviour of fourth order diffusion, observed to occur or not occur on an apparently ad hoc basis, can be predicted from an easily calculated entropy production rate. This is uniquely defined only after a technical definition of the irreducible source term of a reaction diffusion equation. http://www.mdpi.com/1099-4300/10/3/365/ Thu, 25 Sep 2008 00:00:00 CEST Entropy 2008-09-25 10 3 Article 365 379 1099-4300 Entropy Diagnostics for Fourth Order Partial Differential Equations in Conservation Form 2008-09-25 doi: 10.3390/e10030365 Phil Broadbridge http://www.mdpi.com/1099-4300/10/3/319/ We discuss the different roles of the entropy principle in modern thermodynamics. We start with the approach of rational thermodynamics in which the entropy principle becomes a selection rule for physical constitutive equations. Then we discuss the entropy principle for selecting admissible discontinuous weak solutions and to symmetrize general systems of hyperbolic balance laws. A particular attention is given on the local and global well-posedness of the relative Cauchy problem for smooth solutions. Examples are given in the case of extended thermodynamics for rarefied gases and in the case of a multi-temperature mixture of fluids. http://www.mdpi.com/1099-4300/10/3/319/ Wed, 24 Sep 2008 00:00:00 CEST Entropy 2008-09-24 10 3 Article 319 333 1099-4300 The Entropy Principle from Continuum Mechanics to Hyperbolic Systems of Balance Laws: The Modern Theory of Extended Thermodynamics 2008-09-24 doi: 10.3390/e10030319 Tommaso Ruggeri http://www.mdpi.com/1099-4300/10/3/309/ We have produced extended series for prudent self-avoiding walks on the square lattice. These are subsets of self-avoiding walks. We conjecture the exact growth constant and critical exponent for the walks, and show that the (anisotropic) generating function is almost certainly not differentiably-finite. http://www.mdpi.com/1099-4300/10/3/309/ Wed, 24 Sep 2008 00:00:00 CEST Entropy 2008-09-24 10 3 Article 309 318 1099-4300 Prudent Self-Avoiding Walks 2008-09-24 doi: 10.3390/e10030309 John C. Dethridge Anthony J. Guttmann