Special Issue "Concepts of Entropy and Their Applications - Papers presented at the Meeting at University of Melbourne, 26 November - 11 December 2007"

Abstract: We have produced extended series for prudent self-avoiding walks on the square lattice. These are subsets of self-avoiding walks. We conjecture the exact growth constant and critical exponent for the walks, and show that the (anisotropic) generating function is almost certainly not differentiably-finite.

Abstract: We discuss the different roles of the entropy principle in modern thermodynamics. We start with the approach of rational thermodynamics in which the entropy principle becomes a selection rule for physical constitutive equations. Then we discuss the entropy principle for selecting admissible discontinuous weak solutions and to symmetrize general systems of hyperbolic balance laws. A particular attention is given on the local and global well-posedness of the relative Cauchy problem for smooth solutions. Examples are given in the case of extended thermodynamics for rarefied gases and in the case of a multi-temperature mixture of fluids.

Abstract: The entropy evolution behaviour of a partial differential equation (PDE) in conservation form, may be readily discerned from the sign of the local source term of Shannon information density. This can be easily used as a diagnostic tool to predict smoothing and non-smoothing properties, as well as positivity of solutions with conserved mass. The familiar fourth order diffusion equations arising in applications do not have increasing Shannon entropy. However, we obtain a new class of nonlinear fourth order diffusion equations that do indeed have this property. These equations also exhibit smoothing properties and they maintain positivity. The counter-intuitive behaviour of fourth order diffusion, observed to occur or not occur on an apparently ad hoc basis, can be predicted from an easily calculated entropy production rate. This is uniquely defined only after a technical definition of the irreducible source term of a reaction diffusion equation.

Abstract: The fundamental optimization principle for non-equilibrium thermodynamics is given. The second entropy is introduced as the quantity that is maximised to determine the optimum state of a non-equilibrium system. In contrast, the principles of maximum or minimum dissipation, which have previously been proposed by Onsager, Prigogine, and others as the variational principle for such systems, are shown to be incapable of fulfilling that rôle.

Abstract: When a body approaches equilibrium, energy tends to a minimum and entropy tends to a maximum. Often, or usually, the two tendencies favour different configurations of the body. Thus energy is deterministic in the sense that it favours fixed positions for the atoms, while entropy randomizes the positions. Both may exert considerable forces in the attempt to reach their objectives. Therefore they have to compromise; indeed, under most circumstances it is the available free energy which achieves a minimum. For low temperatures that free energy is energy itself, while for high temperatures it is determined by entropy. Several examples are provided for the roles of energy and entropy as competitors: – Planetary atmospheres; – osmosis; – phase transitions in gases and liquids and in shape memory alloys, and – chemical reactions, viz. the Haber Bosch synthesis of ammonia and photosynthesis. Some historical remarks are strewn through the text to make the reader appreciate the difficulties encountered by the pioneers in understanding the subtlety of the concept of entropy, and in convincing others of the validity and relevance of their arguments.

Abstract: Extended thermodynamics is based on a set of equations of balance which are supplemented by local and instantaneous constitutive equations so that the field equations are quasi-linear first order differential equations. If the constitutive functions are subject to the requirements of the entropy principle, one may write them in symmetric hyperbolic form by a suitable choice of fields. The kinetic theory of gases, or the moment theories based on the Boltzmann equation provide an explicit example for extended thermodynamics. The theory proves its usefulness and practicality in the successful treatment of light scattering in rarefied gases. This presentation is based upon the book [1] of which the author of this paper is a co-author. For more details about the motivation and exploitation of the basic principles the interested reader is referred to that reference. It would seem that extended thermodynamics is worthy of the attention of mathematicians. It may offer them a non-trivial field of study concerning hyperbolic equations, if ever they get tired of the Burgers equation. Physicists may prefer to appreciate the success of extended thermodynamics in light scattering and to work on the open problems concerning the modification of the Navier-Stokes-Fourier theory in rarefied gases as predicted by extended thermodynamics of 13, 14, and more moments.

Abstract: We give a survey of the basic statistical ideas underlying the definition of entropy in information theory and their connections with the entropy in the theory of dynamical systems and in statistical mechanics.

Abstract: A model represents the way in which information about the world is captured in a form that can be manipulated for application to new situations. However, quantification of `model error' presents formidable challenges. Various inverse problems in carbon cycle modelling are presented as illustrations of the issues. A `maximum-entropy' representation of carbon cycle response is used to explore techniques for non-parametric estimation of carbon cycle uncertainty.

Abstract: We review the recent progress in the investigation of powerfree words, with particular emphasis on binary cubefree and ternary squarefree words. Besides various bounds on the entropy, we provide bounds on letter frequencies and consider their empirical distribution obtained by an enumeration of binary cubefree words up to length 80.

Abstract: The concept of plasma relaxation as a constrained energy minimization is reviewed. Recent work by the authors on generalizing this approach to partially relaxed threedimensional plasma systems in a way consistent with chaos theory is discussed, with a view to clarifying the thermodynamic aspects of the variational approach used. Other entropy-related approaches to finding long-time steady states of turbulent or chaotic plasma systems are also briefly reviewed.

Abstract: Maximum entropy states or statistical mechanical equilibrium solutions have played an important role in the development of a fundamental understanding of turbulence and its role in geophysical flows. In modern general circulation models of the earth’s atmosphere and oceans most parameterizations of the subgrid-scale energy and enstrophy transfers are based on ad hoc methods or ideas developed from equilibrium statistical mechanics or entropy production hypotheses. In this paper we review recent developments in nonequilibrium statistical dynamical closure theory, its application to subgrid-scale modeling of eddy-eddy, eddy-mean field and eddy-topographic interactions and the relationship to minimum enstrophy, maximum entropy and entropy production arguments.

Abstract: We present a statistical dynamical Kalman filter and compare its performance to deterministic ensemble square root and stochastic ensemble Kalman filters for error covariance modeling with applications to data assimilation. Our studies compare assimilation and error growth in barotropic flows during a period in 1979 in which several large scale atmospheric blocking regime transitions occurred in the Northern Hemisphere. We examine the role of sampling error and its effect on estimating the flow dependent growing error structures and the associated effects on the respective Kalman gains. We also introduce a Shannon entropy reduction measure and relate it to the spectra of the Kalman gain.

Abstract: We consider the problem of intercepting communications signals between Multiple-Input Multiple-Output (MIMO) communication systems. To correctly detect a transmitted message it is necessary to know the gain matrix that represents the channel between the transmitter and the receiver. However, even if the receiver has knowledge of the message symbol set, it may not be possible to estimate the channel matrix. Blind Source Separation (BSS) techniques, such as Independent Component Analysis (ICA) can go some way to extracting independent signals from individual transmission antennae but these may have been preprocessed in a manner unknown to the receiver. In this paper we consider the situation where a communications interception system has prior knowledge of the message symbol set, the channel matrix between the transmission system and the interception system and is able to resolve the transmissionss from independent antennae. The question then becomes: what is the mutual information available to the interceptor when an unknown unitary transformation matrix is employed by the transmitter.

Abstract: At the heart of many ICA techniques is a nonparametric estimate of an information measure, usually via nonparametric density estimation, for example, kernel density estimation. While not as popular as kernel density estimators, orthogonal functions can be used for nonparametric density estimation (via a truncated series expansion whose coefficients are calculated from the observed data). While such estimators do not necessarily yield a valid density, which kernel density estimators do, they are faster to calculate than kernel density estimators, in particular for a modified version of Renyi's entropy of order 2. In this paper, we compare the performance of ICA using Hermite series based estimates of Shannon's and Renyi's mutual information, to that of Gaussian kernel based estimates. The comparisons also include ICA using the RADICAL estimate of Shannon's entropy and a FastICA estimate of neg-entropy.

Abstract: The speed-gradient variational principle (SG-principle) for nonstationary far from equilibrium systems is formulated and illustrated by examples. The SG-model of transient (relaxation) dynamics for systems of a finite number of particles based on maximum entropy principle is derived. It has the form dX(t)/dt = AlnX(t); where X(t) is the vector of the cell populations, A is a symmetric matrix with two zero eigenvalues corresponding to mass and energy conservation laws.

Abstract: We construct non-linear information inequalities from Mat´uˇs’ infinite series of linear information inequalities. Each single non-linear inequality is sufficiently strong to prove that the closure of the set of all entropy functions is not polyhedral for four or more random variables, a fact that was already established using the series of linear inequalities. To the best of our knowledge, they are the first non-trivial examples of non-linear information inequalities.

Abstract: The concepts of temperature and entropy as applied in equilibrium thermodynamics do not easily generalize to nonequilibrium systems and there are transient systems where thermodynamics cannot apply. However, it is possible that nonequilibrium steady states may have a thermodynamics description. We explore the consequences of a particular microscopic thermostat-reservoir contact needed to both stabilize and measure the temperature of a system. One particular mechanical connection mechanism is considered in detail and a contact resistance is observed in the numerical simulations. We propose a microscopic mechanism to explain this effect for both equilibrium and nonequilibrium systems. These results emphasize the difficulty in identifying a microscopic expression for the thermodynamic temperature. It is evident that the kinetic temperature is not necessarily equal to the thermodynamic temperature, especially when used to define the local temperature.

Abstract: Since a connection was made in the 19th Century between increase of entropy and earlier expressions of the Second Law of Thermodynamics, the topic has continued to fascinate engineers, physicists, chemists, computer scientists, mathematicians and philosophers. The topic of entropy is very much alive, as witnessed by the highly cited proceedings of a lively conference on the subject, held in Dresden Germany in 2000 [1]. Our intention in running a theme program seven years after the Dresden conference was to stimulate connections between entropy theory and broader applications. The papers in this special issue arose from a meeting of the AMSI-MASCOS Theme Program, Concepts of Entropy and their Applications, which took place in Melbourne Australia, November 26- December 12, 2007. [...]