http://www.mdpi.com/journal/entropy/special_issues/facets_entropy_copenhagen/ {snippet name="submission_info"} http://www.mdpi.com/1099-4300/10/3/261/ Axiomatic characterizations of Shannon entropy, Kullback I-divergence, and some generalized information measures are surveyed. Three directions are treated: (A) Characterization of functions of probability distributions suitable as information measures. (B) Characterization of set functions on the subsets of {1; : : : ;N} representable by joint entropies of components of an N-dimensional random vector. (C) Axiomatic characterization of MaxEnt and related inference rules. The paper concludes with a brief discussion of the relevance of the axiomatic approach for information theory. http://www.mdpi.com/1099-4300/10/3/261/ Fri, 19 Sep 2008 00:00:00 CEST Entropy 2008-09-19 10 3 Article 261 273 1099-4300 Axiomatic Characterizations of Information Measures 2008-09-19 doi: 10.3390/e10030261 Imre Csiszár http://www.mdpi.com/1099-4300/10/3/240/ We explicitly obtain here a novel expression for the semiclassical Wehrl’s entropy using deformed algebras built up with the q¡coherent states (see Arik and Coon [J.Math.Phys. 17, 524 (1976) and Quesne [J. Phys. A 35, 9213 (2002)]). The generalization is investigated with emphasis on i) its behavior as a function of temperature and ii) the results obtained when the deformation-parameter tends to unity. http://www.mdpi.com/1099-4300/10/3/240/ Fri, 19 Sep 2008 00:00:00 CEST Entropy 2008-09-19 10 3 Article 240 247 1099-4300 Deformed Generalization of the Semiclassical Entropy 2008-09-19 doi: 10.3390/e10030240 Gustavo Ferri Fernando Olivares Flavia Pennini Angel Plastino Anel R. Plastino Montserrat Casas http://www.mdpi.com/1099-4300/10/3/160/ A rate equation for a discrete probability distribution is discussed as a route to describe smooth relaxation towards the maximum entropy distribution compatible at all times with one or more linear constraints. The resulting dynamics follows the path of steepest entropy ascent compatible with the constraints. The rate equation is consistent with the Onsager theorem of reciprocity and the fluctuation-dissipation theorem. The mathematical formalism was originally developed to obtain a quantum theoretical unification of mechanics and thermodinamics. It is presented here in a general, non-quantal formulation as a part of an effort to develop tools for the phenomenological treatment of non-equilibrium problems with applications in engineering, biology, sociology, and economics. The rate equation is also extended to include the case of assigned time-dependences of the constraints and the entropy, such as for modeling non-equilibrium energy and entropy exchanges. http://www.mdpi.com/1099-4300/10/3/160/ Thu, 14 Aug 2008 00:00:00 CEST Entropy 2008-08-14 10 3 Article 160 182 1099-4300 Modeling Non-Equilibrium Dynamics of a Discrete Probability Distribution: General Rate Equation for Maximal Entropy Generation in a Maximum-Entropy Landscape with Time-Dependent Constraints 2008-08-14 doi: 10.3390/entropy-e10030010 Gian Paolo Beretta http://www.mdpi.com/1099-4300/10/3/131/ A generalised notion of exponential families is introduced. It is based on the variational principle, borrowed from statistical physics. It is shown that inequivalent generalised entropy functions lead to distinct generalised exponential families. The well-known result that the inequality of Cram´er and Rao becomes an equality in the case of an exponential family can be generalised. However, this requires the introduction of escort probabilities. http://www.mdpi.com/1099-4300/10/3/131/ Wed, 16 Jul 2008 00:00:00 CEST Entropy 2008-07-16 10 3 Article 131 149 1099-4300 Generalised Exponential Families and Associated Entropy Functions 2008-07-16 doi: 10.3390/entropy-e10030131 Jan Naudts http://www.mdpi.com/1099-4300/10/2/124/ We show that, to generate the statistical operator appropriate for a given system, and as an alternative to Jaynes’ MaxEnt approach, that refers to the entropy S, one can use instead the increments ±S in S. To such an effect, one uses the macroscopic thermodynamic relation that links ±S to changes in i) the internal energy E and ii) the remaining M relevant extensive quantities Ai, i = 1; : : : ;M; that characterize the context one is working with. http://www.mdpi.com/1099-4300/10/2/124/ Tue, 24 Jun 2008 00:00:00 CEST Entropy 2008-06-24 10 2 Article 124 130 1099-4300 Incremental Entropy Relation as an Alternative to MaxEnt 2008-06-24 doi: 10.3390/entropy-e10020124 Angelo Plastino Angel R. Plastino Evaldo M. F. Curado Montse Casas http://www.mdpi.com/1099-4300/10/2/71/ Partly motivated by entropy-estimation problems in neuroscience, we present a detailed and extensive comparison between some of the most popular and effective entropy estimation methods used in practice: The plug-in method, four different estimators based on the Lempel-Ziv (LZ) family of data compression algorithms, an estimator based on the Context-Tree Weighting (CTW) method, and the renewal entropy estimator. METHODOLOGY: Three new entropy estimators are introduced; two new LZ-based estimators, and the “renewal entropy estimator,” which is tailored to data generated by a binary renewal process. For two of the four LZ-based estimators, a bootstrap procedure is described for evaluating their standard error, and a practical rule of thumb is heuristically derived for selecting the values of their parameters in practice. THEORY: We prove that, unlike their earlier versions, the two new LZ-based estimators are universally consistent, that is, they converge to the entropy rate for every finite-valued, stationary and ergodic process. An effective method is derived for the accurate approximation of the entropy rate of a finite-state hidden Markov model (HMM) with known distribution. Heuristic calculations are presented and approximate formulas are derived for evaluating the bias and the standard error of each estimator. SIMULATION: All estimators are applied to a wide range of data generated by numerous different processes with varying degrees of dependence and memory. The main conclusions drawn from these experiments include: (i) For all estimators considered, the main source of error is the bias. (ii) The CTW method is repeatedly and consistently seen to provide the most accurate results. (iii) The performance of the LZ-based estimators is often comparable to that of the plug-in method. (iv) The main drawback of the plug-in method is its computational inefficiency; with small word-lengths it fails to detect longer-range structure in the data, and with longer word-lengths the empirical distribution is severely undersampled, leading to large biases. http://www.mdpi.com/1099-4300/10/2/71/ Tue, 17 Jun 2008 00:00:00 CEST Entropy 2008-06-17 10 2 Article 71 99 1099-4300 Estimating the Entropy of Binary Time Series: Methodology, Some Theory and a Simulation Study 2008-06-17 doi: 10.3390/entropy-e10020071 Yun Gao Ioannis Kontoyiannis Elie Bienenstock