We start with a clear distinction between Shannon’s Measure of Information (SMI) and the Thermodynamic Entropy. The first is defined on any probability distribution; and therefore it is a very general concept. On the other hand Entropy is defined on...
The goal of this comment note is to express our considerations about the recent paper by A. Ben Naim (Entropy 2017, 19, 48). We strongly support the distinguishing between the Shannon measure of information and the thermodynamic entropy, suggested in...
We supply corrected proofs of the invariance of completion and the chain rule for the Shannon information measures of arbitrary fields, as stated by Dębowski in 2009. Our corrected proofs rest on a number of auxiliary approximation results for Shanno...
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a p...
In 2015, I wrote a book with the same title as this article. The book’s subtitle is: “What we know and what we do not know.” On the book’s dedication page, I wrote: “This book is dedicated to readers of popular science b...
We present a comprehensive information-theoretic evaluation of three widely used rigid water models (TIP3P, SPC, and SPC/
Axiomatic characterizations of Shannon entropy, Kullback I-divergence, and some generalized information measures are surveyed. Three directions are treated: (A) Characterization of functions of probability distributions suitable as information measur...
Xylella Fastidiosa has been recently detected for the first time in southern Italy, representing a very dangerous phytobacterium capable of inducing severe diseases in many plants. In particular, the disease induced in olive trees is called olive qui...
The spreading of the stationary states of the multidimensional single-particle systems with a central potential is quantified by means of Heisenberg-like measures (radial and logarithmic expectation values) and entropy-like quantities (Fisher, Shanno...
In this study, we try to examine whether the forecast errors obtained by the ANN models affect the breakout of financial crises. Additionally, we try to investigate how much the asymmetric information and forecast errors are reflected on the output v...
Physical roots, exemplifications and consequences of periodic and aperiodic ordering (represented by Fibonacci series) in biological systems are discussed. The physical and biological roots and role of symmetry and asymmetry appearing in biological p...
We study flow of substance in a channel of network which consists of nodes of network and edges which connect these nodes and form ways for motion of substance. The channel can have arbitrary number of arms and each arm can contain arbitrary number o...
In this paper, informational (Shannon) measures of symmetry are introduced and analyzed for patterns built of 1D and 2D shapes. The informational measure of symmetry
Stationary quantum information sources emit sequences of correlated qudits—that is, structured quantum stochastic processes. If an observer performs identical measurements on a qudit sequence, the outcomes are a realization of a classical stoch...
The theory of Shannon entropy was applied to the Choi-Williams time-frequency distribution (CWD) of time series in order to extract entropy information in both time and frequency domains. In this way, four novel indexes were defined: (1) partial inst...
The idea that entropy is associated with the “arrow of time” has its roots in Clausius’s statement on the Second Law: “Entropy of the Universe always increases.” However, the explicit association of the entropy with time...
This article is about the profound misuses, misunderstanding, misinterpretations and misapplications of entropy, the Second Law of Thermodynamics and Information Theory. It is the story of the “Greatest Blunder Ever in the History of Science&rd...
This paper answers Bell’s question: What does quantum information refer to? It is about quantum properties represented by subspaces of the quantum Hilbert space, or their projectors, to which standard (Kolmogorov) probabilities can be assigned by usi...
We characterize mutual information as the unique map on ordered pairs of discrete random variables satisfying a set of axioms similar to those of Faddeev’s characterization of the Shannon entropy. There is a new axiom in our characterization, h...
This paper explores the application of the Value of Information, (VoI), based on the Claude Shannon/Ruslan Stratonovich framework within economic contexts. Unlike previous studies that examine circular settings and strategic interactions, we focus on...
The emergence of the uncertainty principle has celebrated its 90th anniversary recently. For this occasion, the latest experimental results of uncertainty relations quantified in terms of Shannon entropies are presented, concentrating only on outcome...
We consider a sub-class of the f-divergences satisfying a stronger convexity property, which we refer to as strongly convex, or
The Voronoi entropy for random patterns and patterns demonstrating various elements of symmetry was calculated. The symmetric patterns were characterized by the values of the Voronoi entropy being very close to those inherent to random ones. This con...
In a multi-target complex network, the links (Lij) represent the interactions between the drug (di) and the target (tj), characterized by different experimental measures (Ki, Km, IC50, etc.) obtained in pharmacological assays under diverse boundary c...
We generalize the Jensen-Shannon divergence and the Jensen-Shannon diversity index by considering a variational definition with respect to a generic mean, thereby extending the notion of Sibson’s information radius. The variational definition applies...
Fano’s inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano’s inequality is generalized to a broad class of information measures, which contains those of Shannon...
Is information a real entity, or is it only a useful mathematical notion? Shannon’s equation and a consolidated tradition have led many scientists and laymen to consider information as some kind of physical, additive quantity. The very well-kno...
In this work, we first consider the discrete version of Fisher information measure and then propose Jensen–Fisher information, to develop some associated results. Next, we consider Fisher information and Bayes–Fisher information measures for mixing p...
Shannon entropy, also known as information entropy or entropy, measures the uncertainty or randomness of probability distribution. Entropy is measured in bits, quantifying the average amount of information required to identify an event from the distr...
A definition of three-variable cumulative residual entropy is introduced, and then used to obtain expressions for higher order or triple-wise correlation measures, that are based on cumulative residual densities. These information measures are calcul...
Does semantic communication require a semantic information theory parallel to Shannon’s information theory, or can Shannon’s work be generalized for semantic communication? This paper advocates for the latter and introduces a semantic gen...
This article highlights advantages of entropy-based genetic diversity measures, at levels from gene expression to landscapes. Shannon’s entropy-based diversity is the standard for ecological communities. The exponentials of Shannon’s and the related...
The complexity measure for the distribution in space-time of a finite-velocity diffusion process is calculated. Numerical results are presented for the calculation of Fisher’s information, Shannon’s entropy, and the Cramér–Ra...
The quality of an image affects its utility and image quality assessment has been a hot research topic for many years. One widely used measure for image quality assessment is Shannon entropy, which has a well-established information-theoretic basis....
Information-theoretic-based measures have been useful in quantifying network complexity. Here we briefly survey and contrast (algorithmic) information-theoretic methods which have been used to characterize graphs and networks. We illustrate the stren...
Different probabilities of events attract different attention in many scenarios such as anomaly detection and security systems. To characterize the events’ importance from a probabilistic perspective, the message importance measure (MIM) is pro...
A contradictory and paradoxical situation that currently exists in information studies can be improved by the introduction of a new information approach, which is called the general theory of information. The main achievement of the general theory of...
Two families of dependence measures between random variables are introduced. They are based on the Rényi divergence of order
We extend previously proposed measures of complexity, emergence, and self-organization to continuous distributions using differential entropy. Given that the measures were based on Shannon’s information, the novel continuous complexity measures descr...
In employing MaxEnt, a crucial role is assigned to the reciprocity relations that relate the quantifier to be extremized (Shannon’s entropy S), the Lagrange multipliers that arise during the variational process, and the expectation values that consti...
Information distance has become an important tool in a wide variety of applications. Various types of information distance have been made over the years. These information distance measures are different from entropy metric, as the former is based on...
The article analytically summarizes the idea of applying Shannon’s principle of entropy maximization to sets that represent the results of observations of the “input” and “output” entities of the stochastic model for eva...
In ecology and evolution, entropic methods are now used widely and increasingly frequently. Their use can be traced back to Ramon Margalef’s first attempt 70 years ago to use log-series to quantify ecological diversity, including searching for...
The Shannon entropy in an LS-coupled configuration space has been calculated through a transformation from that in a jj-coupled configuration space for a Ni-like isoelectronic sequence. The sudden change of Shannon entropy, information exchange, eige...
Quantifying the information content of remote sensing images is considered to be a fundamental task in quantitative remote sensing. Traditionally, the grayscale entropy designed by Shannon’s information theory cannot capture the spatial structu...
The informational energy of Onicescu is a positive quantity that measures the amount of uncertainty of a random variable. However, contrary to Shannon’s entropy, the informational energy is strictly convex and increases when randomness decrease...
The Information Causality principle was proposed to re-derive the Tsirelson bound, an upper limit on the strength of quantum correlations, and has been suggested as a candidate law of nature. The principle states that the Shannon information about Al...
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome’s plausibility. Information measures based on Shannon&rsqu...
Entropy has been the main tool in the analysis of the concept of information since information theory was conceived in the work of Shannon more than fifty years ago. There were some attempts to find more general measure of information, but their outc...
The nature of dependence between random variables has always been the subject of many statistical problems for over a century. Yet today, there is a great deal of research on this topic, especially focusing on the analysis of nonlinearity. Shannon mu...
of 8