The present age, which can be called the Information Age, has a core technology constituted by bits transported by photons. Both concepts, bit and photon, originated in the past century: the concept of photon was introduced by Planck in 1900 when he...
We start with a clear distinction between Shannon’s Measure of Information (SMI) and the Thermodynamic Entropy. The first is defined on any probability distribution; and therefore it is a very general concept. On the other hand Entropy is defined on...
The influence of shielding on the Shannon information entropy for atomic states in strong coupled plasma is investigated using the perturbation method and the Ritz variational method. The analytic expressions for the Shannon information entropies of...
The goal of this comment note is to express our considerations about the recent paper by A. Ben Naim (Entropy 2017, 19, 48). We strongly support the distinguishing between the Shannon measure of information and the thermodynamic entropy, suggested in...
The quality of an image affects its utility and image quality assessment has been a hot research topic for many years. One widely used measure for image quality assessment is Shannon entropy, which has a well-established information-theoretic basis....
Knowledge of the electronic structures of atomic and molecular systems deepens our understanding of the desired system. In particular, several information-theoretic quantities, such as Shannon entropy, have been applied to quantify the extent of elec...
The degree of informational redundancy is often examined in genetic studies but not yet detailed for taxa conceived as minimally monophyletic groups (microgenus). Evolutionary processes in microgenera were reviewed, detailing critical sets of traits,...
In the present work, we report an investigation on quantum entanglement in the doubly excited 2s2 1Se resonance state of the positronium negative ion by using highly correlated Hylleraas type wave functions, determined by calculation of the density o...
We provide benchmark values for Shannon information entropies in position space for the ground state and ls2s 1Se excited state of helium confined with finite confinement potentials by employing the highly correlated Hylleraas-type wave functions. Fo...
The theory of Shannon entropy was applied to the Choi-Williams time-frequency distribution (CWD) of time series in order to extract entropy information in both time and frequency domains. In this way, four novel indexes were defined: (1) partial inst...
In this survey, we begin with a concise introduction to information theory within Shannon’s framework, focusing on the key concept of Shannon entropy and its related quantities: relative entropy, joint entropy, conditional entropy, and mutual i...
The informational energy of Onicescu is a positive quantity that measures the amount of uncertainty of a random variable. However, contrary to Shannon’s entropy, the informational energy is strictly convex and increases when randomness decrease...
I study a lattice with periodic boundary conditions using a non-local master equation that evolves over time. I investigate different system regimes using classical theories like Fisher information, Shannon entropy, complexity, and the Cramér&...
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a p...
We study the correlation of the ground state of an N-particle Moshinsky model by computing the Shannon entropy in both position and momentum spaces. We have derived the Shannon entropy and mutual information with analytical forms of such an N-particl...
In this paper we recall, extend and compute some information measures for the concomitants of the generalized order statistics (GOS) from the Farlie–Gumbel–Morgenstern (FGM) family. We focus on two types of information measures: some rela...
We give a detailed analysis of the Gibbs-type entropy notion and its dynamical behavior in case of time-dependent continuous probability distributions of varied origins: related to classical and quantum systems. The purpose-dependent usage of conditi...
Quantifying the information content of remote sensing images is considered to be a fundamental task in quantitative remote sensing. Traditionally, the grayscale entropy designed by Shannon’s information theory cannot capture the spatial structu...
The complexity measure for the distribution in space-time of a finite-velocity diffusion process is calculated. Numerical results are presented for the calculation of Fisher’s information, Shannon’s entropy, and the Cramér–Ra...
In conventional textbook thermodynamics, entropy is a quantity that may be calculated by different methods, for example experimentally from heat capacities (following Clausius) or statistically from numbers of microscopic quantum states (following Bo...
Every Blockchain architecture relies upon two major pillars: (a) the hash-based, block-binding mechanism and (b) the consensus-achievement mechanism. While the entropic behavior of (a) has been extensively studied in literature over the past decades,...
In this work, we first consider the discrete version of Fisher information measure and then propose Jensen–Fisher information, to develop some associated results. Next, we consider Fisher information and Bayes–Fisher information measures for mixing p...
In 2015, I wrote a book with the same title as this article. The book’s subtitle is: “What we know and what we do not know.” On the book’s dedication page, I wrote: “This book is dedicated to readers of popular science b...
Novel measures of symbol dominance (dC1 and dC2), symbol diversity (DC1 = N (1 − dC1) and DC2 = N (1 − dC2)), and information entropy (HC1 = log2 DC1 and HC2 = log2 DC2) are derived from Lorenz-consistent statistics that I had previously...
Axiomatic characterizations of Shannon entropy, Kullback I-divergence, and some generalized information measures are surveyed. Three directions are treated: (A) Characterization of functions of probability distributions suitable as information measur...
A set of Fisher information properties are presented in order to draw a parallel with similar properties of Shannon differential entropy. Already known properties are presented together with new ones, which include: (i) a generalization of mutual inf...
Symbolic encoding of information is the foundation of Shannon’s mathematical theory of communication. The concept of the informational efficiency of capital markets is closely related to the issue of information processing by equity market part...
Information theoretical measures are used to design, from first principles, an objective function that can drive a learning machine process to a solution that is robust to perturbations in parameters. Full analytic derivations are given and tested wi...
The Boltzmann–Gibbs and Tsallis entropies are essential concepts in statistical physics, which have found multiple applications in many engineering and science areas. In particular, we focus our interest on their applications to image processing thro...
The most widely used forms of model selection criteria, the Bayesian Information Criterion (BIC) and the Akaike Information Criterion (AIC), are expressed in terms of synthetic indicators of the residual distribution: the variance and the mean-square...
We propose an information-theoretic encryption scheme consisting of a four-dimensional chaotic map driver in combination with a prediction model using an LSTM neural net to generate a keystream, which was limited only after passing a test based on th...
Friston proposed the Minimum Free Energy Principle (FEP) based on the Variational Bayesian (VB) method. This principle emphasizes that the brain and behavior coordinate with the environment, promoting self-organization. However, it has a theoretical...
We investigate quantum information by a theoretical measurement approach of an Aharanov–Bohm (AB) ring with Yukawa interaction in curved space with disclination. We obtained the so-called Shannon entropy through the eigenfunctions of the system...
The nature of coarse graining is intuitively “obvious”, but it is rather difficult to find explicit and calculable models of the coarse graining process (and the resulting entropy flow) discussed in the literature. What we would like to have at hand...
In this paper, we study the connections between generalized mean operators and entropies, where the mean value operators are related to the strictly monotone logical operators of fuzzy theory. Here, we propose a new entropy measure based on the famil...
Information content of a polymeric macromolecule can be calculated in bits, by multiplying the number of building blocks that encompasses the entire length of the macromolecule with the Shannon’s entropy of each building block, which could be determi...
A definition of three-variable cumulative residual entropy is introduced, and then used to obtain expressions for higher order or triple-wise correlation measures, that are based on cumulative residual densities. These information measures are calcul...
In this review we integrate results of long term experimental study on ant “language” and intelligence which were fully based on fundamental ideas of Information Theory, such as the Shannon entropy, the Kolmogorov complexity, and the Shannon’s equati...
We present a comprehensive information-theoretic evaluation of three widely used rigid water models (TIP3P, SPC, and SPC/
In this paper, we study the information lost when a real-valued statistic is used to reduce or summarize sample data from a discrete random variable with a one-dimensional parameter. We compare the probability that a random sample gives a particular...
Certain aspects of the history, derivation, and physical application of the information-theoretic entropy concept are discussed. Pre-dating Shannon, the concept is traced back to Pauli. A derivation from first principles is given, without use of appr...
In ecology and evolution, entropic methods are now used widely and increasingly frequently. Their use can be traced back to Ramon Margalef’s first attempt 70 years ago to use log-series to quantify ecological diversity, including searching for...
The spreading of the stationary states of the multidimensional single-particle systems with a central potential is quantified by means of Heisenberg-like measures (radial and logarithmic expectation values) and entropy-like quantities (Fisher, Shanno...
In this paper, we study the concomitants of dual generalized order statistics (and consequently generalized order statistics) when the parameters
This paper presents an integrated approach to social network analysis that combines graph theory, social network analysis (SNA), and Shannon’s information theory, applied to a real-world Twitter network built around the political hashtag Zandbe...
The main objective of this study is to assess the prediction and success rate based on bivariate frequency ratio (FR), weight of evidence (WoE), Shannon entropy (SE), and information value (IV) models for landslide susceptibility in the sedimentary t...
We propose a framework to convert the protein intrinsic disorder content to structural entropy (H) using Shannon’s information theory (IT). The structural capacity (C), which is the sum of H and structural information (I), is equal to the amino acid...
Analysis of nonlinear quantitative EEG (qEEG) markers describing complexity of signal in relation to severity of Alzheimer’s disease (AD) was the focal point of this study. In this study, 79 patients diagnosed with probable AD were recruited from the...
This work aims to develop approaches to processing and interpreting spectral entropy outcomes in the context of seismic data, as well as to establish a methodological foundation for subsequent integration into practical monitoring solutions. The obje...
This article discusses how entropy/information methods are well-suited to analyzing and forecasting the four processes of innovation, transmission, movement, and adaptation, which are the common basis to ecology and evolution. Macroecologists study a...
of 12