Entropy

Research

Jump to: Review

42 pages, 4656 KB

Open AccessArticle

Parameter-Independent Feature Ranking with Volume-Integrated Sharma–Mittal Entropy: Kernel-Based Estimation, Theoretical Properties and Empirical Validation

by Nida Oruç Ünal, Muzaffer Göztaş and Doğan Yıldız

Entropy 2026, 28(8), 933; https://doi.org/10.3390/e28080933 - 20 Aug 2026

Viewed by 262

Abstract

Feature selection is a critical step in regression problems where a large number of continuous explanatory variables explain the same target through different dependency structures. Classical filters may remain sensitive to a single form of dependence, a single scale, or a specific discretization [...] Read more.

Feature selection is a critical step in regression problems where a large number of continuous explanatory variables explain the same target through different dependency structures. Classical filters may remain sensitive to a single form of dependence, a single scale, or a specific discretization scheme; generalized entropy measures, on the other hand, typically require the parameters to be fixed at a single point. This study proposes a framework that evaluates the Sharma–Mittal entropy volumetrically across a two-dimensional parameter region rather than for a single parameter pair. For the continuous target and explanatory variables, the marginal, joint, and conditional densities are obtained using a Gaussian kernel density estimation; the conditional entropy and information gain surfaces are integrated across the region Ω = [0.05, 0.95]² in the α-β plane to define three indices: PICSME, which measures the conditional uncertainty volume; PIGSME, which measures the gain volume; and NIGSME, which is the ratio of this gain to the total entropy volume of the target. The method is supported by bandwidth consistency and the renormalization of conditional densities; thus, the issue of negative gain that can occur in the continuous variables is resolved, yielding positive and interpretable scores across all six datasets. It is formally demonstrated that the fact that the three indices produce the same ranking is not an empirical observation but rather the result of a monotonicity relationship valid under a fixed target entropy volume. The method is compared with Pearson and Spearman correlations, the Shannon information gain, mutual information, and random forest variable importance across six regression datasets (Airfoil Self-Noise, AirQualityUCI, BodyFat, Meteorology, Concrete, and WineQualityWhite) that differ in their sample size, dimensions, and application domain. The evaluation is not limited to ranking consistency; the out-of-sample prediction performance is measured using least-squares models on the top-k subsets, with rankings calculated from the training partition. The findings show that NIGSME exhibits a performance comparable to that of built-in filters, outperforms them on the Concrete and Meteorology datasets, and never ranks as the weakest method on any dataset. The results demonstrate that volumetric entropy metrics defined across the entire parameter space provide a feature-ranking tool that is independent of parameter selection for continuous variables. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

13 pages, 355 KB

Open AccessFeature PaperArticle

Occupancy Statistics and Entropy in Bose Systems

by Arnaldo Spalvieri

Entropy 2026, 28(8), 911; https://doi.org/10.3390/e28080911 - 13 Aug 2026

Viewed by 303

Abstract

In this work, we compare three formulations of thermodynamic entropy for a non-interacting bosonic gas: (i) the grand-canonical Bose–Einstein entropy, (ii) the finite-N canonical entropy obtained from the exact partition function (Ziff, Uhlenbeck, and Kac construction), and (iii) the entropy of a multinomial [...] Read more.

In this work, we compare three formulations of thermodynamic entropy for a non-interacting bosonic gas: (i) the grand-canonical Bose–Einstein entropy, (ii) the finite-N canonical entropy obtained from the exact partition function (Ziff, Uhlenbeck, and Kac construction), and (iii) the entropy of a multinomial distribution with Boltzmann categorical probabilities and temperature determined from Clausius’ equation. It is well-known that the grand-canonical Bose–Einstein systematically overestimates entropy of canonical systems in regimes where particle-number fluctuations are significant. The exact canonical entropy correctly enforces the particle-number constraint, but recent experimental results suggest that it also overestimates particle-number fluctuations below the crossover temperature. The multinomial distribution is less common in thermodynamics. It addresses in a mathematically exact way the puzzle of the famous −

\log (N!)

term introduced by Gibbs as a deus ex machina in discussions of thermodynamic entropy. One remarkable consequence is that the entropy of the multinomial distribution overcomes the issue of negative entropy at low temperature that affects the Gibbs and the Sackur–Tetrode entropies at low temperature. The analysis presented in the paper shows that the multinomial distribution, equipped with a categorical distribution calibrated in such a way that the resulting multinomial entropy fits Clausius’ equation, provides accurate approximations to the canonical entropy in the classical regime, while it is smaller than the canonical entropy below the crossover temperature. One feature of the multinomial distribution is that it predicts lower peak variance of the number of particles in the ground state than the canonical distribution. This is in agreement with recent experimental results; hence, this paper identifies the thermodynamically calibrated multinomial distribution as a candidate alternative to the canonical distribution for thermodynamic bosonic entropy in finite systems. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

30 pages, 485 KB

Open AccessFeature PaperArticle

Entropic and Geometric Population–Coherence Complementarity in Finite-Dimensional Quantum States

by José J. Gil

Entropy 2026, 28(8), 877; https://doi.org/10.3390/e28080877 - 4 Aug 2026

Viewed by 308

Abstract

Finite-dimensional density matrices contain two representation-intrinsic sectors after the real part is diagonalized, namely ordered intrinsic populations and antisymmetric imaginary coherences. This article develops exact complementarity identities showing how these sectors determine purity, spectral concentration, and entropy. Populations are described by indices of [...] Read more.

Finite-dimensional density matrices contain two representation-intrinsic sectors after the real part is diagonalized, namely ordered intrinsic populations and antisymmetric imaginary coherences. This article develops exact complementarity identities showing how these sectors determine purity, spectral concentration, and entropy. Populations are described by indices of population asymmetry, while coherences are described by the Youla spectrum of the dimensionless metaspin tensor and by correlation-asymmetry indices. In the aligned class, where Youla two-planes coincide with pairs of intrinsic axes, normalized purity splits into a population hierarchy and pairwise coherence terms weighted by products of intrinsic populations. For arbitrary orientations, the coherence term is expressed as a positive semi-definite bilinear form in population-weighted Plücker coordinates. For fixed populations and pairing, increasing any Youla value sharpens the spectrum by majorization and decreases all Rényi entropies, including the von Neumann limit. For fixed ordered populations, maximum aligned cohesion is obtained by saturating adjacent population pairs. The dimensional transition of the discriminating-component cohesion bound is then interpreted as the change from one to two simultaneously saturating metaspin pairs. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

31 pages, 21575 KB

Open AccessArticle

Structural Entropy, Modal Diversity Entropy, and Accessibility Differentiation: A Study of the Western China–Central Asia Cross-Border Multimodal Transportation Network

by Ruifen Sun, Ying Xin, Yang Shao and Peilun Ju

Entropy 2026, 28(7), 823; https://doi.org/10.3390/e28070823 - 20 Jul 2026

Viewed by 421

Abstract

Cross-border multimodal transportation systems are essential for regional connectivity, yet their structural concentration, modal imbalance, community organization, and accessibility differentiation remain insufficiently understood from an entropy perspective. Based on 2024 data, this study constructs railway, highway, aviation, and integrated transportation networks between Western [...] Read more.

Cross-border multimodal transportation systems are essential for regional connectivity, yet their structural concentration, modal imbalance, community organization, and accessibility differentiation remain insufficiently understood from an entropy perspective. Based on 2024 data, this study constructs railway, highway, aviation, and integrated transportation networks between Western China and the five Central Asian countries. It integrates complex network analysis, structural entropy, modal diversity entropy, Louvain community detection, and accessibility assessment within a structure–organization–function framework. The results reveal a core–periphery pattern, with Xi’an, Urumqi, Almaty, and Tashkent serving as hubs. Structural entropy shows that highway connections are balanced, aviation links are concentrated around core hubs, and the integrated network reflects the coexistence of core-hub agglomeration and multimodal coverage expansion. Modal diversity entropy indicates that high connectivity does not necessarily imply balanced modal configuration, and 41.8% of nodes remain dependent on a single mode. Community detection reveals local cohesion and global segmentation, while accessibility analysis identifies a Western China core and Central Asian periphery. Sensitivity analyses based on common-node normalization, travel-time weighting, and alternative modal weights confirm the robustness of the findings. These results provide an integrated diagnostic framework for identifying structural concentration, modal imbalance, and accessibility inequality in cross-border multimodal transportation networks. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

28 pages, 1179 KB

Open AccessFeature PaperArticle

From Expected Goals to Scoring at Least Once: An Event-Specific Summary of Aggregated Bernoulli Risk

by Tomasz Górecki

Entropy 2026, 28(5), 527; https://doi.org/10.3390/e28050527 - 6 May 2026

Viewed by 851

Abstract

Expected goals (xG) is widely used to quantify offensive performance in football by summarizing the expected number of goals from shot-level scoring probabilities. However, xG reflects only the first moment of the underlying Bernoulli system and does not capture how scoring probability is [...] Read more.

Expected goals (xG) is widely used to quantify offensive performance in football by summarizing the expected number of goals from shot-level scoring probabilities. However, xG reflects only the first moment of the underlying Bernoulli system and does not capture how scoring probability is distributed across shots. As a result, teams with identical total xG may nevertheless have different probabilities of scoring at least once. In this paper, we study the quantity

x G^{+} = - \log P (G = 0)

, which is a monotone transform of the exact no-goal probability and, equivalently, of the probability of scoring at least once. We interpret

x G^{+}

as an additive, event-specific summary of aggregated Bernoulli risk and analyze its main structural properties. In particular, we show that

x G^{+} \geq x G

, with equality only in the degenerate case

p_{i} = 0

for all i, and we derive a second-order approximation linking

x G^{+} - x G

to the second moment of shot probabilities, the effective number of shots, and Rényi-2 entropy. Empirical illustrations on football data show how concentrated shot profiles can increase scoring certainty relative to total xG and how exact Bernoulli aggregation differs from a Poisson approximation based only on the mean. While xG remains an appropriate measure of expected scoring volume,

x G^{+}

provides a complementary summary targeted at the probability of scoring at least once. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

11 pages, 336 KB

Open AccessFeature PaperArticle

Entropy in Exact 2D Navier–Stokes and 3D Burgers Gas Flows

by Philip Broadbridge

Entropy 2026, 28(2), 178; https://doi.org/10.3390/e28020178 - 3 Feb 2026

Viewed by 676

Abstract

Two exact solutions are constructed for viscous compressible gas dynamics in two and three dimensions. The first is a steady vortex, with explicit solutions for the full Navier–Stokes system of velocity, density, temperature and pressure. In contrast, the second is a time-dependent radial [...] Read more.

Two exact solutions are constructed for viscous compressible gas dynamics in two and three dimensions. The first is a steady vortex, with explicit solutions for the full Navier–Stokes system of velocity, density, temperature and pressure. In contrast, the second is a time-dependent radial solution to the 3D vector Burgers’ equation, with a constant injection rate from a spherical interior surface. That solution is shock-like at low Reynolds numbers. In both cases, expressions are given for the local density of entropy production. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

15 pages, 297 KB

Open AccessArticle

Some Results on Cumulative Residual Inaccuracy Measure of k-Record Values

by Ritu Goel, Vikas Kumar, Sarang Vehale and Tony C. Scott

Entropy 2026, 28(1), 17; https://doi.org/10.3390/e28010017 - 24 Dec 2025

Cited by 1 | Viewed by 590

Abstract

Herein, we consider the significance of cumulative residual entropy (CRE) and its numerous generalizations. This article presents an extension of the cumulative residual inaccuracy to k-record values. We examine certain properties of this measure. Additionally, we investigate some stochastic ordering and identify [...] Read more.

Herein, we consider the significance of cumulative residual entropy (CRE) and its numerous generalizations. This article presents an extension of the cumulative residual inaccuracy to k-record values. We examine certain properties of this measure. Additionally, we investigate some stochastic ordering and identify the proposed measure for several distributions that frequently arise in various realistic scenarios and have applications across multiple fields of science and engineering. Full article

(This article belongs to the Special Issue Insight into Entropy)

18 pages, 737 KB

Open AccessArticle

Mutual Information and Quantum Coherence in Minimum Error Discrimination of N Pure Equidistant Quantum States

by Omar Jiménez

Entropy 2025, 27(8), 863; https://doi.org/10.3390/e27080863 - 14 Aug 2025

Viewed by 1577

Abstract

We study the quantum state discrimination problem under the minimum error (ME) strategy for a set of N pure equidistant states. These states are characterized by the property that the inner product between any pair of states is given by a unique complex [...] Read more.

We study the quantum state discrimination problem under the minimum error (ME) strategy for a set of N pure equidistant states. These states are characterized by the property that the inner product between any pair of states is given by a unique complex number S. We provide the explicit form of the states and analyze their main structural properties. The optimal success probability for ME discrimination is evaluated as a function of the number of states, as well as the modulus and phase of the inner product S. Furthermore, we propose an experimental scheme for implementing the ME discrimination of equidistant states. We also investigate the quantum coherence consumed in the implementation of the minimum error discrimination of the equidistant states, which has an established operational interpretation as cryptographic randomness gain. As an application, we propose a quantum communication protocol in which Alice prepares and sends one of the equidistant states, while Bob applies the minimum error discrimination to extract the classical information encoded in the state. Finally, we discuss the optimal conditions under which the protocol achieves an optimal balance of classical correlations and quantum coherence, thereby ensuring effective information transfer and cryptographic security. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

18 pages, 1438 KB

Open AccessFeature PaperArticle

Maximum Entropy Estimates of Hubble Constant from Planck Measurements

by David P. Knobles and Mark F. Westling

Entropy 2025, 27(7), 760; https://doi.org/10.3390/e27070760 - 16 Jul 2025

Cited by 2 | Viewed by 5662

Abstract

A maximum entropy (ME) methodology was used to infer the Hubble constant from the temperature anisotropies in cosmic microwave background (CMB) measurements, as measured by the Planck satellite. A simple cosmological model provided physical insight and afforded robust statistical sampling of a parameter [...] Read more.

A maximum entropy (ME) methodology was used to infer the Hubble constant from the temperature anisotropies in cosmic microwave background (CMB) measurements, as measured by the Planck satellite. A simple cosmological model provided physical insight and afforded robust statistical sampling of a parameter space. The parameter space included the spectral tilt and amplitude of adiabatic density fluctuations of the early universe and the present-day ratios of dark energy, matter, and baryonic matter density. A statistical temperature was estimated by applying the equipartition theorem, which uniquely specifies a posterior probability distribution. The ME analysis inferred the mean value of the Hubble constant to be about 67 km/sec/Mpc with a conservative standard deviation of approximately 4.4 km/sec/Mpc. Unlike standard Bayesian analyses that incorporate specific noise models, the ME approach treats the model error generically, thereby producing broader, but less assumption-dependent, uncertainty bounds. The inferred ME value lies within 1σ of both early-universe estimates (Planck, Dark Energy Signal Instrument (DESI)) and late-universe measurements (e.g., the Chicago Carnegie Hubble Program (CCHP)) using redshift data collected from the James Webb Space Telescope (JWST). Thus, the ME analysis does not appear to support the existence of the Hubble tension. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

17 pages, 10694 KB

Open AccessArticle

Entropy-Inspired Aperture Optimization in Fourier Optics

by Marcos Miotti and Daniel Varela Magalhães

Entropy 2025, 27(7), 730; https://doi.org/10.3390/e27070730 - 7 Jul 2025

Viewed by 1292

Abstract

The trade-off between resolution and contrast is a transcendental problem in optical imaging, spanning from artistic photography to technoscientific applications. To the latter, Fourier-optics-based filters, such as the 4f system, are well-known for their image-enhancement properties, removing high spatial frequencies from an [...] Read more.

The trade-off between resolution and contrast is a transcendental problem in optical imaging, spanning from artistic photography to technoscientific applications. To the latter, Fourier-optics-based filters, such as the 4f system, are well-known for their image-enhancement properties, removing high spatial frequencies from an optically Fourier-transformed light signal through simple aperture adjustment. Nonetheless, assessing the contrast–resolution balance in optical imaging remains a challenging task, often requiring complex mathematical treatment and controlled laboratory conditions to match theoretical predictions. With that in mind, we propose a simple yet robust analytical technique to determine the optimal aperture in a 4f imaging system for static and quasi-static objects. Our technique employs the mathematical formalism of the H-theorem, enabling us to directly access the information of an imaged object. By varying the aperture at the Fourier plane of the 4f system, we have empirically found an optimal aperture region where the imaging entropy is maximum, given that the object is fitted to the imaged area. At that region, the image is lit and well-resolved, and no further aperture decrease improves that, as information of the whole assembly (object plus imaging system) is maximum. With that analysis, we have also been able to investigate how the imperfections in an object affect the entropy during its imaging. Despite its simplicity, our technique is generally applicable and passable for automation, making it interesting for many imaging-based optical devices. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

14 pages, 1997 KB

Open AccessArticle

Shannon Entropy Analysis of a Nuclear Fuel Pin Under Deep Burnup

by Wojciech R. Kubiński, Jan K. Ostrowski and Krzysztof W. Fornalski

Entropy 2024, 26(12), 1124; https://doi.org/10.3390/e26121124 - 22 Dec 2024

Cited by 2 | Viewed by 2158

Abstract

This paper analyzes the behavior of the entropy of a nuclear fuel rod under deep burnup conditions, beyond standard operational ranges, reaching up to 60 years. The evolution of the neutron source distribution in a pressurized water reactor (PWR) fuel pin was analyzed [...] Read more.

This paper analyzes the behavior of the entropy of a nuclear fuel rod under deep burnup conditions, beyond standard operational ranges, reaching up to 60 years. The evolution of the neutron source distribution in a pressurized water reactor (PWR) fuel pin was analyzed using the Monte Carlo method and Shannon information entropy. To maintain proper statistics, a novel scaling method was developed, adjusting the neutron population based on the fission rate. By integrating reactor physics with information theory, this work aimed at the deeper understanding of nuclear fuel behavior under extreme burnup conditions. The results show a “U-shaped” entropy evolution: an initial decrease due to self-organization, followed by stabilization and eventual increase due to degradation. A minimum entropy state is reached after approximately 45 years of pin operation, showing a steady-state condition with no entropy change. This point may indicate a physical limit for fuel utilization. Beyond this point, entropy rises, reflecting system degradation and lower energy efficiency. The results show that entropy analysis can provide valuable insights into fuel behavior and operational limits. The proposed scaling method may also serve to control a Monte Carlo simulation, especially for the analysis of long-life reactors. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

Review

Jump to: Research

13 pages, 1477 KB

Open AccessReview

Translational Entropy-Driven Competitive and Additive Effects on DNA Higher-Order Structure via Ion Exchange Between Cations of Different Valencies

by Takahiro Kenmotsu, Haruto Ogawa, Takashi Nishio and Kenichi Yoshikawa

Entropy 2026, 28(6), 686; https://doi.org/10.3390/e28060686 - 13 Jun 2026

Cited by 1 | Viewed by 764

Abstract

DNA conformational transitions in aqueous environments are strongly influenced by electrostatic interactions with surrounding cations. This review/perspective article summarizes the experimental findings reported during the last decade on the competitive/cooperative effects of cations with different valencies on DNA conformational behavior. Recent experimental studies [...] Read more.

DNA conformational transitions in aqueous environments are strongly influenced by electrostatic interactions with surrounding cations. This review/perspective article summarizes the experimental findings reported during the last decade on the competitive/cooperative effects of cations with different valencies on DNA conformational behavior. Recent experimental studies based on single DNA observations have shown that divalent cations, such as Mg(2+) and Ca(2+), can inhibit DNA compaction induced by the trivalent cation spermidine (SPD(3+)), revealing that the effects of coexisting cations are not simply additive. Such competitive behavior cannot be adequately explained within the conventional Debye–Hückel framework, which predicts always additive electrostatic screening contributions from cations of different valencies. To elucidate the underlying mechanism of competitive effects, a theoretical framework has been proposed by extending the framework of current counterion condensation theory, which incorporates changes in translational entropy arising from the ion-exchange process between monovalent counterions and divalent or trivalent cations interacting with DNA as a highly negatively charged polyelectrolyte. In the theoretical framework, the increase in translational entropy arises from the ion exchange process between monovalent counterions and trivalent cations in the absence of divalent cations, whereas the presence of divalent cations diminishes the entropic gain associated with this exchange. By interpreting the recent experimental findings through the aid of the development of theoretical modeling, this review/perspective article provides a coherent insight on how coexisting multiple cations regulate DNA conformation. Full article

(This article belongs to the Special Issue Insight into Entropy)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Insight into Entropy

Share This Special Issue

Editors

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (12 papers)

Research

Review

Further Information

Guidelines

MDPI Initiatives

Follow MDPI