Entropy

34 pages, 2540 KB

Open AccessEditor’s ChoiceArticle

Way More than the Sum of Their Parts: From Statistical to Structural Mixtures

by James P. Crutchfield

Entropy 2026, 28(1), 111; https://doi.org/10.3390/e28010111 - 16 Jan 2026

Viewed by 270

We show that mixtures comprising multicomponent systems typically are much more structurally complex than the sum of their parts; sometimes, infinitely more complex. We contrast this with the more familiar notion of statistical mixtures, demonstrating how statistical mixtures miss key aspects of emergent [...] Read more.

We show that mixtures comprising multicomponent systems typically are much more structurally complex than the sum of their parts; sometimes, infinitely more complex. We contrast this with the more familiar notion of statistical mixtures, demonstrating how statistical mixtures miss key aspects of emergent hierarchical organization. This leads us to identify a new kind of structural complexity inherent in multicomponent systems and to draw out broad consequences for system ergodicity. Full article

(This article belongs to the Section Statistical Physics)

► Show Figures

Figure 1

10 pages, 257 KB

Open AccessEditor’s ChoiceArticle

Kolmogorovian Censorship, Predictive Incompleteness, and the Locality Loophole in Bell Experiments

by Philippe Grangier

Entropy 2026, 28(1), 80; https://doi.org/10.3390/e28010080 - 10 Jan 2026

Viewed by 336

Abstract

We revisit the status of quantum probabilities in light of Kolmogorovian Censorship (KC) and the Contexts, Systems, and Modalities (CSM) framework, and we discuss KC-based ideas with respect to superdeterminism, counterfactuality, and predictive incompleteness. After briefly recalling the technical content of KC and [...] Read more.

We revisit the status of quantum probabilities in light of Kolmogorovian Censorship (KC) and the Contexts, Systems, and Modalities (CSM) framework, and we discuss KC-based ideas with respect to superdeterminism, counterfactuality, and predictive incompleteness. After briefly recalling the technical content of KC and its scope, we show that KC correctly identifies that probabilities are classical within a fixed measurement context but does not by itself remove the conceptual tension that motivates nonlocal or conspiratorial explanations of Bell inequality violations. We argue that predictive incompleteness—the view that the quantum state is operationally incomplete until the measurement context is specified—provides a simple, minimal, and explanatory framework that preserves relativistic locality while matching experimental practice. Finally we clarify logical relations among these positions, highlight the assumptions behind them, and justify the move from Kolmogorov’s to Gleason’s framework for quantum probabilities. Full article

(This article belongs to the Special Issue Quantum Information and Probability: From Foundations to Engineering IV)

19 pages, 335 KB

Open AccessEditor’s ChoiceArticle

Refinements and Generalizations of the Shannon Lower Bound via Extensions of the Kraft Inequality

by Neri Merhav

Entropy 2026, 28(1), 76; https://doi.org/10.3390/e28010076 - 9 Jan 2026

Viewed by 409

Abstract

We derive a few extended versions of the Kraft inequality for lossy compression, which pave the way to the derivation of several refinements and extensions of the well-known Shannon lower bound in a variety of instances of rate-distortion coding. These refinements and extensions [...] Read more.

We derive a few extended versions of the Kraft inequality for lossy compression, which pave the way to the derivation of several refinements and extensions of the well-known Shannon lower bound in a variety of instances of rate-distortion coding. These refinements and extensions include sharper bounds for one-to-one codes and D-semifaithful codes, a Shannon lower bound for distortion measures based on sliding-window functions, and an individual-sequence counterpart of the Shannon lower bound. Full article

(This article belongs to the Special Issue Information Theory and Data Compression)

20 pages, 1577 KB

Open AccessEditor’s ChoiceArticle

Unraveling the Network Signatures of Oncogenicity in Virus–Human Protein–Protein Interactions

by Francesco Zambelli, Vera Pancaldi and Manlio De Domenico

Entropy 2025, 27(12), 1248; https://doi.org/10.3390/e27121248 - 11 Dec 2025

Viewed by 448

Abstract

Background: Climate change, urbanization, and global mobility increase the risk of emerging infectious diseases with pandemic potential. There is a need for rapid methods that can assess their long-term effects on human health. In silico approaches are particularly suited to study processes that [...] Read more.

Background: Climate change, urbanization, and global mobility increase the risk of emerging infectious diseases with pandemic potential. There is a need for rapid methods that can assess their long-term effects on human health. In silico approaches are particularly suited to study processes that may manifest years later, under the assumption that perturbed biomolecular interactions underlie these outcomes. Here we focus on viral oncogenicity—the ability of viruses to increase cancer risk—which accounts for about 15% of global cancer cases. Methods: We characterize viruses through multilayer representations of protein–protein interaction (PPI) networks reconstructed from the human interactome. Statistical analyses of topological features, combined with interpretable machine learning models, are used to distinguish oncogenic from non-oncogenic viruses and to identify proteins with potential central role in these processes. Results: Our analysis reveals clear statistical differences between the network properties of oncogenic and non-oncogenic viruses. Furthermore, the machine learning approach enables classification of virus–host interaction networks and identification of relevant subsets of proteins associated with oncogenesis. Functional enrichment analysis highlights mechanisms related to viral oncogenicity, including chromatin structure and other processes linked to cancer development. Conclusions: This framework enables virus classification and highlights mechanisms underlying viral oncogenicity, providing a foundation for investigating long-term health effects of emerging pathogens. Full article

(This article belongs to the Special Issue Statistical and Physics-Based Interpretation of AI and Machine Learning Methods for Biomedical Data)

► Show Figures

Figure 1

19 pages, 2361 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Detrended Cross-Correlations and Their Random Matrix Limit: An Example from the Cryptocurrency Market

by Stanisław Drożdż, Paweł Jarosz, Jarosław Kwapień, Maria Skupień and Marcin Wątorek

Entropy 2025, 27(12), 1236; https://doi.org/10.3390/e27121236 - 6 Dec 2025

Viewed by 724

Abstract

Correlations in complex systems are often obscured by nonstationarity, long-range memory, and heavy-tailed fluctuations, which limit the usefulness of traditional covariance-based analyses. To address these challenges, we construct scale- and fluctuation-dependent correlation matrices using the multifractal detrended cross-correlation coefficient

ρ_{r}

that selectively [...] Read more.

Correlations in complex systems are often obscured by nonstationarity, long-range memory, and heavy-tailed fluctuations, which limit the usefulness of traditional covariance-based analyses. To address these challenges, we construct scale- and fluctuation-dependent correlation matrices using the multifractal detrended cross-correlation coefficient

ρ_{r}

that selectively emphasizes fluctuations of different amplitudes. We examine the spectral properties of these detrended correlation matrices and compare them to the spectral properties of the matrices calculated in the same way from synthetic Gaussian and q-Gaussian signals. Our results show that detrending, heavy tails, and the fluctuation-order parameter r jointly produce spectra, which substantially depart from the random case even under the absence of cross-correlations in time series. Applying this framework to one-minute returns of 140 major cryptocurrencies from 2021 to 2024 reveals robust collective modes, including a dominant market factor and several sectoral components whose strength depends on the analyzed scale and fluctuation order. After filtering out the market mode, the empirical eigenvalue bulk aligns closely with the limit of random detrended cross-correlations, enabling clear identification of structurally significant outliers. Overall, the study provides a refined spectral baseline for detrended cross-correlations and offers a promising tool for distinguishing genuine interdependencies from noise in complex, nonstationary, heavy-tailed systems. Full article

(This article belongs to the Special Issue Entropy, Econophysics, and Complexity)

► Show Figures

Figure 1

104 pages, 2313 KB

Open AccessEditor’s ChoiceReview

Twist Fields in Many-Body Physics

by Benjamin Doyon

Entropy 2025, 27(12), 1230; https://doi.org/10.3390/e27121230 - 4 Dec 2025

Viewed by 410

Abstract

The notion of twist fields has played a fundamental role in many-body physics. It is used to construct the so-called disorder parameter for the study of phase transitions in the classical Ising model of statistical mechanics, it is involved in the Jordan–Wigner transformation [...] Read more.

The notion of twist fields has played a fundamental role in many-body physics. It is used to construct the so-called disorder parameter for the study of phase transitions in the classical Ising model of statistical mechanics, it is involved in the Jordan–Wigner transformation in quantum chains and bosonisation in quantum field theory, and it is related to measures of entanglement in many-body quantum systems. I provide a pedagogical introduction to the notion of twist field and the concepts at its roots, and review some of its applications, focussing on the 1 + 1 dimension. This includes locality and extensivity, internal symmetries, semi-locality, the standard exponential form and HEGT fields, path-integral defects and Riemann surfaces, topological invariance, and twist families. Additional topics touched upon include renormalisation and form factors in relativistic quantum field theory, tau functions of integrable PDEs, thermodynamic and hydrodynamic principles, and branch-point twist fields for entanglement entropy. One-dimensional quantum systems such as chains (e.g., quantum Heisenberg model) and field theory (e.g., quantum sine-Gordon model) are the main focus, but I also explain how the notion applies to equilibrium statistical mechanics (e.g., classical Ising lattice model), and how some aspects can be adapted to one-dimensional classical dynamical systems (e.g., classical Toda chain). Full article

(This article belongs to the Special Issue Entanglement Entropy in Quantum Field Theory)

► Show Figures

Figure 1

20 pages, 4235 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Geometry-Based Bounds on the Capacity of Peak-Limited and Band-Limited Signals over the Additive White Gaussian Noise Channel at a High SNR

by Michael Peleg and Shlomo Shamai

Entropy 2025, 27(12), 1192; https://doi.org/10.3390/e27121192 - 24 Nov 2025

Viewed by 497

Abstract

We present a new computable geometry-based upper bound on the capacity of peak-power-limited and band-limited signal over the Additive White Gaussian Noise Channel. The peak limit applies at continuous time. The bound is a function of the volume and shape of the transmitted [...] Read more.

We present a new computable geometry-based upper bound on the capacity of peak-power-limited and band-limited signal over the Additive White Gaussian Noise Channel. The peak limit applies at continuous time. The bound is a function of the volume and shape of the transmitted signal set, namely the body, in the space of Nyquist-rate samples, comprising all of the points the transmitted signal can reach. At a high SNR, the bound is tight, better than previously known upper bounds and, together with a known lower bound, provides the capacity at an asymptotically high SNR. We found, using a numerical evaluation, the high-SNR capacity of signals with the structure used in Cyclic Prefix assisted Frequency Domain Equalization (CP-FDE) and OFDM for sequence length of up to 100 Nyquist intervals, and we present a conjecture that this result is correct for any sequence length and does not depend on the CPA-FDE structure. This paper extends the methodology developed in previous works. The penalty in power efficiency at a high SNR due to the peak power constraint relative to an average power constraint is about 7.5 dB in the low-pass case and about 5.4 dB in the band-pass case. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

22 pages, 3086 KB

Open AccessEditor’s ChoiceArticle

Nonclassicality and Coherent Error Detection via Pseudo-Entropy

by Assaf Katz, Shalom Bloch and Eliahu Cohen

Entropy 2025, 27(11), 1165; https://doi.org/10.3390/e27111165 - 17 Nov 2025

Viewed by 671

Abstract

Pseudo-entropy is a complex-valued generalization of entanglement entropy defined on non-Hermitian transition operators and induced by post-selection. We present a simulation-based protocol for detecting nonclassicality and coherent errors in quantum circuits using this pseudo-entropy measure

\overset{ˇ}{S}

, focusing on its imaginary part [...] Read more.

Pseudo-entropy is a complex-valued generalization of entanglement entropy defined on non-Hermitian transition operators and induced by post-selection. We present a simulation-based protocol for detecting nonclassicality and coherent errors in quantum circuits using this pseudo-entropy measure

\overset{ˇ}{S}

, focusing on its imaginary part

ℑ \overset{ˇ}{S}

as a diagnostic tool. Our method enables resource-efficient classification of phase-coherent errors, such as those from miscalibrated CNOT gates, even under realistic noise conditions. By quantifying the transition between classical-like and quantum-like behavior through threshold analysis, we provide theoretical benchmarks for error classification that can inform hardware calibration strategies. Numerical simulations demonstrate that 55% of the parameter space remains classified as classical-like (below classification thresholds) at hardware-calibrated sensitivity levels, with statistical significance confirmed through rigorous sensitivity analysis. Robustness to noise and comparison with standard entropy-based methods are demonstrated in a simulation. While hardware validation remains necessary, this work bridges theoretical concepts of nonclassicality with practical quantum error classification frameworks, providing a foundation for experimental quantum computing applications. Full article

(This article belongs to the Special Issue Classical and Quantum Networks: Theory, Modeling and Optimization, 2nd Edition)

► Show Figures

Figure 1

19 pages, 321 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Entropy Production and Irreversibility in the Linearized Stochastic Amari Neural Model

by Dario Lucente, Giacomo Gradenigo and Luca Salasnich

Entropy 2025, 27(11), 1104; https://doi.org/10.3390/e27111104 - 25 Oct 2025

Viewed by 1243

Abstract

One among the most intriguing results coming from the application of statistical mechanics to the study of the brain is the understanding that it, as a dynamical system, is inherently out of equilibrium. In the realm of non-equilibrium statistical mechanics and stochastic processes, [...] Read more.

One among the most intriguing results coming from the application of statistical mechanics to the study of the brain is the understanding that it, as a dynamical system, is inherently out of equilibrium. In the realm of non-equilibrium statistical mechanics and stochastic processes, the standard observable computed to determine whether a system is at equilibrium or not is the entropy produced along the dynamics. For this reason, we present here a detailed calculation of the entropy production in the Amari model, a coarse-grained model of the brain neural network, consisting of an integro-differential equation for the neural activity field, when stochasticity is added to the original dynamics. Since the way to add stochasticity is always to some extent arbitrary, particularly for coarse-grained models, there is no general prescription to do so. We precisely investigate the interplay between noise properties and the original model features, discussing in which cases the stationary state is in thermal equilibrium and which cases it is out of equilibrium, providing explicit and simple formulae. Following the derivation for the particular case considered, we also show how the entropy production rate is related to the variation in time of the Shannon entropy of the system. Full article

(This article belongs to the Section Non-equilibrium Phenomena)

16 pages, 1206 KB

Open AccessEditor’s ChoiceArticle

Contrast Analysis on Spin Transport of Multi-Periodic Exotic States in the XXZ Chain

by Shixian Jiang, Jianpeng Liu and Yongqiang Li

Entropy 2025, 27(10), 1070; https://doi.org/10.3390/e27101070 - 15 Oct 2025

Viewed by 741

Abstract

Quantum spin transport in integrable systems reveals a rich nonequilibrium phenomena that challenges the conventional hydrodynamic framework. Recent advances in ultracold atom experiments with state preparation and single-site addressing have enabled the understanding of this anomalous behavior. Particularly, the full universality characterization of [...] Read more.

Quantum spin transport in integrable systems reveals a rich nonequilibrium phenomena that challenges the conventional hydrodynamic framework. Recent advances in ultracold atom experiments with state preparation and single-site addressing have enabled the understanding of this anomalous behavior. Particularly, the full universality characterization of exotic initial states, as well as their measurement representation, remain unknown. By employing tensor network and contrast methods, we systematically investigate spin transport in the quantum XXZ spin chain and extract dynamical scaling exponents emerging from two paradigmatic and experimentally attainable initial states, i.e., multi-periodic domain-wall (MPDW) and spin-helix (SH) states. Our results using different values of anisotropic parameters

Δ \in [0, 1.2]

demonstrate the evident impeded transport and the difference between the two states with increasing

Δ

values. Large-scale and consistent simulations confirm the contrast method as a viable scaling extraction approach for exotic states with periodicity within experimentally accessible timescales. Our work establishes a foundation for studying initial memory and the corresponding relations of emergent transport behavior in nonequilibrium quantum systems, opening avenues for the identification of their unique universality classes. Full article

(This article belongs to the Special Issue Emergent Phenomena in Quantum Many-Body Systems)

► Show Figures

Figure 1

11 pages, 275 KB

Open AccessEditor’s ChoiceArticle

Relativistic Limits on the Discretization and Temporal Resolution of a Quantum Clock

by Tommaso Favalli

Entropy 2025, 27(10), 1068; https://doi.org/10.3390/e27101068 - 14 Oct 2025

Viewed by 527

Abstract

We provide a brief discussion regarding relativistic limits on the discretization and temporal resolution of time values in a quantum clock. Our clock is characterized by a time observable chosen to be the complement of a bounded and discrete Hamiltonian that can have [...] Read more.

We provide a brief discussion regarding relativistic limits on the discretization and temporal resolution of time values in a quantum clock. Our clock is characterized by a time observable chosen to be the complement of a bounded and discrete Hamiltonian that can have an equally spaced or a generic spectrum. In the first case, the time observable can be described by a Hermitian operator, and we find a limit in the discretization for the time eigenvalues. Nevertheless, in both cases, the time observable can be described by a POVM, and, by increasing the number of time states, we show how the bound on the minimum time quantum can be reduced and identify the conditions under which the clock values can be treated as continuous. Finally, we find a limit for the temporal resolution of our time observable when the clock is used (together with light signals) in a relativistic framework for the measurement of spacetime distances. Full article

(This article belongs to the Special Issue Time in Quantum Mechanics)

11 pages, 285 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Local Invariance of Divergence-Based Quantum Information Measures

by Christopher Popp, Tobias C. Sutter and Beatrix C. Hiesmayr

Entropy 2025, 27(10), 1051; https://doi.org/10.3390/e27101051 - 10 Oct 2025

Viewed by 684

Abstract

Quantum information quantities, such as mutual information and entropies, are essential for characterizing quantum systems and protocols in quantum information science. In this contribution, we identify types of information measures based on generalized divergences and prove their invariance under local isometric or unitary [...] Read more.

Quantum information quantities, such as mutual information and entropies, are essential for characterizing quantum systems and protocols in quantum information science. In this contribution, we identify types of information measures based on generalized divergences and prove their invariance under local isometric or unitary transformations. Leveraging the reversal channel for local isometries together with the data-processing inequality, we establish invariance for information quantities used in both asymptotic and one-shot regimes without relying on the specific functional form of the underlying divergence. These invariances can be applied to improve the computation of such information quantities or optimize protocols and their output states, whose performance is determined by some invariant measure. Our results improve the capability to characterize and compute many operationally relevant information measures with application across the field of quantum information processing. Full article

(This article belongs to the Section Quantum Information)

► Show Figures

Figure 1

12 pages, 1262 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Ordinal Spectrum: Mapping Ordinal Patterns into Frequency Domain

by Mario Chavez and Johann H. Martínez

Entropy 2025, 27(10), 1027; https://doi.org/10.3390/e27101027 - 30 Sep 2025

Viewed by 902

Abstract

Classical spectral analysis characterizes linear systems effectively but often fails to reveal the nonlinear temporal structure of chaotic dynamics. We introduce the ordinal spectrum, a frequency-domain characterization derived from the ordinal-pattern representation of a time series. Applied to both synthetic and real-world [...] Read more.

Classical spectral analysis characterizes linear systems effectively but often fails to reveal the nonlinear temporal structure of chaotic dynamics. We introduce the ordinal spectrum, a frequency-domain characterization derived from the ordinal-pattern representation of a time series. Applied to both synthetic and real-world datasets—including periodic, stochastic, and chaotic signals from physical, biological, and astronomical sources—the ordinal spectrum identifies the temporal scales implied in a possible chaotic behavior. By providing an interpretable, data-driven view of symbolic dynamics in the frequency domain, this approach complements state–space reconstructions and enhances the detection of nonlinear temporal organization that classical spectra may obscure. Its ability to distinguish between qualitatively different dynamics make it a useful tool for exploring complex time series across diverse scientific domains. Full article

(This article belongs to the Special Issue Ordinal Patterns-Based Tools and Their Applications)

► Show Figures

Figure 1

33 pages, 3062 KB

Open AccessEditor’s ChoiceArticle

Gradient-Free De Novo Learning

by Karl Friston, Thomas Parr, Conor Heins, Lancelot Da Costa, Tommaso Salvatori, Alexander Tschantz, Magnus Koudahl, Toon Van de Maele, Christopher Buckley and Tim Verbelen

Entropy 2025, 27(9), 992; https://doi.org/10.3390/e27090992 - 22 Sep 2025

Cited by 1 | Viewed by 2500

Abstract

This technical note applies active inference to the problem of learning goal-directed behaviour from scratch, namely, de novo learning. By de novo learning, we mean discovering, directly from observations, the structure and parameters of a discrete generative model for sequential policy optimisation. Concretely, [...] Read more.

This technical note applies active inference to the problem of learning goal-directed behaviour from scratch, namely, de novo learning. By de novo learning, we mean discovering, directly from observations, the structure and parameters of a discrete generative model for sequential policy optimisation. Concretely, our procedure grows and then reduces a model until it discovers a pullback attractor over (generalised) states; this attracting set supplies paths of least action among goal states while avoiding costly states. The implicit efficiency rests upon reframing the learning problem through the lens of the free energy principle, under which it is sufficient to learn a generative model whose dynamics feature such an attracting set. For context, we briefly relate this perspective to value-based formulations (e.g., Bellman optimality) and then apply the active inference formulation to a small arcade game to illustrate de novo structure learning and ensuing agency. Full article

(This article belongs to the Special Issue Active Inference in Cognitive Neuroscience)

► Show Figures

Figure 1

15 pages, 17666 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Multi-Dimensional Quantum-like Resources from Complex Synchronized Networks

by Debadrita Saha and Gregory D. Scholes

Entropy 2025, 27(9), 963; https://doi.org/10.3390/e27090963 - 16 Sep 2025

Viewed by 725

Abstract

Recent publications have introduced the concept of quantum-like (QL) bits, along with their associated QL states and QL gate operations, which emerge from the dynamics of complex, synchronized networks. The present work extends these ideas to multi-level QL resources, referred to as QL [...] Read more.

Recent publications have introduced the concept of quantum-like (QL) bits, along with their associated QL states and QL gate operations, which emerge from the dynamics of complex, synchronized networks. The present work extends these ideas to multi-level QL resources, referred to as QL dits, as higher-dimensional analogs of QL bits. We employ systems of k-regular graphs to construct QL-dits for arbitrary dimensions, where the emergent eigenspectrum of their adjacency matrices defines the QL-state space. The tensor product structure of multi-QL dit systems is realized through the Cartesian product of graphs. Furthermore, we examine the potential computational advantages of employing d-nary QL systems over two-level QL bit systems, particularly in terms of classical resource efficiency. Overall, this study generalizes the paradigm of using synchronized network dynamics for QL information processing to include higher-dimensional QL resources. Full article

(This article belongs to the Special Issue Quantum Information and Probability: From Foundations to Engineering IV)

► Show Figures

Figure 1

24 pages, 756 KB

Open AccessEditor’s ChoiceArticle

Complex Time Approach to the Hamiltonian and the Entropy Production of the Damped Harmonic Oscillator

by Kyriaki-Evangelia Aslani

Entropy 2025, 27(8), 883; https://doi.org/10.3390/e27080883 - 21 Aug 2025

Cited by 1 | Viewed by 1789

Abstract

The present work applies and extends the previously developed Quantitative Geometrical Thermodynamics (QGT) formalism to the derivation of a Hamiltonian for the damped harmonic oscillator (DHO) across all damping regimes. By introducing complex time, with the real part encoding entropy production and the [...] Read more.

The present work applies and extends the previously developed Quantitative Geometrical Thermodynamics (QGT) formalism to the derivation of a Hamiltonian for the damped harmonic oscillator (DHO) across all damping regimes. By introducing complex time, with the real part encoding entropy production and the imaginary part governing reversible dynamics, QGT provides a unified geometric framework for irreversible thermodynamics, showing that the DHO Hamiltonian can be obtained directly from the (complex) entropy production in a simple exponential form that is generalized across all damping regimes. The derived Hamiltonian preserves a modified Poisson bracket structure and embeds thermodynamic irreversibility into the system’s evolution. Moreover, the resulting expression coincides in form with the well-known Caldirola–Kanai Hamiltonian, despite arising from fundamentally different principles, reinforcing the validity of the QGT approach. The results are also compared with the GENERIC framework, showing that QGT offers an elegant alternative to existing approaches that maintains consistency with symplectic geometry. Furthermore, the imaginary time component is interpreted as isomorphic to the antisymmetric Poisson matrix through the lens of geometric algebra. The formalism opens promising avenues for extending Hamiltonian mechanics to dissipative systems, with potential applications in nonlinear dynamics, quantum thermodynamics, and spacetime algebra. Full article

(This article belongs to the Special Issue Geometry in Thermodynamics, 4th Edition)

► Show Figures

Figure 1

23 pages, 8311 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Active Inference with Dynamic Planning and Information Gain in Continuous Space by Inferring Low-Dimensional Latent States

by Takazumi Matsumoto, Kentaro Fujii, Shingo Murata and Jun Tani

Entropy 2025, 27(8), 846; https://doi.org/10.3390/e27080846 - 9 Aug 2025

Viewed by 2674

Abstract

Active inference offers a unified framework in which agents can exhibit both goal-directed and epistemic behaviors. However, implementing policy search in high-dimensional continuous action spaces presents challenges in terms of scalability and stability. Our previously proposed model, T-GLean, addressed this issue by enabling [...] Read more.

Active inference offers a unified framework in which agents can exhibit both goal-directed and epistemic behaviors. However, implementing policy search in high-dimensional continuous action spaces presents challenges in terms of scalability and stability. Our previously proposed model, T-GLean, addressed this issue by enabling efficient goal-directed planning through low-dimensional latent space search, further reduced by conditioning on prior habituated behavior. However, the lack of an epistemic term in minimizing expected free energy limited the agent’s ability to engage in information-seeking behavior that can be critical for attaining preferred outcomes. In this study, we present EFE-GLean, an extended version of T-GLean that overcomes this limitation by integrating epistemic value into the planning process. EFE-GLean generates goal-directed policies by inferring low-dimensional future posterior trajectories while maximizing expected information gain. Simulation experiments using an extended T-maze task—implemented in both discrete and continuous domains—demonstrate that the agent can successfully achieve its goals by exploiting hidden environmental information. Furthermore, we show that the agent is capable of adapting to abrupt environmental changes by dynamically revising plans through simultaneous minimization of past variational free energy and future expected free energy. Finally, analytical evaluations detail the underlying mechanisms and computational properties of the model. Full article

(This article belongs to the Special Issue Active Inference in Cognitive Neuroscience)

► Show Figures

Figure 1

24 pages, 9086 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Linking Optimization Success and Stability of Finite-Time Thermodynamics Heat Engines

by Julian Gonzalez-Ayala, David Pérez-Gallego, Alejandro Medina, José M. Mateos Roco, Antonio Calvo Hernández, Santiago Velasco and Fernando Angulo-Brown

Entropy 2025, 27(8), 822; https://doi.org/10.3390/e27080822 - 2 Aug 2025

Viewed by 895

Abstract

In celebration of 50 years of the endoreversible Carnot-like heat engine, this work aims to link the thermodynamic success of the irreversible Carnot-like heat engine with the stability dynamics of the engine. This region of success is defined by two extreme configurations in [...] Read more.

In celebration of 50 years of the endoreversible Carnot-like heat engine, this work aims to link the thermodynamic success of the irreversible Carnot-like heat engine with the stability dynamics of the engine. This region of success is defined by two extreme configurations in the interaction between heat reservoirs and the working fluid. The first corresponds to a fully reversible limit, and the second one is the fully dissipative limit; in between both limits, the heat exchange between reservoirs and working fluid produces irreversibilities and entropy generation. The distance between these two extremal configurations is minimized, independently of the chosen metric, in the state where the efficiency is half the Carnot efficiency. This boundary encloses the region where irreversibilities dominate or the reversible behavior dominates (region of success). A general stability dynamics is proposed based on the endoreversible nature of the model and the operation parameter in charge of defining the operation regime. For this purpose, the maximum ecological and maximum Omega regimes are considered. The results show that for single perturbations, the dynamics rapidly directs the system towards the success region, and under random perturbations producing stochastic trajectories, the system remains always in this region. The results are contrasted with the case in which no restitution dynamics exist. It is shown that stability allows the system to depart from the original steady state to other states that enhance the system’s performance, which could favor the evolution and specialization of systems in nature and in artificial devices. Full article

(This article belongs to the Special Issue The First Half Century of Finite-Time Thermodynamics)

► Show Figures

Figure 1

25 pages, 8472 KB

Open AccessEditor’s ChoiceArticle

Harnessing the Power of Pre-Trained Models for Efficient Semantic Communication of Text and Images

by Emrecan Kutay and Aylin Yener

Entropy 2025, 27(8), 813; https://doi.org/10.3390/e27080813 - 29 Jul 2025

Cited by 1 | Viewed by 1736

Abstract

This paper investigates point-to-point multimodal digital semantic communications in a task-oriented setup, where messages are classified at the receiver. We employ a pre-trained transformer model to extract semantic information and propose three methods for generating semantic codewords. First, we propose semantic quantization that [...] Read more.

This paper investigates point-to-point multimodal digital semantic communications in a task-oriented setup, where messages are classified at the receiver. We employ a pre-trained transformer model to extract semantic information and propose three methods for generating semantic codewords. First, we propose semantic quantization that uses quantized embeddings of source realizations as a codebook. We investigate the fixed-length coding, considering the source semantic structure and end-to-end semantic distortion. We propose a neural network-based codeword assignment mechanism incorporating codeword transition probabilities to minimize the expected semantic distortion. Second, we present semantic compression that clusters embeddings, exploiting the inherent semantic redundancies to reduce the codebook size, i.e., further compression. Third, we introduce a semantic vector-quantized autoencoder (VQ-AE) that learns a codebook through training. In all cases, we follow this semantic source code with a standard channel code to transmit over the wireless channel. In addition to classification accuracy, we assess pre-communication overhead via a novel metric we term system time efficiency. Extensive experiments demonstrate that our proposed semantic source-coding approaches provide comparable accuracy and better system time efficiency compared to their learning-based counterparts. Full article

(This article belongs to the Special Issue Semantic Information Theory)

► Show Figures

Figure 1

23 pages, 758 KB

Open AccessEditor’s ChoiceArticle

Low-Complexity Automorphism Ensemble Decoding of Reed-Muller Codes Using Path Pruning

by Kairui Tian, Rongke Liu and Zheng Lu

Entropy 2025, 27(8), 808; https://doi.org/10.3390/e27080808 - 28 Jul 2025

Viewed by 1013

Abstract

The newly developed automorphism ensemble decoder (AED) leverages the rich automorphisms of Reed–Muller (RM) codes to achieve near maximum likelihood (ML) performance at short code lengths. However, the performance gain of AED comes at the cost of high complexity, as the ensemble size [...] Read more.

The newly developed automorphism ensemble decoder (AED) leverages the rich automorphisms of Reed–Muller (RM) codes to achieve near maximum likelihood (ML) performance at short code lengths. However, the performance gain of AED comes at the cost of high complexity, as the ensemble size required for near ML decoding grows exponentially with the code length. In this work, we address this complexity issue by focusing on the factor graph permutation group (FGPG), a subgroup of the full automorphism group of RM codes, to generate permutations for AED. We propose a uniform partitioning of FGPG based on the affine bijection permutation matrices of automorphisms, where each subgroup of FGPG exhibits permutation invariance (PI) in a Plotkin construction-based information set partitioning for RM codes. Furthermore, from the perspective of polar codes, we exploit the PI property to prove a subcode estimate convergence (SEC) phenomenon in the AED that utilizes successive cancellation (SC) or SC list (SCL) constituent decoders. Observing that strong SEC correlates with low noise levels, where the full decoding capacity of AED is often unnecessary, we perform path pruning to reduce the decoding complexity without compromising the performance. Our proposed SEC-aided path pruning allows only a subset of constituent decoders to continue decoding when the intensity of SEC exceeds a preset threshold during decoding. Numerical results demonstrate that, for the FGPG-based AED of various short RM codes, the proposed SEC-aided path pruning technique incurs negligible performance degradation, while achieving a complexity reduction of up to 67.6%. Full article

(This article belongs to the Special Issue Next-Generation Channel Coding: Theory and Applications)

► Show Figures

Figure 1

15 pages, 5676 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Transverse Self-Propulsion Enhances the Aggregation of Active Dumbbells

by Pasquale Digregorio, Claudio Basilio Caporusso, Lucio Mauro Carenza, Giuseppe Gonnella, Daniela Moretti, Giuseppe Negro, Massimiliano Semeraro and Antonio Suma

Entropy 2025, 27(7), 692; https://doi.org/10.3390/e27070692 - 27 Jun 2025

Cited by 1 | Viewed by 1042

Abstract

We investigate a two-dimensional system of active Brownian dumbbells using molecular dynamics simulations. In this model, each dumbbell is driven by an active force oriented perpendicular to the axis connecting its two constituent beads. We characterize the resulting phase behavior and find that, [...] Read more.

We investigate a two-dimensional system of active Brownian dumbbells using molecular dynamics simulations. In this model, each dumbbell is driven by an active force oriented perpendicular to the axis connecting its two constituent beads. We characterize the resulting phase behavior and find that, across all values of activity, the system undergoes phase separation between dilute and dense phases. The dense phase exhibits hexatic order, and for large enough activity, we observe a marked increase in local polarization, with dumbbells predominantly oriented towards the interior of the clusters. Compared to the case of axially self-propelled dumbbells, we find that the binodal region is enlarged towards lower densities at all activities. This shift arises because dumbbells with transverse propulsion can more easily form stable cluster cores, serving as nucleation seeds, and show a highly suppressed escaping rate from the cluster boundary. Finally, we observe that clusters exhibit spontaneous rotation, with the modulus of the angular velocity scaling as

ω \sim r_{g}^{- 2}

, where

r_{g}

is the cluster’s radius of gyration. This contrasts with axially propelled dumbbells, where the scaling follows

ω \sim r_{g}^{- 1}

. We develop a simplified analytical model to rationalize this scaling behavior. Full article

(This article belongs to the Section Non-equilibrium Phenomena)

► Show Figures

Figure 1

24 pages, 5959 KB

Open AccessEditor’s ChoiceArticle

An Information Geometry-Based Track-Before-Detect Algorithm for Range-Azimuth Measurements in Radar Systems

by Jinguo Liu, Hao Wu, Zheng Yang, Xiaoqiang Hua and Yongqiang Cheng

Entropy 2025, 27(6), 637; https://doi.org/10.3390/e27060637 - 14 Jun 2025

Cited by 1 | Viewed by 1303

Abstract

The detection of weak moving targets in heterogeneous clutter backgrounds is a significant challenge in radar systems. In this paper, we propose a track-before-detect (TBD) method based on information geometry (IG) theory applied to range-azimuth measurements, which extends the IG detectors to multi-frame [...] Read more.

The detection of weak moving targets in heterogeneous clutter backgrounds is a significant challenge in radar systems. In this paper, we propose a track-before-detect (TBD) method based on information geometry (IG) theory applied to range-azimuth measurements, which extends the IG detectors to multi-frame detection through inter-frame information integration. The approach capitalizes on the distinctive benefits of the information geometry detection framework in scenarios with strong clutter, while enhancing the integration of information across multiple frames within the TBD approach. Specifically, target and clutter trajectories in multi-frame range-azimuth measurements are modeled on the Hermitian positive definite (HPD) and power spectrum (PS) manifolds. A scoring function based on information geometry, which uses Kullback–Leibler (KL) divergence as a geometric metric, is then devised to assess these motion trajectories. Moreover, this study devises a solution framework employing dynamic programming (DP) with constraints on state transitions, culminating in an integrated merit function. This algorithm identifies target trajectories by maximizing the integrated merit function. Experimental validation using real-recorded sea clutter datasets showcases the effectiveness of the proposed algorithm, yielding a minimum 3 dB enhancement in signal-to-clutter ratio (SCR) compared to traditional approaches. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

29 pages, 3108 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Soft Classification in a Composite Source Model

by Yuefeng Cao, Jiakun Liu and Wenyi Zhang

Entropy 2025, 27(6), 620; https://doi.org/10.3390/e27060620 - 11 Jun 2025

Cited by 1 | Viewed by 890

Abstract

A composite source model consists of an intrinsic state and an extrinsic observation. The fundamental performance limit of reproducing the intrinsic state is characterized by the indirect rate–distortion function. In a remote classification application, a source encoder encodes the extrinsic observation (e.g., image) [...] Read more.

A composite source model consists of an intrinsic state and an extrinsic observation. The fundamental performance limit of reproducing the intrinsic state is characterized by the indirect rate–distortion function. In a remote classification application, a source encoder encodes the extrinsic observation (e.g., image) into bits, and a source decoder plays the role of a classifier that reproduces the intrinsic state (e.g., label of image). In this work, we characterize the general structure of the optimal transition probability distribution, achieving the indirect rate–distortion function. This optimal solution can be interpreted as a “soft classifier”, which generalizes the conventionally adopted “classify-then-compress” scheme. We then apply the soft classification to aid the lossy compression of the extrinsic observation of a composite source. This leads to a coding scheme that exploits the soft classifier to guide reproduction, outperforming existing coding schemes without classification or with hard classification. Full article

(This article belongs to the Special Issue Semantic Information Theory)

► Show Figures

Figure 1

15 pages, 419 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Ordinal Random Processes

by Christoph Bandt

Entropy 2025, 27(6), 610; https://doi.org/10.3390/e27060610 - 7 Jun 2025

Cited by 1 | Viewed by 733

Abstract

Ordinal patterns have proven to be a valuable tool in many fields. Here, we address the need for theoretical models. A paradigmatic example shows that a model for frequencies of ordinal patterns can be determined without any numerical values. We specify the important [...] Read more.

Ordinal patterns have proven to be a valuable tool in many fields. Here, we address the need for theoretical models. A paradigmatic example shows that a model for frequencies of ordinal patterns can be determined without any numerical values. We specify the important concept of stationary order and the fundamental problems to be solved in order to establish a genuine statistical methodology for ordinal time series. Full article

(This article belongs to the Special Issue Ordinal Patterns-Based Tools and Their Applications)

► Show Figures

Figure 1

8 pages, 379 KB

Open AccessEditor’s ChoiceArticle

Scaling Laws in Language Families

by Maelyson Rolim Fonseca dos Santos and Marcelo Andrade de Filgueiras Gomes

Entropy 2025, 27(6), 588; https://doi.org/10.3390/e27060588 - 31 May 2025

Cited by 1 | Viewed by 1016

Abstract

This article investigates scaling laws within language families using data from over six thousand languages and analyzes emergent patterns observed in Zipf-like classification graphs. Both macroscopic (based on the number of languages by family) and microscopic (based on the number of speakers by [...] Read more.

This article investigates scaling laws within language families using data from over six thousand languages and analyzes emergent patterns observed in Zipf-like classification graphs. Both macroscopic (based on the number of languages by family) and microscopic (based on the number of speakers by language within a family) aspects of these classifications are examined. Particularly noteworthy is the discovery of a distinct division among the fourteen largest contemporary language families, excluding Afro-Asiatic and Nilo-Saharan languages. These families are found to be distributed across three language family quadruplets, each characterized by significantly different exponents in the Zipf graphs. This finding sheds light on the underlying structure and organization of major language families, revealing intriguing insights into the nature of linguistic diversity and distribution. Full article

(This article belongs to the Special Issue Complexity Characteristics of Natural Language)

► Show Figures

Figure 1

23 pages, 2439 KB

Open AccessEditor’s ChoiceArticle

The Origin of Shared Emergent Properties in Discrete Systems

by Les Hatton and Greg Warr

Entropy 2025, 27(6), 561; https://doi.org/10.3390/e27060561 - 26 May 2025

Cited by 2 | Viewed by 1409

Abstract

Here, we propose that the shared emergent properties reproducibly observed in discrete systems can be explained by a theory that embeds the Conservation of Hartley–Shannon Information (CoHSI) in a statistical mechanics framework. Specific predictions of global properties that represent the most likely equilibrium [...] Read more.

Here, we propose that the shared emergent properties reproducibly observed in discrete systems can be explained by a theory that embeds the Conservation of Hartley–Shannon Information (CoHSI) in a statistical mechanics framework. Specific predictions of global properties that represent the most likely equilibrium state should be apparent in all qualifying systems, regardless of provenance. We demonstrate that these predictions of emergent global properties hold true in systems as disparate as collections of software written in the programming language C and collections of proteins. The implication is that the emergence of such shared properties is not driven by any specific local mechanism as the systems are so different. This raises the interesting prospect that important properties of biological systems (exemplified here by the length and multiplicity distributions of proteins) have little, if anything, to do with natural selection. Similarly, the size distribution of components and the frequency of tokens observed in computer software in C emerge as the most likely states, and are thus properties that are divorced from human agency, regardless of functionality. Full article

(This article belongs to the Section Entropy and Biology)

► Show Figures

Figure 1

14 pages, 262 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Universal Encryption of Individual Sequences Under Maximal Information Leakage

by Neri Merhav

Entropy 2025, 27(6), 551; https://doi.org/10.3390/e27060551 - 24 May 2025

Viewed by 637

Abstract

We consider the Shannon cipher system in the framework of individual sequences and finite-state encrypters under the metric of maximal information leakage. A lower bound and an asymptotically matching upper bound on the leakage are derived, which lead to the conclusion that asymptotically [...] Read more.

We consider the Shannon cipher system in the framework of individual sequences and finite-state encrypters under the metric of maximal information leakage. A lower bound and an asymptotically matching upper bound on the leakage are derived, which lead to the conclusion that asymptotically minimum leakage can be attained by Lempel–Ziv compression followed by one-time pad encryption of the compressed bitstream. Full article

(This article belongs to the Special Issue Information Theory and Data Compression)

10 pages, 8363 KB

Open AccessEditor’s ChoiceArticle

Improved Reconstruction of Chaotic Signals from Ordinal Networks

by Antonio Politi and Leonardo Ricci

Entropy 2025, 27(5), 499; https://doi.org/10.3390/e27050499 - 6 May 2025

Viewed by 818

Abstract

Permutation entropy is customarily implemented to quantify the intrinsic indeterminacy of complex time series, under the assumption that determinism manifests itself by lowering the (permutation) entropy of the resulting symbolic sequence. We expect this to be roughly true, but, in general, it is [...] Read more.

Permutation entropy is customarily implemented to quantify the intrinsic indeterminacy of complex time series, under the assumption that determinism manifests itself by lowering the (permutation) entropy of the resulting symbolic sequence. We expect this to be roughly true, but, in general, it is not clear to what extent a given ordinal pattern indeed provides a faithful reconstruction of the original signal. Here, we address this question by attempting the reconstruction of the original time series by invoking an ergodic Markov approximation of the symbolic dynamics, thereby inverting the encoding procedure. Using the Hénon map as a testbed, we show that a meaningful reconstruction can also be made in the presence of a small observational noise. Full article

(This article belongs to the Special Issue Ordinal Patterns-Based Tools and Their Applications)

► Show Figures

Figure 1

14 pages, 617 KB

Open AccessEditor’s ChoiceArticle

Iterative Forecasting of Financial Time Series: The Greek Stock Market from 2019 to 2024

by Evangelos Bakalis and Francesco Zerbetto

Entropy 2025, 27(5), 497; https://doi.org/10.3390/e27050497 - 4 May 2025

Cited by 2 | Viewed by 2518

Abstract

Predicting the evolution of financial data, if at all possible, would be very beneficial in revealing the ways in which different aspects of a global environment can impact local economies. We employ an iterative stochastic differential equation that accurately forecasts an economic time [...] Read more.

Predicting the evolution of financial data, if at all possible, would be very beneficial in revealing the ways in which different aspects of a global environment can impact local economies. We employ an iterative stochastic differential equation that accurately forecasts an economic time series’s next value by analysing its past. The input financial data are assumed to be consistent with an

α

-stable Lévy motion. The computation of the scaling exponent and the value of

α

, which characterises the type of the

α

-stable Lévy motion, are crucial for the iterative scheme. These two indices can be determined at each iteration from the form of the structure function, for the computation of which we use the method of generalised moments. Their values are used for the creation of the corresponding

α

-stable Lévy noise, which acts as a seed for the stochastic component. Furthermore, the drift and diffusion terms are calculated at each iteration. The proposed model is general, allowing the kind of stochastic process to vary from one iterative step to another, and its applicability is not restricted to financial data. As a case study, we consider Greece’s stock market general index over a period of five years, from September 2019 to September 2024, after the completion of bailout programmes. Greece’s economy changed from a restricted to a free market over the chosen era, and its stock market trading increments are likely to be describable by an

α

-stable L’evy motion. We find that

α = 2

and the scaling exponent H varies over time for every iterative step we perform. The forecasting points follow the same trend, are in good agreement with the actual data, and for most of the forecasts, the percentage error is less than 2%. Full article

(This article belongs to the Special Issue Entropy-Based Applications in Sociophysics II)

► Show Figures

Figure 1

24 pages, 1419 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Measurement-Induced Symmetry Restoration and Quantum Mpemba Effect

by Giuseppe Di Giulio, Xhek Turkeshi and Sara Murciano

Entropy 2025, 27(4), 407; https://doi.org/10.3390/e27040407 - 10 Apr 2025

Cited by 11 | Viewed by 2258

Abstract

Monitoring a quantum system can profoundly alter its dynamical properties, leading to non-trivial emergent phenomena. In this work, we demonstrate that dynamical measurements strongly influence the evolution of symmetry in many-body quantum systems. Specifically, we demonstrate that monitored systems governed by non-Hermitian dynamics [...] Read more.

Monitoring a quantum system can profoundly alter its dynamical properties, leading to non-trivial emergent phenomena. In this work, we demonstrate that dynamical measurements strongly influence the evolution of symmetry in many-body quantum systems. Specifically, we demonstrate that monitored systems governed by non-Hermitian dynamics exhibit a quantum Mpemba effect, where systems with stronger initial asymmetry relax faster to a symmetric state. Crucially, this phenomenon is purely measurement-induced: in the absence of measurements, we find states where the corresponding unitary evolution does not display any Mpemba effect. Furthermore, we uncover a novel measurement-induced symmetry restoration mechanism: below a critical measurement rate, the symmetry remains broken, but beyond a threshold, it is fully restored in the thermodynamic limit—along with the emergence of the quantum Mpemba effect. Full article

(This article belongs to the Special Issue Entanglement Entropy in Quantum Field Theory)

► Show Figures

Figure 1

27 pages, 15276 KB

Open AccessEditor’s ChoiceArticle

The Dynamics of Shannon Entropy in Analyzing Climate Variability for Modeling Temperature and Precipitation Uncertainty in Poland

by Bernard Twaróg

Entropy 2025, 27(4), 398; https://doi.org/10.3390/e27040398 - 8 Apr 2025

Cited by 2 | Viewed by 2301

Abstract

The aim of this study is to quantitatively analyze the long-term climate variability in Poland during the period 1901–2010, using Shannon entropy as a measure of uncertainty and complexity within the atmospheric system. The analysis is based on the premise that variations in [...] Read more.

The aim of this study is to quantitatively analyze the long-term climate variability in Poland during the period 1901–2010, using Shannon entropy as a measure of uncertainty and complexity within the atmospheric system. The analysis is based on the premise that variations in temperature and precipitation reflect the dynamic nature of the climate, understood as a nonlinear system sensitive to fluctuations. This study focuses on monthly distributions of temperature and precipitation, modeled using the bivariate Clayton copula function. A normal marginal distribution was adopted for temperature and a gamma distribution for precipitation, both validated using the Anderson–Darling test. To improve estimation accuracy, a bootstrap resampling technique and numerical integration were applied to calculate Shannon entropy at each of the 396 grid points, with a spatial resolution of 0.25° × 0.25°. The results indicate a significant increase in Shannon entropy during the summer months, particularly in July (+0.203 bits) and January (+0.221 bits), compared to the baseline period (1901–1971), suggesting a growing unpredictability of the climate. The most pronounced trend changes were identified in the years 1985–1996 (as indicated by the Pettitt test), while seasonal trends were confirmed using the Mann–Kendall test. A spatial analysis of entropy at the levels of administrative regions and catchments revealed notable regional disparities—entropy peaked in January in the West Pomeranian Voivodeship (4.919 bits) and reached its minimum in April in Greater Poland (3.753 bits). Additionally, this study examined the relationship between Shannon entropy and global climatic indicators, including the Land–Ocean Temperature Index (NASA GISTEMP) and the ENSO index (NINO3.4). Statistically significant positive correlations were observed between entropy and global temperature anomalies during both winter (

ρ

= 0.826) and summer (

ρ

= 0.650), indicating potential linkages between local climate variability and global warming trends. To explore the direction of this relationship, a Granger causality test was conducted, which did not reveal statistically significant causality between NINO3.4 and Shannon entropy (

p

> 0.05 for all lags tested), suggesting that the observed relationships are likely co-varying rather than causal in the Granger sense. Further phase–space analysis (with a delay of

τ

= 3 months) allowed for the identification of attractors characteristic of chaotic systems. The entropy trajectories revealed transitions from equilibrium states (average entropy: 4.124–4.138 bits) to highly unstable states (up to 4.768 bits), confirming an increase in the complexity of the climate system. Shannon entropy thus proves to be a valuable tool for monitoring local climatic instability and may contribute to improved risk modeling of droughts and floods in the context of climate change in Poland. Full article

(This article belongs to the Special Issue 25 Years of Sample Entropy)

► Show Figures

Figure 1

18 pages, 314 KB

Open AccessEditor’s ChoiceArticle

The POVM Theorem in Bohmian Mechanics

by Christian Beck and Dustin Lazarovici

Entropy 2025, 27(4), 391; https://doi.org/10.3390/e27040391 - 7 Apr 2025

Viewed by 2306

Abstract

The POVM theorem is a central result in Bohmian mechanics, grounding the measurement formalism of standard quantum mechanics in a statistical analysis based on the quantum equilibrium hypothesis (the Born rule for Bohmian particle positions). It states that the outcome statistics of an [...] Read more.

The POVM theorem is a central result in Bohmian mechanics, grounding the measurement formalism of standard quantum mechanics in a statistical analysis based on the quantum equilibrium hypothesis (the Born rule for Bohmian particle positions). It states that the outcome statistics of an experiment are described by a positive operator-valued measure (POVM) acting on the Hilbert space of the measured system. In light of recent debates about the scope and status of this result, we provide a systematic presentation of the POVM theorem and its underlying assumptions with a focus on their conceptual foundations and physical justifications. We conclude with a brief discussion of the scope of the POVM theorem—especially the sense in which it does (and does not) place limits on what is “measurable” in Bohmian mechanics. Full article

(This article belongs to the Special Issue Quantum Foundations: 100 Years of Born’s Rule)

14 pages, 265 KB

Open AccessEditor’s ChoiceArticle

Successive Refinement for Lossy Compression of Individual Sequences

by Neri Merhav

Entropy 2025, 27(4), 370; https://doi.org/10.3390/e27040370 - 31 Mar 2025

Viewed by 471

Abstract

We consider the problem of successive-refinement coding for lossy compression of individual sequences, namely, compression in two stages, where in the first stage, a coarse description at a relatively low rate is sent from the encoder to the decoder, and in the second [...] Read more.

We consider the problem of successive-refinement coding for lossy compression of individual sequences, namely, compression in two stages, where in the first stage, a coarse description at a relatively low rate is sent from the encoder to the decoder, and in the second stage, an additional coding rate is allocated in order to refine the description and thereby improve the reproduction. Our main result is in establishing outer bounds (converse theorems) for the rate region where we limit the encoders to be finite-state machines in the spirit of Ziv and Lempel’s 1978 model. The matching achievability scheme is conceptually straightforward. We also consider the more general multiple description coding problem on a similar footing and propose achievability schemes that are analogous to the well-known El Gamal–Cover and the Zhang–Berger achievability schemes of memoryless sources and additive distortion measures. Full article

(This article belongs to the Collection Feature Papers in Information Theory)

25 pages, 1002 KB

Open AccessEditor’s ChoiceArticle

InfoMat: Leveraging Information Theory to Visualize and Understand Sequential Data

by Dor Tsur and Haim Permuter

Entropy 2025, 27(4), 357; https://doi.org/10.3390/e27040357 - 28 Mar 2025

Viewed by 1029

Abstract

Despite the widespread use of information measures in analyzing probabilistic systems, effective visualization tools for understanding complex dependencies in sequential data are scarce. In this work, we introduce the information matrix (InfoMat), a novel and intuitive matrix representation of information transfer in sequential [...] Read more.

Despite the widespread use of information measures in analyzing probabilistic systems, effective visualization tools for understanding complex dependencies in sequential data are scarce. In this work, we introduce the information matrix (InfoMat), a novel and intuitive matrix representation of information transfer in sequential systems. InfoMat provides a structured visual perspective on mutual information decompositions, enabling the discovery of new relationships between sequential information measures and enhancing interpretability in time series data analytics. We demonstrate how InfoMat captures key sequential information measures, such as directed information and transfer entropy. To facilitate its application in real-world datasets, we propose both an efficient Gaussian mutual information estimator and a neural InfoMat estimator based on masked autoregressive flows to model more complex dependencies. These estimators make InfoMat a valuable tool for uncovering hidden patterns in data analytics applications, encompassing neuroscience, finance, communication systems, and machine learning. We further illustrate the utility of InfoMat in visualizing information flow in real-world sequential physiological data analysis and in visualizing information flow in communication channels under various coding schemes. By mapping visual patterns in InfoMat to various modes of dependence structures, we provide a data-driven framework for analyzing causal relationships and temporal interactions. InfoMat thus serves as both a theoretical and empirical tool for data-driven decision making, bridging the gap between information theory and applied data analytics. Full article

(This article belongs to the Special Issue Information-Theoretic Methods in Data Analytics)

► Show Figures

Figure 1

21 pages, 737 KB

Open AccessEditor’s ChoiceArticle

A Model for the Formation of Beliefs and Social Norms Based on the Satisfaction Problem (SAT)

by Bastien Chopard, Franck Raynaud and Julien Stalhandske

Entropy 2025, 27(4), 358; https://doi.org/10.3390/e27040358 - 28 Mar 2025

Cited by 2 | Viewed by 797

Abstract

We propose a numerical representation of beliefs in social systems based on the so-called SAT problem in computer science. The main idea is that a belief system is a set of true/false values associated with claims or propositions. Each individual assigns these values [...] Read more.

We propose a numerical representation of beliefs in social systems based on the so-called SAT problem in computer science. The main idea is that a belief system is a set of true/false values associated with claims or propositions. Each individual assigns these values according to its cognitive system in order to minimize logical contradictions, thus trying to solve a satisfaction problem. Social interactions between agents that disagree on a proposition can be introduced in order to see how, in the long term, social norms and competing belief systems build up in a population. Among other metrics, entropy is used to characterize the diversity of belief systems. Full article

(This article belongs to the Special Issue Entropy-Based Applications in Sociophysics II)

► Show Figures

Figure 1

13 pages, 2121 KB

Open AccessEditor’s ChoiceArticle

Structural Transitions and Melting of Two-Dimensional Ion Crystals in RF Traps

by Boris V. Pashinsky, Alexander Kato and Boris B. Blinov

Entropy 2025, 27(4), 325; https://doi.org/10.3390/e27040325 - 21 Mar 2025

Cited by 2 | Viewed by 1543

Abstract

We investigate the structural properties and melting behaviors of two-dimensional ion crystals in an RF trap, focusing on the effects of ion temperature and trap potential symmetry. We identify distinct crystal structures that form under varying trapping conditions and temperatures through experimental observations [...] Read more.

We investigate the structural properties and melting behaviors of two-dimensional ion crystals in an RF trap, focusing on the effects of ion temperature and trap potential symmetry. We identify distinct crystal structures that form under varying trapping conditions and temperatures through experimental observations and theoretical analyses. As the temperature increases or the trap potential becomes more symmetric, we observe a transition from a lattice arrangement to elongated ring-like formations aligned along the trap axes. Our experimental and theoretical efforts enhance our understanding of phase transitions in low-dimensional, confined systems, offering insights into the controlled formation of quantum crystals for applications in quantum simulations and many-body physics. Full article

(This article belongs to the Special Issue Quantum Computing with Trapped Ions)

► Show Figures

Figure 1

10 pages, 586 KB

Open AccessEditor’s ChoiceArticle

The Quantum Relative Entropy of the Schwarzschild Black Hole and the Area Law

by Ginestra Bianconi

Entropy 2025, 27(3), 266; https://doi.org/10.3390/e27030266 - 4 Mar 2025

Cited by 3 | Viewed by 3199

Abstract

The area law obeyed by the thermodynamic entropy of black holes is one of the fundamental results relating gravity to statistical mechanics. In this work, we provide a derivation of the area law for the quantum relative entropy of the Schwarzschild black hole [...] Read more.

The area law obeyed by the thermodynamic entropy of black holes is one of the fundamental results relating gravity to statistical mechanics. In this work, we provide a derivation of the area law for the quantum relative entropy of the Schwarzschild black hole for an arbitrary Schwarzschild radius. The quantum relative entropy between the metric of the manifold and the metric induced by the geometry and the matter field has been proposed in G. Bianconi as the action for entropic quantum gravity leading to modified Einstein equations. The quantum relative entropy generalizes Araki’s entropy and treats the metrics between zero-forms, one-forms, and two-forms as quantum operators. Although the Schwarzschild metric is not an exact solution of the modified Einstein equations of the entropic quantum gravity, it is an approximate solution valid in the low-coupling, small-curvature limit. Here, we show that the quantum relative entropy associated to the Schwarzschild metric obeys the area law for a large Schwarzschild radius. We provide a full statistical mechanics interpretation of the results. Full article

(This article belongs to the Special Issue Entropy in Quantum Systems and Quantum Field Theory (QFT): Gravity, 3rd Edition)

► Show Figures

Figure 1

47 pages, 4735 KB

Open AccessEditor’s ChoiceTutorial

Principles Entailed by Complexity, Crucial Events, and Multifractal Dimensionality

by Bruce J. West and Senthil Mudaliar

Entropy 2025, 27(3), 241; https://doi.org/10.3390/e27030241 - 26 Feb 2025

Cited by 3 | Viewed by 2419

Abstract

Complexity is one of those descriptive terms adopted in science that we think we understand until it comes time to form a coherent definition upon which everyone can agree. Suddenly, we are awash in conditions that qualify this or that situation, much like [...] Read more.

Complexity is one of those descriptive terms adopted in science that we think we understand until it comes time to form a coherent definition upon which everyone can agree. Suddenly, we are awash in conditions that qualify this or that situation, much like we were in the middle of the last century when it came time to determine the solutions to differential equations that were not linear. Consequently, this tutorial is not an essay on the mathematics of complexity nor is it a rigorous review of the recent growth spurt of complexity science, but is rather an exploration of how physiologic time series (PTS) in the life sciences that have eluded traditional mathematical modeling become less mysterious when certain historical assumptions are discarded and so-called ordinary statistical events in PTS are replaced with crucial events (CEs) using mutifractal dimensionality as the working measure of complexity. The empirical datasets considered include respiration, electrocardiograms (ECGs), and electroencephalograms (EEGs), and as different as these time series appear from one another when recorded, they are in fact shown to be in synchrony when properly processed using the technique of modified diffusion entropy analysis (MDEA). This processing reveals a new synchronization mechanism among the time series which simultaneously measures their complexity by means of the multifractal dimension of each time series and are shown to track one another across time. These results reveal a set of priciples that capture the manner in which information is exchanged among physiologic organ networks. Full article

(This article belongs to the Section Complexity)

► Show Figures

Figure 1

17 pages, 660 KB

Open AccessEditor’s ChoiceArticle

User-Centric Cell-Free Massive Multiple-Input-Multiple-Output System with Noisy Channel Gain Estimation and Line of Sight: A Beckmann Distribution Approach

by Danilo B. T. Almeida, Marcelo S. Alencar, Wamberto J. L. Queiroz, Rafael M. Duarte and Francisco Madeiro

Entropy 2025, 27(3), 223; https://doi.org/10.3390/e27030223 - 21 Feb 2025

Viewed by 1589

Abstract

This paper analyzes for the first time how the Beckmann distribution can be used to characterize the random variable that represents the envelope of the effective channel gain experienced by the k-th user equipment (UE) of a user-centric (UC) cell-free (CF) system [...] Read more.

This paper analyzes for the first time how the Beckmann distribution can be used to characterize the random variable that represents the envelope of the effective channel gain experienced by the k-th user equipment (UE) of a user-centric (UC) cell-free (CF) system in a scenario with noisy channel state information (CSI) estimation and line of sight (LoS). Additionally, it is shown how the Beckmann probability density function (PDF) can be used to derive the PDF and the cumulative density function (CDF) of the instantaneous signal-to-interference-plus-noise ratio (SINR) of the UC CF k-th UE, followed by applications in the ergodic capacity (EC) and outage probability (OP) expression derivations. It is shown that, regardless of the type of distribution considered for the channel gain between each access point (AP) and UE links, the effective gain presents a Beckmann distribution. Full article

(This article belongs to the Special Issue Wireless Communications: Signal Processing Perspectives)

► Show Figures

Figure 1

22 pages, 750 KB

Open AccessEditor’s ChoiceArticle

Levy Noise Affects Ornstein–Uhlenbeck Memory

by Iddo Eliazar

Entropy 2025, 27(2), 157; https://doi.org/10.3390/e27020157 - 2 Feb 2025

Cited by 2 | Viewed by 1644

Abstract

This paper investigates the memory of the Ornstein–Uhlenbeck process (OUP) via three ratios of the OUP increments: signal-to-noise, noise-to-noise, and tail-to-tail. Intuition suggests the following points: (1) changing the noise that drives the OUP from Gauss to Levy will not affect the memory, [...] Read more.

This paper investigates the memory of the Ornstein–Uhlenbeck process (OUP) via three ratios of the OUP increments: signal-to-noise, noise-to-noise, and tail-to-tail. Intuition suggests the following points: (1) changing the noise that drives the OUP from Gauss to Levy will not affect the memory, as both noises share the common ‘independent increments’ property; (2) changing the auto-correlation of the OUP from exponential to slowly decaying will affect the memory, as the change yields a process with long-range correlations; and (3) with regard to Levy driving noise, the greater the noise fluctuations, the noisier the prediction of the OUP increments. This paper shows that intuition is plain wrong. Indeed, a detailed analysis establishes that for each of the three above-mentioned points, the very converse holds. Hence, Levy noise has a significant and counter-intuitive effect on Ornstein–Uhlenbeck memory. Full article

(This article belongs to the Collection Foundations of Statistical Mechanics)

► Show Figures

Figure 1

15 pages, 4130 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Protocells Either Synchronize or Starve

by Marco Villani and Roberto Serra

Entropy 2025, 27(2), 154; https://doi.org/10.3390/e27020154 - 2 Feb 2025

Cited by 1 | Viewed by 1266

Abstract

Two different processes take place in self-reproducing protocells, i.e., (i) cell reproduction by fission and (ii) duplication of the genetic material. One major problem is indeed that of assuring that the two processes take place at the same pace, i.e., that they synchronize, [...] Read more.

Two different processes take place in self-reproducing protocells, i.e., (i) cell reproduction by fission and (ii) duplication of the genetic material. One major problem is indeed that of assuring that the two processes take place at the same pace, i.e., that they synchronize, which is a necessary condition for sustainable growth. In previous theoretical works, using dynamical models, we had shown that such synchronization can spontaneously emerge, generation after generation, under a broad set of hypotheses about the architecture of the protocell, the nature of the self-replicating molecules, and the types of kinetic equations. However, an important class of cases (quadratic or higher-order self-replication) did not synchronize in the models we had used, but could actually lead to divergence of the concentration of replicators. We show here that this behavior is due to a simplification of the previous models, i.e., the “buffering” hypothesis, which assumes instantaneous equilibrium of the internal and external concentrations of those compounds which can cross the cell membrane. That divergence disappears if we make use of more realistic dynamical models, with finite transmembrane diffusion rates of the precursors of replicators. Full article

(This article belongs to the Section Entropy and Biology)

► Show Figures

Figure 1

30 pages, 1295 KB

Open AccessEditor’s ChoiceArticle

Why Uncertainty Is Essential for Consciousness: Local Prospect Theory vs. Predictive Processing

by Francis Heylighen and Shima Beigi

Entropy 2025, 27(2), 140; https://doi.org/10.3390/e27020140 - 28 Jan 2025

Viewed by 4855

Abstract

We present and develop local prospect theory (LPT), a novel framework for understanding consciousness, and, in particular, subjective experience and free will. While predictive processing (PP) theories model the brain as trying to optimize the accuracy of predictions, LPT sees uncertainty as an [...] Read more.

We present and develop local prospect theory (LPT), a novel framework for understanding consciousness, and, in particular, subjective experience and free will. While predictive processing (PP) theories model the brain as trying to optimize the accuracy of predictions, LPT sees uncertainty as an essential feature of conscious decision-making. This is achieved by creating a “local prospect”—a range of potential developments colored by subjective experience from which an agent can freely choose how to react. Drawing on global workspace theory, LPT conceptualizes consciousness as a self-maintaining process of circulating neural activation, creating a temporary working memory where thoughts and feelings coming from different brain modules enter into an asynchronous, non-linear interaction. This contrasts with unconscious processes, which operate automatically and deterministically. LPT proposes entropy-based measures, including the determination of actions by conditions and the breadth of prospect, to quantify the range of potential developments considered. This framework allows us to understand Buddhist practices and concepts, such as mindfulness, liberation from attachments, and meditation, which broaden consciousness and de-automatize reactions by reducing the influence of conditioning. The proposed prospect measure may be operationalized by indicators such as the variety of action, breadth of perception, and unpredictability of behavior, thus allowing for the empirical testing of the theory. Full article

(This article belongs to the Special Issue Complexity and Evolution, 2nd Edition)

► Show Figures

Figure 1

18 pages, 3516 KB

Open AccessEditor’s ChoiceArticle

Temperature Gradients as a Data Storage Principle

by Jeroen Schoenmaker, Pâmella Gonçalves Martins and Julio Carlos Teixeira

Entropy 2025, 27(2), 129; https://doi.org/10.3390/e27020129 - 26 Jan 2025

Cited by 1 | Viewed by 2049

Abstract

In this work, we analyze the thermodynamic principles underlying modern data storage systems, including Random Access Memory (RAM), hard disk drive (HDD), flash memory, magnetic RAM (MRAM), ferroelectric RAM (FeRAM), and phase-change RAM (PCRAM), as well as other less well-known data storage mechanisms. [...] Read more.

In this work, we analyze the thermodynamic principles underlying modern data storage systems, including Random Access Memory (RAM), hard disk drive (HDD), flash memory, magnetic RAM (MRAM), ferroelectric RAM (FeRAM), and phase-change RAM (PCRAM), as well as other less well-known data storage mechanisms. The analysis is conducted in the context of data storage and processing in relation to Landauer’s principle, with special emphasis on hysteresis. Analogous to how heat engines are characterized by thermodynamic cycles, data storage systems are examined in terms of the hysteresis loop of their fundamental data unit. We explore the role of heat in data storage systems. Afterward, we introduce the concept of temperature gradient memory (TeGraM) along with a detailed layout of a realizable device. Experimental results demonstrating this technology are also presented. Full article

(This article belongs to the Special Issue 200 Years Anniversary of “Sadi Carnot, Réflexions Sur La Puissance Motrice Du Feu”; Bachelier: Paris, France, 1824)

► Show Figures

Figure 1

21 pages, 10983 KB

Open AccessFeature PaperEditor’s ChoiceReview

Machine Learning Advances in High-Entropy Alloys: A Mini-Review

by Yibo Sun and Jun Ni

Entropy 2024, 26(12), 1119; https://doi.org/10.3390/e26121119 - 20 Dec 2024

Cited by 11 | Viewed by 6142

Abstract

The efficacy of machine learning has increased exponentially over the past decade. The utilization of machine learning to predict and design materials has become a pivotal tool for accelerating materials development. High-entropy alloys are particularly intriguing candidates for exemplifying the potency of machine [...] Read more.

The efficacy of machine learning has increased exponentially over the past decade. The utilization of machine learning to predict and design materials has become a pivotal tool for accelerating materials development. High-entropy alloys are particularly intriguing candidates for exemplifying the potency of machine learning due to their superior mechanical properties, vast compositional space, and intricate chemical interactions. This review examines the general process of developing machine learning models. The advances and new algorithms of machine learning in the field of high-entropy alloys are presented in each part of the process. These advances are based on both improvements in computer algorithms and physical representations that focus on the unique ordering properties of high-entropy alloys. We also show the results of generative models, data augmentation, and transfer learning in high-entropy alloys and conclude with a summary of the challenges still faced in machine learning high-entropy alloys today. Full article

(This article belongs to the Special Issue Exploring Complexity: Short-Range Order in Medium and High-Entropy Materials)

► Show Figures

Figure 1

33 pages, 5394 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Carnot and the Archetype of Waterfalls

by Hans U. Fuchs, Elisabeth Dumont and Federico Corni

Entropy 2024, 26(12), 1066; https://doi.org/10.3390/e26121066 - 7 Dec 2024

Cited by 2 | Viewed by 2250

Abstract

Carnot treats Heat as a Force of Nature, with its typical fundamental characteristics of intensity and thermal tension (temperature and temperature difference), extension (amount of heat, i.e., caloric), and power. To suggest how the three aspects are related, he applies the imagery of [...] Read more.

Carnot treats Heat as a Force of Nature, with its typical fundamental characteristics of intensity and thermal tension (temperature and temperature difference), extension (amount of heat, i.e., caloric), and power. To suggest how the three aspects are related, he applies the imagery of waterfalls to causative thermal processes: heat powers motion in a heat engine just as falling water does when activating rotation in a water wheel. We understand Carnot’s waterfall imagery as an archetype of human reasoning—as an embodiment of how we experience and understand causative (agentive) phenomena. We project it onto the macroscopic phenomena identified in physical science and so unlock the power of analogical structure mapping between theories of fluids, electricity and magnetism, heat, substances, gravity, and linear and rotational motion. In particular, the notion of (motive) power of a waterfall lets us create imaginative explanations of the interactions of Forces of Nature and helps us construct a generalized energy principle. Two-hundred years after Carnot made us aware of it, his Waterfall Analogy is a powerful example of theory construction with roots deep in how we experience phenomena as caused by natural agents. Full article

(This article belongs to the Special Issue 200 Years Anniversary of “Sadi Carnot, Réflexions Sur La Puissance Motrice Du Feu”; Bachelier: Paris, France, 1824)

► Show Figures

Figure 1

11 pages, 286 KB

Open AccessEditor’s ChoiceArticle

Entropic Order Parameters for Categorical Symmetries in 2D-CFT

by Javier Molina-Vilaplana, Pablo Saura-Bastida and Germán Sierra

Entropy 2024, 26(12), 1064; https://doi.org/10.3390/e26121064 - 6 Dec 2024

Cited by 1 | Viewed by 1172

Abstract

In this work, we propose an information theoretic order parameter able to characterize the presence and breaking of categorical symmetries in

(1 + 1)

-d rational conformal field theories (RCFTs). Specifically, we compute the quantum relative entropy between the ground states [...] Read more.

In this work, we propose an information theoretic order parameter able to characterize the presence and breaking of categorical symmetries in

(1 + 1)

-d rational conformal field theories (RCFTs). Specifically, we compute the quantum relative entropy between the ground states of RCFTs representing the critical point of phase transitions between different symmetry-broken phases of theories with categorical symmetries, and their symmetrized versions. We find that, at leading order in the high temperature limit, this relative entropy only depends on the expectation values of the quantum dimensions of the topological operators implementing the categorical symmetry. This dependence suggests that our proposal can be used to characterize the different broken phases of

(1 + 1)

-d theories with categorical symmetries. Full article

(This article belongs to the Special Issue Entanglement Entropy in Quantum Field Theory)

20 pages, 845 KB

Open AccessFeature PaperEditor’s ChoiceArticle

Kinetic Theory of Self-Propelled Particles with Nematic Alignment

by Horst-Holger Boltz, Benjamin Kohler and Thomas Ihle

Entropy 2024, 26(12), 1054; https://doi.org/10.3390/e26121054 - 4 Dec 2024

Cited by 5 | Viewed by 2770

Abstract

We present the results from kinetic theory for a system of self-propelled particles with alignment interactions of higher-order symmetry, particularly nematic ones. To this end, we use the Landau equation approach, a systematic approximation to the BBGKY hierarchy for small effective couplings. Our [...] Read more.

We present the results from kinetic theory for a system of self-propelled particles with alignment interactions of higher-order symmetry, particularly nematic ones. To this end, we use the Landau equation approach, a systematic approximation to the BBGKY hierarchy for small effective couplings. Our calculations are presented in a pedagogical way with the explicit goal of serving as a tutorial from a physicists’ perspective into applying kinetic theory ideas beyond mean-field to active matter systems with essentially no prerequisites and yield predictions without free parameters that are in quantitative agreement with direct agent-based simulations Full article

(This article belongs to the Collection Foundations of Statistical Mechanics)

► Show Figures

Graphical abstract

32 pages, 686 KB

Open AccessEditor’s ChoiceArticle

Opening the AI Black Box: Distilling Machine-Learned Algorithms into Code

by Eric J. Michaud, Isaac Liao, Vedang Lad, Ziming Liu, Anish Mudide, Chloe Loughridge, Zifan Carl Guo, Tara Rezaei Kheirkhah, Mateja Vukelić and Max Tegmark

Entropy 2024, 26(12), 1046; https://doi.org/10.3390/e26121046 - 2 Dec 2024

Cited by 2 | Viewed by 3918

Abstract

Can we turn AI black boxes into code? Although this mission sounds extremely challenging, we show that it is not entirely impossible by presenting a proof-of-concept method, MIPS, that can synthesize programs based on the automated mechanistic interpretability of neural networks trained to [...] Read more.

Can we turn AI black boxes into code? Although this mission sounds extremely challenging, we show that it is not entirely impossible by presenting a proof-of-concept method, MIPS, that can synthesize programs based on the automated mechanistic interpretability of neural networks trained to perform the desired task, auto-distilling the learned algorithm into Python code. We test MIPS on a benchmark of 62 algorithmic tasks that can be learned by an RNN and find it highly complementary to GPT-4: MIPS solves 32 of them, including 13 that are not solved by GPT-4 (which also solves 30). MIPS uses an integer autoencoder to convert the RNN into a finite state machine, then applies Boolean or integer symbolic regression to capture the learned algorithm. As opposed to large language models, this program synthesis technique makes no use of (and is therefore not limited by) human training data such as algorithms and code from GitHub. We discuss opportunities and challenges for scaling up this approach to make machine-learned models more interpretable and trustworthy. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

7 pages, 216 KB

Open AccessEditor’s ChoiceArticle

A Characterization of Optimal Prefix Codes

by Spencer Congero and Kenneth Zeger

Entropy 2024, 26(12), 1000; https://doi.org/10.3390/e26121000 - 21 Nov 2024

Cited by 2 | Viewed by 1570

Abstract

A property of prefix codes called strong monotonicity is introduced, and it is proven that for a given source, a prefix code is optimal if and only if it is complete and strongly monotone. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

► Show Figures

Figure 1

19 pages, 903 KB

Open AccessFeature PaperEditor’s ChoiceArticle

A Contemporary View on Carnot’s Réflexions

by Jan-Peter Meyn

Entropy 2024, 26(12), 1002; https://doi.org/10.3390/e26121002 - 21 Nov 2024

Cited by 1 | Viewed by 1585

Abstract

Entropy and energy had not yet been introduced to physics by the time Carnot wrote his seminal Réflexions. Scholars continue to discuss what he really had in mind and what misconceptions he might have had. Actually, his work can be read as a [...] Read more.

Entropy and energy had not yet been introduced to physics by the time Carnot wrote his seminal Réflexions. Scholars continue to discuss what he really had in mind and what misconceptions he might have had. Actually, his work can be read as a correct introduction to the physics of heat engines when the term calorique is replaced by entropy and entropy is used as the other fundamental thermal quantity besides temperature. Carnot’s concepts of falling entropy as an analogy to the waterfall, and the separation of real thermal processes into reversible and irreversible processes are adopted. Some details of Carnot’s treatise are ignored, but the principal ideas are quoted and assumed without modification. With only two thermal quantities, temperature and entropy, modern heat engines can be explained in detail. Only after the principal function of heat engines is developed is energy introduced as physical quantity in order to compare thermal engines with mechanical and electrical engines and, specifically, to calculate efficiency. Full article

(This article belongs to the Special Issue 200 Years Anniversary of “Sadi Carnot, Réflexions Sur La Puissance Motrice Du Feu”; Bachelier: Paris, France, 1824)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Editor’s Choice Articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI