Entropy

15 pages, 331 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

The Onset of Parisi’s Complexity in a Mismatched Inference Problem

by Francesco Camilli, Pierluigi Contucci and Emanuele Mingione

Entropy 2024, 26(1), 42; https://doi.org/10.3390/e26010042 - 30 Dec 2023

Cited by 2 | Viewed by 1626

We show that a statistical mechanics model where both the Sherringhton–Kirkpatrick and Hopfield Hamiltonians appear, which is equivalent to a high-dimensional mismatched inference problem, is described by a replica symmetry-breaking Parisi solution. Full article

(This article belongs to the Special Issue A Journey Through Complex Landscapes—Dedicated to Professor Giorgio Parisi to Celebrate the Nobel Prize & His 75th Birthday)

23 pages, 325 KiB

Open AccessEditor’s ChoiceArticle

Exact Response Theory for Time-Dependent and Stochastic Perturbations

by Leonardo Iannella and Lamberto Rondoni

Entropy 2024, 26(1), 12; https://doi.org/10.3390/e26010012 - 21 Dec 2023

Cited by 1 | Viewed by 1599

Abstract

The exact, non perturbative, response theory developed within the field of non-equilibrium molecular dynamics, also known as TTCF (transient time correlation function), applies to quite general dynamical systems. Its key element is called the dissipation function because it represents the power dissipated by [...] Read more.

The exact, non perturbative, response theory developed within the field of non-equilibrium molecular dynamics, also known as TTCF (transient time correlation function), applies to quite general dynamical systems. Its key element is called the dissipation function because it represents the power dissipated by external fields acting on the particle system of interest, whose coupling with the environment is given by deterministic thermostats. This theory has been initially developed for time-independent external perturbations, and then it has been extended to time-dependent perturbations. It has also been applied to dynamical systems of different nature, and to oscillator models undergoing phase transitions, which cannot be treated with, e.g., linear response theory. The present work includes time-dependent stochastic perturbations in the theory using the Karhunen–Loève theorem. This leads to three different investigations of a given process. In the first, a single realization of the stochastic coefficients is fixed, and averages are taken only over the initial conditions, as in a deterministic process. In the second, the initial condition is fixed, and averages are taken with respect to the distribution of stochastic coefficients. In the last investigation, one averages over both initial conditions and stochastic coefficients. We conclude by illustrating the applicability of the resulting exact response theory with simple examples. Full article

(This article belongs to the Section Non-equilibrium Phenomena)

36 pages, 2377 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

A Macroscopic Quantum Three-Box Paradox: Finding Consistency with Weak Macroscopic Realism

by Channa Hatharasinghe, Manushan Thenabadu, Peter D. Drummond and Margaret D. Reid

Entropy 2023, 25(12), 1620; https://doi.org/10.3390/e25121620 - 4 Dec 2023

Cited by 5 | Viewed by 2135

Abstract

The quantum three-box paradox considers a ball prepared in a superposition of being in any one of three boxes. Bob makes measurements by opening either box 1 or box 2. After performing some unitary operations (shuffling), Alice can infer with certainty that the [...] Read more.

The quantum three-box paradox considers a ball prepared in a superposition of being in any one of three boxes. Bob makes measurements by opening either box 1 or box 2. After performing some unitary operations (shuffling), Alice can infer with certainty that the ball was detected by Bob, regardless of which box he opened, if she detects the ball after opening box 3. The paradox is that the ball would have been found with certainty by Bob in either box if that box had been opened. Resolutions of the paradox include that Bob’s measurement cannot be made non-invasively or else that realism cannot be assumed at the quantum level. Here, we strengthen the case for the former argument by constructing macroscopic versions of the paradox. Macroscopic realism implies that the ball is in one of the boxes prior to Bob or Alice opening any boxes. We demonstrate the consistency of the paradox with macroscopic realism, if carefully defined (as weak macroscopic realism, wMR) to apply to the system at the times prior to Alice or Bob opening any boxes but after the unitary operations associated with preparation or shuffling. By solving for the dynamics of the unitary operations and comparing with mixed states, we demonstrate agreement between the predictions of wMR and quantum mechanics: the paradox only manifests if Alice’s shuffling combines both local operations (on box 3) and nonlocal operations, on the other boxes. Following previous work, the macroscopic paradox is shown to correspond to a violation of a Leggett–Garg inequality, which implies failure of non-invasive measurability if wMR holds. Full article

(This article belongs to the Special Issue Quantum Correlations, Contextuality, and Quantum Nonlocality)

► Show Figures

Figure 1

11 pages, 549 KiB

Open AccessEditor’s ChoiceArticle

Parity-Time Symmetric Holographic Principle

by Xingrui Song and Kater Murch

Entropy 2023, 25(11), 1523; https://doi.org/10.3390/e25111523 - 7 Nov 2023

Viewed by 1970

Abstract

Originating from the Hamiltonian of a single qubit system, the phenomenon of the avoided level crossing is ubiquitous in multiple branches of physics, including the Landau–Zener transition in atomic, molecular, and optical physics, the band structure of condensed matter physics and the dispersion [...] Read more.

Originating from the Hamiltonian of a single qubit system, the phenomenon of the avoided level crossing is ubiquitous in multiple branches of physics, including the Landau–Zener transition in atomic, molecular, and optical physics, the band structure of condensed matter physics and the dispersion relation of relativistic quantum physics. We revisit this fundamental phenomenon in the simple example of a spinless relativistic quantum particle traveling in (1+1)-dimensional space-time and establish its relation to a spin-1/2 system evolving under a

PT

-symmetric Hamiltonian. This relation allows us to simulate 1-dimensional eigenvalue problems with a single qubit. Generalizing this relation to the eigenenergy problem of a bulk system with N spatial dimensions reveals that its eigenvalue problem can be mapped onto the time evolution of the edge state with

(N - 1)

spatial dimensions governed by a non-Hermitian Hamiltonian. In other words, the bulk eigenenergy state is encoded in the edge state as a hologram, which can be decoded by the propagation of the edge state in the temporal dimension. We argue that the evolution will be

PT

-symmetric as long as the bulk system admits parity symmetry. Our work finds the application of

PT

-symmetric and non-Hermitian physics in quantum simulation and provides insights into the fundamental symmetries. Full article

(This article belongs to the Special Issue Quantum Dynamics with Non-hermitian Hamiltonians II)

► Show Figures

Figure 1

33 pages, 627 KiB

Open AccessEditor’s ChoiceArticle

Energy-Limited Joint Source–Channel Coding of Gaussian Sources over Gaussian Channels with Unknown Noise Level

by Omri Lev and Anatoly Khina

Entropy 2023, 25(11), 1522; https://doi.org/10.3390/e25111522 - 6 Nov 2023

Cited by 1 | Viewed by 1546

Abstract

We consider the problem of transmitting a Gaussian source with minimum mean square error distortion over an infinite-bandwidth additive white Gaussian noise channel with an unknown noise level and under an input energy constraint. We construct a universal joint source–channel coding scheme with [...] Read more.

We consider the problem of transmitting a Gaussian source with minimum mean square error distortion over an infinite-bandwidth additive white Gaussian noise channel with an unknown noise level and under an input energy constraint. We construct a universal joint source–channel coding scheme with respect to the noise level, that uses modulo-lattice modulation with multiple layers. For each layer, we employ either analog linear modulation or analog pulse-position modulation (PPM). We show that the designed scheme with linear layers requires less energy compared to existing solutions to achieve the same quadratically increasing distortion profile with the noise level; replacing the linear layers with PPM layers offers an additional improvement. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory II)

► Show Figures

Figure 1

10 pages, 556 KiB

Open AccessEditor’s ChoiceArticle

Large Coherent States Formed from Disordered k-Regular Random Graphs

by Gregory D. Scholes

Entropy 2023, 25(11), 1519; https://doi.org/10.3390/e25111519 - 6 Nov 2023

Cited by 6 | Viewed by 1816

Abstract

The present work is motivated by the need for robust, large-scale coherent states that can play possible roles as quantum resources. A challenge is that large, complex systems tend to be fragile. However, emergent phenomena in classical systems tend to become more robust [...] Read more.

The present work is motivated by the need for robust, large-scale coherent states that can play possible roles as quantum resources. A challenge is that large, complex systems tend to be fragile. However, emergent phenomena in classical systems tend to become more robust with scale. Do these classical systems inspire ways to think about robust quantum networks? This question is studied by characterizing the complex quantum states produced by mapping interactions between a set of qubits from structure in graphs. We focus on maps based on k-regular random graphs where many edges were randomly deleted. We ask how many edge deletions can be tolerated. Surprisingly, it was found that the emergent coherent state characteristic of these graphs was robust to a substantial number of edge deletions. The analysis considers the possible role of the expander property of k-regular random graphs. Full article

(This article belongs to the Special Issue Quantum Information and Probability: From Foundations to Engineering II)

► Show Figures

Figure 1

32 pages, 660 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Smoothing of Binary Codes, Uniform Distributions, and Applications

by Madhura Pathegama and Alexander Barg

Entropy 2023, 25(11), 1515; https://doi.org/10.3390/e25111515 - 5 Nov 2023

Cited by 5 | Viewed by 1964

Abstract

The action of a noise operator on a code transforms it into a distribution on the respective space. Some common examples from information theory include Bernoulli noise acting on a code in the Hamming space and Gaussian noise acting on a lattice in [...] Read more.

The action of a noise operator on a code transforms it into a distribution on the respective space. Some common examples from information theory include Bernoulli noise acting on a code in the Hamming space and Gaussian noise acting on a lattice in the Euclidean space. We aim to characterize the cases when the output distribution is close to the uniform distribution on the space, as measured by the Rényi divergence of order

α \in (1, \infty]

. A version of this question is known as the channel resolvability problem in information theory, and it has implications for security guarantees in wiretap channels, error correction, discrepancy, worst-to-average case complexity reductions, and many other problems. Our work quantifies the requirements for asymptotic uniformity (perfect smoothing) and identifies explicit code families that achieve it under the action of the Bernoulli and ball noise operators on the code. We derive expressions for the minimum rate of codes required to attain asymptotically perfect smoothing. In proving our results, we leverage recent results from harmonic analysis of functions on the Hamming space. Another result pertains to the use of code families in Wyner’s transmission scheme on the binary wiretap channel. We identify explicit families that guarantee strong secrecy when applied in this scheme, showing that nested Reed–Muller codes can transmit messages reliably and securely over a binary symmetric wiretap channel with a positive rate. Finally, we establish a connection between smoothing and error correction in the binary symmetric channel. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

► Show Figures

Figure 1

21 pages, 392 KiB

Open AccessEditor’s ChoiceArticle

Testing the Minimum System Entropy and the Quantum of Entropy

by Uwe Hohm and Christoph Schiller

Entropy 2023, 25(11), 1511; https://doi.org/10.3390/e25111511 - 3 Nov 2023

Cited by 1 | Viewed by 3475

Abstract

Experimental and theoretical results about entropy limits for macroscopic and single-particle systems are reviewed. All experiments confirm the minimum system entropy

S ⩾ k ln 2

. We clarify in which cases it is possible to speak about a minimum system entropy [...] Read more.

Experimental and theoretical results about entropy limits for macroscopic and single-particle systems are reviewed. All experiments confirm the minimum system entropy

S ⩾ k ln 2

. We clarify in which cases it is possible to speak about a minimum system entropy

k ln 2

and in which cases about a quantum of entropy. Conceptual tensions with the third law of thermodynamics, with the additivity of entropy, with statistical calculations, and with entropy production are resolved. Black hole entropy is surveyed. Claims for smaller system entropy values are shown to contradict the requirement of observability, which, as possibly argued for the first time here, also implies the minimum system entropy

k ln 2

. The uncertainty relations involving the Boltzmann constant and the possibility of deriving thermodynamics from the existence of minimum system entropy enable one to speak about a general principle that is valid across nature. Full article

(This article belongs to the Special Issue 200 Years Anniversary of “Sadi Carnot, Réflexions Sur La Puissance Motrice Du Feu”; Bachelier: Paris, France, 1824)

37 pages, 543 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Variable-Length Resolvability for General Sources and Channels

by Hideki Yagi and Te Sun Han

Entropy 2023, 25(10), 1466; https://doi.org/10.3390/e25101466 - 19 Oct 2023

Cited by 1 | Viewed by 1416

Abstract

We introduce the problem of variable-length (VL) source resolvability, in which a given target probability distribution is approximated by encoding a VL uniform random number, and the asymptotically minimum average length rate of the uniform random number, called the VL resolvability, is investigated. [...] Read more.

We introduce the problem of variable-length (VL) source resolvability, in which a given target probability distribution is approximated by encoding a VL uniform random number, and the asymptotically minimum average length rate of the uniform random number, called the VL resolvability, is investigated. We first analyze the VL resolvability with the variational distance as an approximation measure. Next, we investigate the case under the divergence as an approximation measure. When the asymptotically exact approximation is required, it is shown that the resolvability under two kinds of approximation measures coincides. We then extend the analysis to the case of channel resolvability, where the target distribution is the output distribution via a general channel due to a fixed general source as an input. The obtained characterization of channel resolvability is fully general in the sense that, when the channel is just an identity mapping, it reduces to general formulas for source resolvability. We also analyze the second-order VL resolvability. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory II)

► Show Figures

Figure 1

12 pages, 762 KiB

Open AccessEditor’s ChoiceArticle

Pre-Configured Error Pattern Ordered Statistics Decoding for CRC-Polar Codes

by Xuanyu Li, Kai Niu, Yuxin Han, Jincheng Dai, Zhiyuan Tan and Zhiheng Guo

Entropy 2023, 25(10), 1405; https://doi.org/10.3390/e25101405 - 30 Sep 2023

Cited by 1 | Viewed by 1539

Abstract

In this paper, we propose a pre-configured error pattern ordered statistics decoding (PEPOSD) algorithm and discuss its application to short cyclic redundancy check (CRC)-polar codes. Unlike the traditional OSD that changes the most reliable independent symbols, we regard the decoding process as testing [...] Read more.

In this paper, we propose a pre-configured error pattern ordered statistics decoding (PEPOSD) algorithm and discuss its application to short cyclic redundancy check (CRC)-polar codes. Unlike the traditional OSD that changes the most reliable independent symbols, we regard the decoding process as testing the error patterns, like guessing random additive noise decoding (GRAND). Also, the pre-configurator referred from ordered reliability bits (ORB) GRAND can better control the range and testing order of EPs. An offline–online structure can accelerate the decoding process. Additionally, we also introduce two orders to optimize the search order for testing EPs. Compared with CRC-aided OSD and list decoding, PEPOSD can achieve a better trade-off between accuracy and complexity. Full article

(This article belongs to the Special Issue Information Theory and Coding for Wireless Communications II)

► Show Figures

Figure 1

53 pages, 15682 KiB

Open AccessEditor’s ChoiceReview

Non-Hermitian Floquet Topological Matter—A Review

by Longwen Zhou and Da-Jian Zhang

Entropy 2023, 25(10), 1401; https://doi.org/10.3390/e25101401 - 29 Sep 2023

Cited by 34 | Viewed by 5069

Abstract

The past few years have witnessed a surge of interest in non-Hermitian Floquet topological matter due to its exotic properties resulting from the interplay between driving fields and non-Hermiticity. The present review sums up our studies on non-Hermitian Floquet topological matter in one [...] Read more.

The past few years have witnessed a surge of interest in non-Hermitian Floquet topological matter due to its exotic properties resulting from the interplay between driving fields and non-Hermiticity. The present review sums up our studies on non-Hermitian Floquet topological matter in one and two spatial dimensions. We first give a bird’s-eye view of the literature for clarifying the physical significance of non-Hermitian Floquet systems. We then introduce, in a pedagogical manner, a number of useful tools tailored for the study of non-Hermitian Floquet systems and their topological properties. With the aid of these tools, we present typical examples of non-Hermitian Floquet topological insulators, superconductors, and quasicrystals, with a focus on their topological invariants, bulk-edge correspondences, non-Hermitian skin effects, dynamical properties, and localization transitions. We conclude this review by summarizing our main findings and presenting our vision of future directions. Full article

(This article belongs to the Section Entropy Reviews)

► Show Figures

Figure 1

19 pages, 4974 KiB

Open AccessEditor’s ChoiceArticle

Complexity Synchronization of Organ Networks

by Bruce J. West, Paolo Grigolini, Scott E. Kerick, Piotr J. Franaszczuk and Korosh Mahmoodi

Entropy 2023, 25(10), 1393; https://doi.org/10.3390/e25101393 - 28 Sep 2023

Cited by 11 | Viewed by 2239

Abstract

The transdisciplinary nature of science as a whole became evident as the necessity for the complex nature of phenomena to explain social and life science, along with the physical sciences, blossomed into complexity theory and most recently into complexitysynchronization. This science motif is [...] Read more.

The transdisciplinary nature of science as a whole became evident as the necessity for the complex nature of phenomena to explain social and life science, along with the physical sciences, blossomed into complexity theory and most recently into complexitysynchronization. This science motif is based on the scaling arising from the 1/f-variability in complex dynamic networks and the need for a network of networks to exchange information internally during intra-network dynamics and externally during inter-network dynamics. The measure of complexity adopted herein is the multifractal dimension of the crucial event time series generated by an organ network, and the difference in the multifractal dimensions of two organ networks quantifies the relative complexity between interacting complex networks. Information flows from dynamic networks at a higher level of complexity to those at lower levels of complexity, as summarized in the ‘complexity matching effect’, and the flow is maximally efficient when the complexities are equal. Herein, we use the scaling of empirical datasets from the brain, cardiovascular and respiratory networks to support the hypothesis that complexity synchronization occurs between scaling indices or equivalently with the matching of the time dependencies of the networks’ multifractal dimensions. Full article

(This article belongs to the Special Issue Fractional Calculus and Fractional Dynamics)

► Show Figures

Figure 1

15 pages, 2968 KiB

Open AccessEditor’s ChoiceArticle

Accelerating Quantum Decay by Multiple Tunneling Barriers

by Ermanno Pinotti and Stefano Longhi

Entropy 2023, 25(9), 1345; https://doi.org/10.3390/e25091345 - 16 Sep 2023

Cited by 1 | Viewed by 2439

Abstract

A quantum particle constrained between two high potential barriers provides a paradigmatic example of a system sustaining quasi-bound (or resonance) states. When the system is prepared in one of such quasi-bound states, the wave function approximately maintains its shape but decays in time [...] Read more.

A quantum particle constrained between two high potential barriers provides a paradigmatic example of a system sustaining quasi-bound (or resonance) states. When the system is prepared in one of such quasi-bound states, the wave function approximately maintains its shape but decays in time in a nearly exponential manner radiating into the surrounding space, the lifetime being of the order of the reciprocal of the width of the resonance peak in the transmission spectrum. Naively, one could think that adding more lateral barriers would preferentially slow down or prevent the quantum decay since tunneling is expected to become less probable and due to quantum backflow induced by multiple scattering processes. However, this is not always the case and in the early stage of the dynamics quantum decay can be accelerated (rather than decelerated) by additional lateral barriers, even when the barrier heights are arbitrarily large. The decay acceleration originates from resonant tunneling effects and is associated to large deviations from an exponential decay law. We discuss such a counterintuitive phenomenon by considering the hopping dynamics of a quantum particle on a tight-binding lattice with on-site potential barriers. Full article

(This article belongs to the Special Issue Tunneling in Complex Systems)

► Show Figures

Figure 1

27 pages, 509 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Joint Detection and Communication over Type-Sensitive Networks

by Joni Shaska and Urbashi Mitra

Entropy 2023, 25(9), 1313; https://doi.org/10.3390/e25091313 - 8 Sep 2023

Viewed by 1315

Abstract

Due to the difficulty of decentralized inference with conditional dependent observations, and motivated by large-scale heterogeneous networks, we formulate a framework for decentralized detection with coupled observations. Each agent has a state, and the empirical distribution of all agents’ states or the type [...] Read more.

Due to the difficulty of decentralized inference with conditional dependent observations, and motivated by large-scale heterogeneous networks, we formulate a framework for decentralized detection with coupled observations. Each agent has a state, and the empirical distribution of all agents’ states or the type of network dictates the individual agents’ behavior. In particular, agents’ observations depend on both the underlying hypothesis as well as the empirical distribution of the agents’ states. Hence, our framework captures a high degree of coupling, in that an individual agent’s behavior depends on both the underlying hypothesis and the behavior of all other agents in the network. Considering this framework, the method of types, and a series of equicontinuity arguments, we derive the error exponent for the case in which all agents are identical and show that this error exponent depends on only a single empirical distribution. The analysis is extended to the multi-class case, and numerical results with state-dependent agent signaling and state-dependent channels highlight the utility of the proposed framework for analysis of highly coupled environments. Full article

(This article belongs to the Collection Feature Papers in Information Theory)

► Show Figures

Figure 1

15 pages, 343 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Assisted Identification over Modulo-Additive Noise Channels

by Amos Lapidoth and Baohua Ni

Entropy 2023, 25(9), 1314; https://doi.org/10.3390/e25091314 - 8 Sep 2023

Viewed by 1369

Abstract

The gain in the identification capacity afforded by a rate-limited description of the noise sequence corrupting a modulo-additive noise channel is studied. Both the classical Ahlswede–Dueck version and the Ahlswede–Cai–Ning–Zhang version, which does not allow for missed identifications, are studied. Irrespective of whether [...] Read more.

The gain in the identification capacity afforded by a rate-limited description of the noise sequence corrupting a modulo-additive noise channel is studied. Both the classical Ahlswede–Dueck version and the Ahlswede–Cai–Ning–Zhang version, which does not allow for missed identifications, are studied. Irrespective of whether the description is provided to the receiver, to the transmitter, or to both, the two capacities coincide and both equal the helper-assisted Shannon capacity. Full article

(This article belongs to the Special Issue Extremal and Additive Combinatorial Aspects in Information Theory)

12 pages, 690 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Fundamental Relation for Gas of Interacting Particles in a Heat Flow

by Robert Hołyst, Karol Makuch, Konrad Giżyński, Anna Maciołek and Paweł J. Żuk

Entropy 2023, 25(9), 1295; https://doi.org/10.3390/e25091295 - 4 Sep 2023

Cited by 8 | Viewed by 2159

Abstract

There is a long-standing question of whether it is possible to extend the formalism of equilibrium thermodynamics to the case of nonequilibrium systems in steady-states. We have made such an extension for an ideal gas in a heat flow. Here, we investigated whether [...] Read more.

There is a long-standing question of whether it is possible to extend the formalism of equilibrium thermodynamics to the case of nonequilibrium systems in steady-states. We have made such an extension for an ideal gas in a heat flow. Here, we investigated whether such a description exists for the system with interactions: the van der Waals gas in a heat flow. We introduced a steady-state fundamental relation and the parameters of state, each associated with a single way of changing energy. The first law of nonequilibrium thermodynamics follows from these parameters. The internal energy U for the nonequilibrium states has the same form as in equilibrium thermodynamics. For the van der Waals gas,

U (S^{*}, V, N, a^{*}, b^{*})

is a function of only five parameters of state (irrespective of the number of parameters characterizing the boundary conditions): the effective entropy

S^{*}

, volume V, number of particles N, and rescaled van der Waals parameters

a^{*}

,

b^{*}

. The state parameters,

a^{*}

,

b^{*}

, together with

S^{*}

, determine the net heat exchange with the environment. The net heat differential does not have an integrating factor. As in equilibrium thermodynamics, the steady-state fundamental equation also leads to the thermodynamic Maxwell relations for measurable steady-state properties. Full article

(This article belongs to the Special Issue Entropy Production and Nonequilibrium Thermodynamics in Materials)

► Show Figures

Figure 1

37 pages, 1441 KiB

Open AccessEditor’s ChoiceReview

The Entropy of Entropy: Are We Talking about the Same Thing?

by Søren Nors Nielsen and Felix Müller

Entropy 2023, 25(9), 1288; https://doi.org/10.3390/e25091288 - 1 Sep 2023

Cited by 5 | Viewed by 4789

Abstract

In the last few decades, the number of published papers that include search terms such as thermodynamics, entropy, ecology, and ecosystems has grown rapidly. Recently, background research carried out during the development of a paper on “thermodynamics in ecology” revealed huge variation in [...] Read more.

In the last few decades, the number of published papers that include search terms such as thermodynamics, entropy, ecology, and ecosystems has grown rapidly. Recently, background research carried out during the development of a paper on “thermodynamics in ecology” revealed huge variation in the understanding of the meaning and the use of some of the central terms in this field—in particular, entropy. This variation seems to be based primarily on the differing educational and scientific backgrounds of the researchers responsible for contributions to this field. Secondly, some ecological subdisciplines also seem to be better suited and applicable to certain interpretations of the concept than others. The most well-known seems to be the use of the Boltzmann–Gibbs equation in the guise of the Shannon–Weaver/Wiener index when applied to the estimation of biodiversity in ecology. Thirdly, this tendency also revealed that the use of entropy-like functions could be diverted into an area of statistical and distributional analyses as opposed to real thermodynamic approaches, which explicitly aim to describe and account for the energy fluxes and dissipations in the systems. Fourthly, these different ways of usage contribute to an increased confusion in discussions about efficiency and possible telos in nature, whether at the developmental level of the organism, a population, or an entire ecosystem. All the papers, in general, suffer from a lack of clear definitions of the thermodynamic functions used, and we, therefore, recommend that future publications in this area endeavor to achieve a more precise use of language. Only by increasing such efforts it is possible to understand and resolve some of the significant and possibly misleading discussions in this area. Full article

(This article belongs to the Special Issue Entropy in Biological Systems)

► Show Figures

Figure 1

13 pages, 397 KiB

Open AccessEditor’s ChoiceArticle

Quantum-Walk-Inspired Dynamic Adiabatic Local Search

by Chen-Fu Chiang and Paul M. Alsing

Entropy 2023, 25(9), 1287; https://doi.org/10.3390/e25091287 - 31 Aug 2023

Cited by 1 | Viewed by 1634

Abstract

We investigate the irreconcilability issue that arises when translating the search algorithm from the Continuous Time Quantum Walk (CTQW) framework to the Adiabatic Quantum Computing (AQC) framework. For the AQC formulation to evolve along the same path as the CTQW, it requires a [...] Read more.

We investigate the irreconcilability issue that arises when translating the search algorithm from the Continuous Time Quantum Walk (CTQW) framework to the Adiabatic Quantum Computing (AQC) framework. For the AQC formulation to evolve along the same path as the CTQW, it requires a constant energy gap in the Hamiltonian throughout the AQC schedule. To resolve the constant gap issue, we modify the CTQW-inspired AQC catalyst Hamiltonian from an XZ operator to a Z oracle operator. Through simulation, we demonstrate that the total running time for the proposed approach for AQC with the modified catalyst Hamiltonian remains optimal as CTQW. Inspired by this solution, we further investigate adaptive scheduling for the catalyst Hamiltonian and its coefficient function in the adiabatic path of Grover-inspired AQC to improve the adiabatic local search. Full article

(This article belongs to the Special Issue Advances in Quantum Computing)

► Show Figures

Figure 1

21 pages, 465 KiB

Open AccessEditor’s ChoiceArticle

Generalized Bell Scenarios: Disturbing Consequences on Local-Hidden-Variable Models

by André Mazzari, Gabriel Ruffolo, Carlos Vieira, Tassius Temistocles, Rafael Rabelo and Marcelo Terra Cunha

Entropy 2023, 25(9), 1276; https://doi.org/10.3390/e25091276 - 30 Aug 2023

Cited by 1 | Viewed by 1607

Abstract

Bell nonlocality and Kochen–Specker contextuality are among the main topics in the foundations of quantum theory. Both of them are related to stronger-than-classical correlations, with the former usually referring to spatially separated systems, while the latter considers a single system. In recent works, [...] Read more.

Bell nonlocality and Kochen–Specker contextuality are among the main topics in the foundations of quantum theory. Both of them are related to stronger-than-classical correlations, with the former usually referring to spatially separated systems, while the latter considers a single system. In recent works, a unified framework for these phenomena was presented. This article reviews, expands, and obtains new results regarding this framework. Contextual and disturbing features inside the local models are explored, which allows for the definition of different local sets with a non-trivial relation among them. The relations between the set of quantum correlations and these local sets are also considered, and post-quantum local behaviours are found. Moreover, examples of correlations that are both local and non-contextual but such that these two classical features cannot be expressed by the same hidden variable model are shown. Extensions of the Fine–Abramsky–Brandenburger theorem are also discussed. Full article

(This article belongs to the Special Issue Quantum Correlations, Contextuality, and Quantum Nonlocality)

► Show Figures

Figure 1

15 pages, 499 KiB

Open AccessEditor’s ChoiceArticle

Investigation of Feature Engineering Methods for Domain-Knowledge-Assisted Bearing Fault Diagnosis

by Christoph Bienefeld, Florian Michael Becker-Dombrowsky, Etnik Shatri and Eckhard Kirchner

Entropy 2023, 25(9), 1278; https://doi.org/10.3390/e25091278 - 30 Aug 2023

Cited by 5 | Viewed by 2071

Abstract

The engineering challenge of rolling bearing condition monitoring has led to a large number of method developments over the past few years. Most commonly, vibration measurement data are used for fault diagnosis using machine learning algorithms. In current research, purely data-driven deep learning [...] Read more.

The engineering challenge of rolling bearing condition monitoring has led to a large number of method developments over the past few years. Most commonly, vibration measurement data are used for fault diagnosis using machine learning algorithms. In current research, purely data-driven deep learning methods are becoming increasingly popular, aiming for accurate predictions of bearing faults without requiring bearing-specific domain knowledge. Opposing this trend in popularity, the present paper takes a more traditional approach, incorporating domain knowledge by evaluating a variety of feature engineering methods in combination with a random forest classifier. For a comprehensive feature engineering study, a total of 42 mathematical feature formulas are combined with the preprocessing methods of envelope analysis, empirical mode decomposition, wavelet transforms, and frequency band separations. While each single processing method and feature formula is known from the literature, the presented paper contributes to the body of knowledge by investigating novel series connections of processing methods and feature formulas. Using the CWRU bearing fault data for performance evaluation, feature calculation based on the processing method of frequency band separation leads to particularly high prediction accuracies, while at the same time being very efficient in terms of low computational effort. Additionally, in comparison with deep learning approaches, the proposed feature engineering method provides excellent accuracies and enables explainability. Full article

(This article belongs to the Special Issue New Trends in Fault Diagnosis and Prognosis for Engineering Applications: From Signal Processing to Machine Learning and Deep Learning)

► Show Figures

Figure 1

21 pages, 422 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Non-Local Vectorial Internal Variables and Generalized Guyer-Krumhansl Evolution Equations for the Heat Flux

by Liliana Restuccia and David Jou

Entropy 2023, 25(9), 1259; https://doi.org/10.3390/e25091259 - 24 Aug 2023

Cited by 2 | Viewed by 1333

Abstract

In this paper, we ask ourselves how non-local effects affect the description of thermodynamic systems with internal variables. Usually, one assumes that the internal variables are local, but that their evolution equations are non-local, i.e., for instance, that their evolution equations contain non-local [...] Read more.

In this paper, we ask ourselves how non-local effects affect the description of thermodynamic systems with internal variables. Usually, one assumes that the internal variables are local, but that their evolution equations are non-local, i.e., for instance, that their evolution equations contain non-local differential terms (gradients, Laplacians) or integral terms with memory kernels. In contrast to this typical situation, which has led to substantial progress in several fields, we ask ourselves whether in some cases it would be convenient to start from non-local internal variables with non-local evolution equations. We examine this point by considering three main lengths: the observation scale R defining the elementary volumes used in the description of the system, the mean free path l of the microscopic elements of the fluid (particles, phonons, photons, and molecules), and the overall characteristic size L of the global system. We illustrate these ideas by considering three-dimensional rigid heat conductors within the regime of phonon hydrodynamics in the presence of thermal vortices. In particular, we obtain a generalization of the Guyer–Krumhansl equation, which may be of interest for heat transport in nanosystems or in systems with small-scale inhomogeneities. Full article

(This article belongs to the Special Issue Thermodynamic Constitutive Theory and Its Application)

17 pages, 887 KiB

Open AccessEditor’s ChoiceArticle

Emulating Non-Hermitian Dynamics in a Finite Non-Dissipative Quantum System

by Eloi Flament, François Impens and David Guéry-Odelin

Entropy 2023, 25(9), 1256; https://doi.org/10.3390/e25091256 - 24 Aug 2023

Viewed by 1981

Abstract

We discuss the emulation of non-Hermitian dynamics during a given time window using a low-dimensional quantum system coupled to a finite set of equidistant discrete states acting as an effective continuum. We first emulate the decay of an unstable state and map the [...] Read more.

We discuss the emulation of non-Hermitian dynamics during a given time window using a low-dimensional quantum system coupled to a finite set of equidistant discrete states acting as an effective continuum. We first emulate the decay of an unstable state and map the quasi-continuum parameters, enabling the precise approximation of non-Hermitian dynamics. The limitations of this model, including in particular short- and long-time deviations, are extensively discussed. We then consider a driven two-level system and establish criteria for non-Hermitian dynamics emulation with a finite quasi-continuum. We quantitatively analyze the signatures of the finiteness of the effective continuum, addressing the possible emergence of non-Markovian behavior during the time interval considered. Finally, we investigate the emulation of dissipative dynamics using a finite quasi-continuum with a tailored density of states. We show through the example of a two-level system that such a continuum can reproduce non-Hermitian dynamics more efficiently than the usual equidistant quasi-continuum model. Full article

(This article belongs to the Special Issue Dynamics of Open Quantum Systems: Quantum Fluctuations, Decoherence and Emergent Phenomena)

► Show Figures

Figure 1

10 pages, 299 KiB

Open AccessEditor’s ChoiceArticle

A Maximum Entropy Resolution to the Wine/Water Paradox

by Michael C. Parker and Chris Jeynes

Entropy 2023, 25(8), 1242; https://doi.org/10.3390/e25081242 - 21 Aug 2023

Cited by 2 | Viewed by 1764

Abstract

The Principle of Indifference (‘PI’: the simplest non-informative prior in Bayesian probability) has been shown to lead to paradoxes since Bertrand (1889). Von Mises (1928) introduced the ‘Wine/Water Paradox’ as a resonant example of a ‘Bertrand paradox’, which has been presented as demonstrating [...] Read more.

The Principle of Indifference (‘PI’: the simplest non-informative prior in Bayesian probability) has been shown to lead to paradoxes since Bertrand (1889). Von Mises (1928) introduced the ‘Wine/Water Paradox’ as a resonant example of a ‘Bertrand paradox’, which has been presented as demonstrating that the PI must be rejected. We now resolve these paradoxes using a Maximum Entropy (MaxEnt) treatment of the PI that also includes information provided by Benford’s ‘Law of Anomalous Numbers’ (1938). We show that the PI should be understood to represent a family of informationally identical MaxEnt solutions, each solution being identified with its own explicitly justified boundary condition. In particular, our solution to the Wine/Water Paradox exploits Benford’s Law to construct a non-uniform distribution representing the universal constraint of scale invariance, which is a physical consequence of the Second Law of Thermodynamics. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

17 pages, 1695 KiB

Open AccessEditor’s ChoiceArticle

First Detection and Tunneling Time of a Quantum Walk

by Zhenbo Ni and Yujun Zheng

Entropy 2023, 25(8), 1231; https://doi.org/10.3390/e25081231 - 18 Aug 2023

Cited by 3 | Viewed by 1598

Abstract

We consider the first detection problem for a one-dimensional quantum walk with repeated local measurements. Employing the stroboscopic projective measurement protocol and the renewal equation, we study the effect of tunneling on the detection time. Specifically, we study the continuous-time quantum walk on [...] Read more.

We consider the first detection problem for a one-dimensional quantum walk with repeated local measurements. Employing the stroboscopic projective measurement protocol and the renewal equation, we study the effect of tunneling on the detection time. Specifically, we study the continuous-time quantum walk on an infinite tight-binding lattice for two typical situations with physical reality. The first is the case of a quantum walk in the absence of tunneling with a Gaussian initial state. The second is the case where a barrier is added to the system. It is shown that the transition of the decay behavior of the first detection probability can be observed by modifying the initial condition, and in the presence of a tunneling barrier, the particle can be detected earlier than the impurity-free lattice. This suggests that the evolution of the walker is expedited when it tunnels through the barrier under repeated measurement. The first detection tunneling time is introduced to investigate the tunneling time of the quantum walk. In addition, we analyze the critical transitive point by deriving an asymptotic formula. Full article

(This article belongs to the Section Quantum Information)

► Show Figures

Figure 1

23 pages, 1111 KiB

Open AccessEditor’s ChoiceArticle

Students’ Learning Behaviour in Programming Education Analysis: Insights from Entropy and Community Detection

by Tai Tan Mai, Martin Crane and Marija Bezbradica

Entropy 2023, 25(8), 1225; https://doi.org/10.3390/e25081225 - 17 Aug 2023

Cited by 7 | Viewed by 3275

Abstract

The high dropout rates in programming courses emphasise the need for monitoring and understanding student engagement, enabling early interventions. This activity can be supported by insights into students’ learning behaviours and their relationship with academic performance, derived from student learning log data in [...] Read more.

The high dropout rates in programming courses emphasise the need for monitoring and understanding student engagement, enabling early interventions. This activity can be supported by insights into students’ learning behaviours and their relationship with academic performance, derived from student learning log data in learning management systems. However, the high dimensionality of such data, along with their numerous features, pose challenges to their analysis and interpretability. In this study, we introduce entropy-based metrics as a novel manner to represent students’ learning behaviours. Employing these metrics, in conjunction with a proven community detection method, we undertake an analysis of learning behaviours across higher- and lower-performing student communities. Furthermore, we examine the impact of the COVID-19 pandemic on these behaviours. The study is grounded in the analysis of empirical data from 391 Software Engineering students over three academic years. Our findings reveal that students in higher-performing communities typically tend to have lower volatility in entropy values and reach stable learning states earlier than their lower-performing counterparts. Importantly, this study provides evidence of the use of entropy as a simple yet insightful metric for educators to monitor study progress, enhance understanding of student engagement, and enable timely interventions. Full article

(This article belongs to the Special Issue Entropy-Based Applications in Sociophysics)

► Show Figures

Figure 1

16 pages, 326 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

A Universal Random Coding Ensemble for Sample-Wise Lossy Compression

by Neri Merhav

Entropy 2023, 25(8), 1199; https://doi.org/10.3390/e25081199 - 11 Aug 2023

Cited by 6 | Viewed by 1316

Abstract

We propose a universal ensemble for the random selection of rate–distortion codes which is asymptotically optimal in a sample-wise sense. According to this ensemble, each reproduction vector,

\hat{x}

, is selected independently at random under the probability distribution that is proportional to [...] Read more.

We propose a universal ensemble for the random selection of rate–distortion codes which is asymptotically optimal in a sample-wise sense. According to this ensemble, each reproduction vector,

\hat{x}

, is selected independently at random under the probability distribution that is proportional to

2^{- L Z (\hat{x})}

, where

L Z (\hat{x})

is the code length of

\hat{x}

pertaining to the 1978 version of the Lempel–Ziv (LZ) algorithm. We show that, with high probability, the resulting codebook gives rise to an asymptotically optimal variable-rate lossy compression scheme under an arbitrary distortion measure, in the sense that a matching converse theorem also holds. According to the converse theorem, even if the decoder knew the ℓ-th order type of source vector in advance (ℓ being a large but fixed positive integer), the performance of the above-mentioned code could not have been improved essentially for the vast majority of codewords pertaining to source vectors in the same type. Finally, we present a discussion of our results, which includes among other things, a clear indication that our coding scheme outperforms the one that selects the reproduction vector with the shortest LZ code length among all vectors that are within the allowed distortion from the source vector. Full article

(This article belongs to the Collection Feature Papers in Information Theory)

18 pages, 1066 KiB

Open AccessEditor’s ChoiceArticle

Hybrid Threshold Denoising Framework Using Singular Value Decomposition for Side-Channel Analysis Preprocessing

by Yuanzhen Wang, Hongxin Zhang, Xing Fang, Xiaotong Cui, Wenxu Ning, Danzhi Wang, Fan Fan and Lei Shu

Entropy 2023, 25(8), 1133; https://doi.org/10.3390/e25081133 - 28 Jul 2023

Cited by 3 | Viewed by 2099

Abstract

The traces used in side-channel analysis are essential to breaking the key of encryption and the signal quality greatly affects the correct rate of key guessing. Therefore, the preprocessing of side-channel traces plays an important role in side-channel analysis. The process of side-channel [...] Read more.

The traces used in side-channel analysis are essential to breaking the key of encryption and the signal quality greatly affects the correct rate of key guessing. Therefore, the preprocessing of side-channel traces plays an important role in side-channel analysis. The process of side-channel leakage signal acquisition is usually affected by internal circuit noise, external environmental noise, and other factors, so the collected signal is often mixed with strong noise. In order to extract the feature information of side-channel signals from very low signal-to-noise ratio traces, a hybrid threshold denoising framework using singular value decomposition is proposed for side-channel analysis preprocessing. This framework is based on singular value decomposition and introduces low-rank matrix approximation theory to improve the rank selection methods of singular value decomposition. This paper combines the hard threshold method of truncated singular value decomposition with the soft threshold method of singular value shrinkage damping and proposes a hybrid threshold denoising framework using singular value decomposition for the data preprocessing step of side-channel analysis as a general preprocessing method for non-profiled side-channel analysis. The data used in the experimental evaluation are from the raw traces of the public database of DPA contest V2 and AES_HD. The success rate curve of non-profiled side-channel analysis further confirms the effectiveness of the proposed framework. Moreover, the signal-to-noise ratio of traces is significantly improved after preprocessing, and the correlation with the correct key is also significantly enhanced. Experimental results on DPA v2 and AES_HD show that the proposed noise reduction framework can be effectively applied to the side-channel analysis preprocessing step, and can successfully improve the signal-to-noise ratio of the traces and the attack efficiency. Full article

(This article belongs to the Section Signal and Data Analysis)

► Show Figures

Figure 1

26 pages, 417 KiB

Open AccessEditor’s ChoiceArticle

Multiple Linear-Combination Security Network Coding

by Yang Bai, Xuan Guang and Raymond W. Yeung

Entropy 2023, 25(8), 1135; https://doi.org/10.3390/e25081135 - 28 Jul 2023

Cited by 1 | Viewed by 1634

Abstract

In this paper, we put forward the model of multiple linear-combination security multicast network coding, where the wiretapper desires to obtain some information about a predefined set of multiple linear combinations of the source symbols by eavesdropping any one (but not more than [...] Read more.

In this paper, we put forward the model of multiple linear-combination security multicast network coding, where the wiretapper desires to obtain some information about a predefined set of multiple linear combinations of the source symbols by eavesdropping any one (but not more than one) channel subset up to a certain size r, referred to as the security level. For this model, the security capacity is defined as the maximum average number of source symbols that can be securely multicast to all sink nodes for one use of the network under the linear-combination security constraint. For any security level and any linear-combination security constraint, we fully characterize the security capacity in terms of the ratio of the rank of the linear-combination security constraint to the number of source symbols. Also, we develop a general construction of linear security network codes. Finally, we investigate the asymptotic behavior of the security capacity for a sequence of linear-combination security models and discuss the asymptotic optimality of our code construction. Full article

(This article belongs to the Special Issue Information Theory and Network Coding II)

► Show Figures

Figure 1

17 pages, 4797 KiB

Open AccessEditor’s ChoiceArticle

Unstable Points, Ergodicity and Born’s Rule in 2d Bohmian Systems

by Athanasios C. Tzemos and George Contopoulos

Entropy 2023, 25(7), 1089; https://doi.org/10.3390/e25071089 - 20 Jul 2023

Cited by 5 | Viewed by 1490

Abstract

We study the role of unstable points in the Bohmian flow of a 2d system composed of two non-interacting harmonic oscillators. In particular, we study the unstable points in the inertial frame of reference as well as in the frame of reference of [...] Read more.

We study the role of unstable points in the Bohmian flow of a 2d system composed of two non-interacting harmonic oscillators. In particular, we study the unstable points in the inertial frame of reference as well as in the frame of reference of the moving nodal points, in cases with 1, 2 and multiple nodal points. Then, we find the contributions of the ordered and chaotic trajectories in the Born distribution, and when the latter is accessible by an initial particle distribution which does not satisfy Born’s rule. Full article

(This article belongs to the Special Issue Quantum Probability and Randomness IV)

► Show Figures

Figure 1

9 pages, 278 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Thermodynamic Entropy as a Noether Invariant from Contact Geometry

by Alessandro Bravetti, Miguel Ángel García-Ariza and Diego Tapias

Entropy 2023, 25(7), 1082; https://doi.org/10.3390/e25071082 - 19 Jul 2023

Cited by 9 | Viewed by 2459

Abstract

We use a formulation of Noether’s theorem for contact Hamiltonian systems to derive a relation between the thermodynamic entropy and the Noether invariant associated with time-translational symmetry. In the particular case of thermostatted systems at equilibrium, we show that the total entropy of [...] Read more.

We use a formulation of Noether’s theorem for contact Hamiltonian systems to derive a relation between the thermodynamic entropy and the Noether invariant associated with time-translational symmetry. In the particular case of thermostatted systems at equilibrium, we show that the total entropy of the system plus the reservoir are conserved as a consequence thereof. Our results contribute to understanding thermodynamic entropy from a geometric point of view. Full article

(This article belongs to the Special Issue Geometric Structure of Thermodynamics: Theory and Applications)

28 pages, 487 KiB

Open AccessEditor’s ChoiceArticle

Design and Analysis of Systematic Batched Network Codes

by Licheng Mao, Shenghao Yang, Xuan Huang and Yanyan Dong

Entropy 2023, 25(7), 1055; https://doi.org/10.3390/e25071055 - 13 Jul 2023

Cited by 1 | Viewed by 1579

Abstract

Systematic codes are of important practical interest for communications. Network coding, however, seems to conflict with systematic codes: although the source node can transmit message packets, network coding at the intermediate network nodes may significantly reduce the number of message packets received by [...] Read more.

Systematic codes are of important practical interest for communications. Network coding, however, seems to conflict with systematic codes: although the source node can transmit message packets, network coding at the intermediate network nodes may significantly reduce the number of message packets received by the destination node. Is it possible to obtain the benefit of network coding while preserving some properties of the systematic codes? In this paper, we study the systematic design of batched network coding, which is a general network coding framework that includes random linear network coding as a special case. A batched network code has an outer code and an inner code, where the latter is formed by linear network coding. A systematic batched network code must take both the outer code and the inner code into consideration. Based on the outer code of a BATS code, which is a matrix-generalized fountain code, we propose a general systematic outer code construction that achieves a low encoding/decoding computation cost. To further reduce the number of random trials required to search a code with a close-to-optimal coding overhead, a triangular embedding approach is proposed for the construction of the systematic batches. We introduce new inner codes that provide protection for the systematic batches during transmission and show that it is possible to significantly increase the expected number of message packets in a received batch at the destination node, without harm to the expected rank of the batch transfer matrix generated by network coding. Full article

(This article belongs to the Special Issue Information Theory and Network Coding II)

► Show Figures

Figure 1

39 pages, 766 KiB

Open AccessEditor’s ChoiceArticle

BAR: Blockwise Adaptive Recoding for Batched Network Coding

by Hoover H. F. Yin, Shenghao Yang, Qiaoqiao Zhou, Lily M. L. Yung and Ka Hei Ng

Entropy 2023, 25(7), 1054; https://doi.org/10.3390/e25071054 - 13 Jul 2023

Cited by 6 | Viewed by 1659

Abstract

Multi-hop networks have become popular network topologies in various emerging Internet of Things (IoT) applications. Batched network coding (BNC) is a solution to reliable communications in such networks with packet loss. By grouping packets into small batches and restricting recoding to the packets [...] Read more.

Multi-hop networks have become popular network topologies in various emerging Internet of Things (IoT) applications. Batched network coding (BNC) is a solution to reliable communications in such networks with packet loss. By grouping packets into small batches and restricting recoding to the packets belonging to the same batch; BNC has much smaller computational and storage requirements at intermediate nodes compared with direct application of random linear network coding. In this paper, we discuss a practical recoding scheme called blockwise adaptive recoding (BAR) which learns the latest channel knowledge from short observations so that BAR can adapt to fluctuations in channel conditions. Due to the low computational power of remote IoT devices, we focus on investigating practical concerns such as how to implement efficient BAR algorithms. We also design and investigate feedback schemes for BAR under imperfect feedback systems. Our numerical evaluations show that BAR has significant throughput gain for small batch sizes compared with existing baseline recoding schemes. More importantly, this gain is insensitive to inaccurate channel knowledge. This encouraging result suggests that BAR is suitable to be used in practice as the exact channel model and its parameters could be unknown and subject to changes from time to time. Full article

(This article belongs to the Special Issue Information Theory and Network Coding II)

► Show Figures

Figure 1

19 pages, 703 KiB

Open AccessEditor’s ChoiceArticle

Stronger Quantum Speed Limit for Mixed Quantum States

by Shrobona Bagchi, Dimpi Thakuria and Arun Kumar Pati

Entropy 2023, 25(7), 1046; https://doi.org/10.3390/e25071046 - 12 Jul 2023

Cited by 2 | Viewed by 2643

Abstract

In this paper, we derive a quantum speed limit for unitary evolution for the case of mixed quantum states using the stronger uncertainty relation for mixed quantum states. This bound can be optimized over different choices of Hermitian operators for a better bound. [...] Read more.

In this paper, we derive a quantum speed limit for unitary evolution for the case of mixed quantum states using the stronger uncertainty relation for mixed quantum states. This bound can be optimized over different choices of Hermitian operators for a better bound. We illustrate this with some examples and show its better performance with respect to three existing bounds for mixed quantum states. Full article

(This article belongs to the Collection Quantum Information)

► Show Figures

Figure 1

24 pages, 518 KiB

Open AccessEditor’s ChoiceArticle

Optimal Resource Allocation for Loss-Tolerant Multicast Video Streaming

by Sadaf ul Zuhra, Karl-Ludwig Besser, Prasanna Chaporkar, Abhay Karandikar and H. Vincent Poor

Entropy 2023, 25(7), 1045; https://doi.org/10.3390/e25071045 - 11 Jul 2023

Cited by 2 | Viewed by 2195

Abstract

In video streaming applications, especially during live streaming events, video traffic can account for a significant portion of the network traffic and can lead to severe network congestion. For such applications, multicast provides an efficient means to deliver the same content to a [...] Read more.

In video streaming applications, especially during live streaming events, video traffic can account for a significant portion of the network traffic and can lead to severe network congestion. For such applications, multicast provides an efficient means to deliver the same content to a large number of users simultaneously. However, in multicast, if the base station transmits content at rates higher than what can be decoded by users with the worst channels, these users will experience outages. This makes the multicast system’s performance dependent on the weakest users in the system. Interestingly, video streams can tolerate some packet loss without a significant degradation in the quality experienced by the users. This property can be leveraged to improve the multicast system’s performance by reducing the dependence of the multicast transmissions on the weakest users. In this work, we design a loss-tolerant video multicasting system that allows for some controlled packet loss while satisfying the quality requirements of the users. In particular, we solve the resource allocation problem in a multimedia broadcast multicast services (MBMS) system by transforming it into the problem of stabilizing a virtual queuing system. We propose two loss-optimal policies and demonstrate their effectiveness using numerical examples with realistic traffic patterns from real video streams. It is shown that the proposed policies are able to keep the loss encountered by every user below its tolerable loss. The proposed policies are also able to achieve a significantly lower peak SNR degradation than the existing schemes. Full article

(This article belongs to the Collection Feature Papers in Information Theory)

► Show Figures

Figure 1

11 pages, 1090 KiB

Open AccessEditor’s ChoiceArticle

Ergodic Measure and Potential Control of Anomalous Diffusion

by Bao Wen, Ming-Gen Li, Jian Liu and Jing-Dong Bao

Entropy 2023, 25(7), 1012; https://doi.org/10.3390/e25071012 - 30 Jun 2023

Cited by 3 | Viewed by 1526

Abstract

In statistical mechanics, the ergodic hypothesis (i.e., the long-time average is the same as the ensemble average) accompanying anomalous diffusion has become a continuous topic of research, being closely related to irreversibility and increasing entropy. While measurement time is finite for a given [...] Read more.

In statistical mechanics, the ergodic hypothesis (i.e., the long-time average is the same as the ensemble average) accompanying anomalous diffusion has become a continuous topic of research, being closely related to irreversibility and increasing entropy. While measurement time is finite for a given process, the time average of an observable quantity might be a random variable, whose distribution width narrows with time, and one wonders how long it takes for the convergence rate to become a constant. This is also the premise of ergodic establishment, because the ensemble average is always equal to the constant. We focus on the time-dependent fluctuation width for the time average of both the velocity and kinetic energy of a force-free particle described by the generalized Langevin equation, where the stationary velocity autocorrelation function is considered. Subsequently, the shortest time scale can be estimated for a system transferring from a stationary state to an effective ergodic state. Moreover, a logarithmic spatial potential is used to modulate the processes associated with free ballistic diffusion and the control of diffusion, as well as the minimal realization of the whole power-law regime. The results presented suggest that non-ergodicity mimics the sparseness of the medium and reveals the unique role of logarithmic potential in modulating diffusion behavior. Full article

(This article belongs to the Section Statistical Physics)

► Show Figures

Figure 1

20 pages, 2464 KiB

Open AccessEditor’s ChoiceArticle

Quantum Entropies and Decoherence for the Multiparticle Quantum Arnol’d Cat

by Giorgio Mantica

Entropy 2023, 25(7), 1004; https://doi.org/10.3390/e25071004 - 29 Jun 2023

Cited by 1 | Viewed by 1695

Abstract

I study the scaling behavior in the physical parameters of dynamical entropies, classical and quantum, in a specifically devised model of collision-induced decoherence in a chaotic system. The treatment is fully canonical and no approximations are involved or infinite limits taken. I present [...] Read more.

I study the scaling behavior in the physical parameters of dynamical entropies, classical and quantum, in a specifically devised model of collision-induced decoherence in a chaotic system. The treatment is fully canonical and no approximations are involved or infinite limits taken. I present this model in a detailed way, in order to clarify my views in the debate about the nature, definition, and relevance of quantum chaos. Full article

(This article belongs to the Special Issue Quantum Chaos—Dedicated to Professor Giulio Casati on the Occasion of His 80th Birthday)

► Show Figures

Figure 1

23 pages, 2881 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Differential Shannon Entropies Characterizing Electron–Nuclear Dynamics and Correlation: Momentum-Space Versus Coordinate-Space Wave Packet Motion

by Peter Schürger and Volker Engel

Entropy 2023, 25(7), 970; https://doi.org/10.3390/e25070970 - 23 Jun 2023

Cited by 5 | Viewed by 1874

Abstract

We calculate differential Shannon entropies derived from time-dependent coordinate-space and momentum-space probability densities. This is performed for a prototype system of a coupled electron–nuclear motion. Two situations are considered, where one is a Born–Oppenheimer adiabatic dynamics, and the other is a diabatic motion [...] Read more.

We calculate differential Shannon entropies derived from time-dependent coordinate-space and momentum-space probability densities. This is performed for a prototype system of a coupled electron–nuclear motion. Two situations are considered, where one is a Born–Oppenheimer adiabatic dynamics, and the other is a diabatic motion involving strong non-adiabatic transitions. The information about coordinate- and momentum-space dynamics derived from the total and single-particle entropies is discussed and interpreted with the help of analytical models. From the entropies, we derive mutual information, which is a measure for the electron–nuclear correlation. In the adiabatic case, it is found that such correlations are manifested differently in coordinate- and momentum space. For the diabatic dynamics, we show that it is possible to decompose the entropies into state-specific contributions. Full article

(This article belongs to the Special Issue Quantum Probability and Randomness IV)

► Show Figures

Figure 1

23 pages, 4268 KiB

Open AccessEditor’s ChoiceArticle

A Variational Synthesis of Evolutionary and Developmental Dynamics

by Karl Friston, Daniel A. Friedman, Axel Constant, V. Bleu Knight, Chris Fields, Thomas Parr and John O. Campbell

Entropy 2023, 25(7), 964; https://doi.org/10.3390/e25070964 - 21 Jun 2023

Cited by 21 | Viewed by 5094

Abstract

This paper introduces a variational formulation of natural selection, paying special attention to the nature of ‘things’ and the way that different ‘kinds’ of ‘things’ are individuated from—and influence—each other. We use the Bayesian mechanics of particular partitions to understand how slow phylogenetic [...] Read more.

This paper introduces a variational formulation of natural selection, paying special attention to the nature of ‘things’ and the way that different ‘kinds’ of ‘things’ are individuated from—and influence—each other. We use the Bayesian mechanics of particular partitions to understand how slow phylogenetic processes constrain—and are constrained by—fast, phenotypic processes. The main result is a formulation of adaptive fitness as a path integral of phenotypic fitness. Paths of least action, at the phenotypic and phylogenetic scales, can then be read as inference and learning processes, respectively. In this view, a phenotype actively infers the state of its econiche under a generative model, whose parameters are learned via natural (Bayesian model) selection. The ensuing variational synthesis features some unexpected aspects. Perhaps the most notable is that it is not possible to describe or model a population of conspecifics per se. Rather, it is necessary to consider populations of distinct natural kinds that influence each other. This paper is limited to a description of the mathematical apparatus and accompanying ideas. Subsequent work will use these methods for simulations and numerical analyses—and identify points of contact with related mathematical formulations of evolution. Full article

(This article belongs to the Special Issue Entropies, Information Geometry and Fluctuations in Non-equilibrium Systems II)

► Show Figures

Figure 1

27 pages, 381 KiB

Open AccessEditor’s ChoiceArticle

Uniform Treatment of Integral Majorization Inequalities with Applications to Hermite-Hadamard-Fejér-Type Inequalities and f-Divergences

by László Horváth

Entropy 2023, 25(6), 954; https://doi.org/10.3390/e25060954 - 19 Jun 2023

Cited by 6 | Viewed by 2581

Abstract

In this paper, we present a general framework that provides a comprehensive and uniform treatment of integral majorization inequalities for convex functions and finite signed measures. Along with new results, we present unified and simple proofs of classical statements. To apply our results, [...] Read more.

In this paper, we present a general framework that provides a comprehensive and uniform treatment of integral majorization inequalities for convex functions and finite signed measures. Along with new results, we present unified and simple proofs of classical statements. To apply our results, we deal with Hermite-Hadamard-Fejér-type inequalities and their refinements. We present a general method to refine both sides of Hermite-Hadamard-Fejér-type inequalities. The results of many papers on the refinement of the Hermite-Hadamard inequality, whose proofs are based on different ideas, can be treated in a uniform way by this method. Finally, we establish a necessary and sufficient condition for when a fundamental inequality of f-divergences can be refined by another f-divergence. Full article

(This article belongs to the Special Issue Shannon Entropy: Mathematical View)

11 pages, 587 KiB

Open AccessEditor’s ChoiceArticle

Threshold Cascade Dynamics in Coevolving Networks

by Byungjoon Min and Maxi San Miguel

Entropy 2023, 25(6), 929; https://doi.org/10.3390/e25060929 - 13 Jun 2023

Cited by 9 | Viewed by 2310

Abstract

We study the coevolutionary dynamics of network topology and social complex contagion using a threshold cascade model. Our coevolving threshold model incorporates two mechanisms: the threshold mechanism for the spreading of a minority state such as a new opinion, idea, or innovation and [...] Read more.

We study the coevolutionary dynamics of network topology and social complex contagion using a threshold cascade model. Our coevolving threshold model incorporates two mechanisms: the threshold mechanism for the spreading of a minority state such as a new opinion, idea, or innovation and the network plasticity, implemented as the rewiring of links to cut the connections between nodes in different states. Using numerical simulations and a mean-field theoretical analysis, we demonstrate that the coevolutionary dynamics can significantly affect the cascade dynamics. The domain of parameters, i.e., the threshold and mean degree, for which global cascades occur shrinks with an increasing network plasticity, indicating that the rewiring process suppresses the onset of global cascades. We also found that during evolution, non-adopting nodes form denser connections, resulting in a wider degree distribution and a non-monotonous dependence of cascades sizes on plasticity. Full article

(This article belongs to the Special Issue Statistical Physics of Opinion Formation and Social Phenomena)

► Show Figures

Figure 1

19 pages, 1221 KiB

Open AccessEditor’s ChoiceArticle

Models of the Translation Process and the Free Energy Principle

by Michael Carl

Entropy 2023, 25(6), 928; https://doi.org/10.3390/e25060928 - 12 Jun 2023

Cited by 7 | Viewed by 3435

Abstract

Translation process research (TPR) has generated a large number of models that aim at explaining human translation processes. In this paper, I suggest an extension of the monitor model to incorporate aspects of relevance theory (RT) and to adopt the free energy principle [...] Read more.

Translation process research (TPR) has generated a large number of models that aim at explaining human translation processes. In this paper, I suggest an extension of the monitor model to incorporate aspects of relevance theory (RT) and to adopt the free energy principle (FEP) as a generative model to elucidate translational behaviour. The FEP—and its corollary, active inference—provide a general, mathematical framework to explain how organisms resist entropic erosion so as to remain within their phenotypic bounds. It posits that organisms reduce the gap between their expectations and observations by minimising a quantity called free energy. I map these concepts on the translation process and exemplify them with behavioural data. The analysis is based on the notion of translation units (TUs) which exhibit observable traces of the translator’s epistemic and pragmatic engagement with their translation environment, (i.e., the text) that can be measured in terms of translation effort and effects. Sequences of TUs cluster into translation states (steady state, orientation, and hesitation). Drawing on active inference, sequences of translation states combine into translation policies that reduce expected free energy. I show how the notion of free energy is compatible with the concept of relevance, as developed in RT, and how essential concepts of the monitor model and RT can be formalised as deep temporal generative models that can be interpreted under a representationalist view, but also support a non-representationalist account. Full article

► Show Figures

Figure 1

32 pages, 1868 KiB

Open AccessEditor’s ChoiceArticle

Predicting Network Hardware Faults through Layered Treatment of Alarms Logs

by Antonio Massaro, Dimitre Kostadinov, Alonso Silva, Alexander Obeid Guzman and Armen Aghasaryan

Entropy 2023, 25(6), 917; https://doi.org/10.3390/e25060917 - 9 Jun 2023

Cited by 3 | Viewed by 2510

Abstract

Maintaining and managing ever more complex telecommunication networks is an increasingly difficult task, which often challenges the capabilities of human experts. There is a consensus both in academia and in the industry on the need to enhance human capabilities with sophisticated algorithmic tools [...] Read more.

Maintaining and managing ever more complex telecommunication networks is an increasingly difficult task, which often challenges the capabilities of human experts. There is a consensus both in academia and in the industry on the need to enhance human capabilities with sophisticated algorithmic tools for decision-making, with the aim of transitioning towards more autonomous, self-optimizing networks. We aimed to contribute to this larger project. We tackled the problem of detecting and predicting the occurrence of faults in hardware components in a radio access network, leveraging the alarm logs produced by the network elements. We defined an end-to-end method for data collection, preparation, labelling, and fault prediction. We proposed a layered approach to fault prediction: we first detected the base station that is going to be faulty and at a second stage, and using a different algorithm, we detected the component of the base station that is going to be faulty. We designed a range of algorithmic solutions and tested them on real data collected from a major telecommunication operator. We concluded that we are able to predict the failure of a network component with satisfying precision and recall. Full article

(This article belongs to the Special Issue Fault Diagnosis Methods Based on Information Theory or Machine Learning: From Theory to Application)

► Show Figures

Figure 1

17 pages, 8181 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

The Entropy Density Behavior across a Plane Shock Wave

by Rosa M. Velasco and Francisco J. Uribe

Entropy 2023, 25(6), 906; https://doi.org/10.3390/e25060906 - 7 Jun 2023

Cited by 1 | Viewed by 1849

Abstract

Entropy density behavior poses many problems when we study non-equilibrium situations. In particular, the local equilibrium hypothesis (LEH) has played a very important role and is taken for granted in non-equilibrium problems, no matter how extreme they are. In this paper we would [...] Read more.

Entropy density behavior poses many problems when we study non-equilibrium situations. In particular, the local equilibrium hypothesis (LEH) has played a very important role and is taken for granted in non-equilibrium problems, no matter how extreme they are. In this paper we would like to calculate the Boltzmann entropy balance equation for a plane shock wave and show its performance for Grad’s 13-moment approximation and the Navier–Stokes–Fourier equations. In fact, we calculate the correction for the LEH in Grad’s case and discuss its properties. Full article

(This article belongs to the Special Issue Entropy in Fluids)

► Show Figures

Figure 1

23 pages, 1208 KiB

Open AccessEditor’s ChoiceReview

Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review

by Man-Fai Wong, Shangxin Guo, Ching-Nam Hang, Siu-Wai Ho and Chee-Wei Tan

Entropy 2023, 25(6), 888; https://doi.org/10.3390/e25060888 - 1 Jun 2023

Cited by 69 | Viewed by 17189

Abstract

This paper provides a comprehensive review of the literature concerning the utilization of Natural Language Processing (NLP) techniques, with a particular focus on transformer-based large language models (LLMs) trained using Big Code, within the domain of AI-assisted programming tasks. LLMs, augmented with software [...] Read more.

This paper provides a comprehensive review of the literature concerning the utilization of Natural Language Processing (NLP) techniques, with a particular focus on transformer-based large language models (LLMs) trained using Big Code, within the domain of AI-assisted programming tasks. LLMs, augmented with software naturalness, have played a crucial role in facilitating AI-assisted programming applications, including code generation, code completion, code translation, code refinement, code summarization, defect detection, and clone detection. Notable examples of such applications include the GitHub Copilot powered by OpenAI’s Codex and DeepMind AlphaCode. This paper presents an overview of the major LLMs and their applications in downstream tasks related to AI-assisted programming. Furthermore, it explores the challenges and opportunities associated with incorporating NLP techniques with software naturalness in these applications, with a discussion on extending AI-assisted programming capabilities to Apple’s Xcode for mobile software development. This paper also presents the challenges of and opportunities for incorporating NLP techniques with software naturalness, empowering developers with advanced coding assistance and streamlining the software development process. Full article

(This article belongs to the Special Issue Statistical Machine Learning with High-Dimensional Data and Image Analysis)

► Show Figures

Figure 1

23 pages, 2054 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

On Sequential Bayesian Inference for Continual Learning

by Samuel Kessler, Adam Cobb, Tim G. J. Rudner, Stefan Zohren and Stephen J. Roberts

Entropy 2023, 25(6), 884; https://doi.org/10.3390/e25060884 - 31 May 2023

Cited by 3 | Viewed by 4437

Abstract

Sequential Bayesian inference can be used for continual learning to prevent catastrophic forgetting of past tasks and provide an informative prior when learning new tasks. We revisit sequential Bayesian inference and assess whether using the previous task’s posterior as a prior for a [...] Read more.

Sequential Bayesian inference can be used for continual learning to prevent catastrophic forgetting of past tasks and provide an informative prior when learning new tasks. We revisit sequential Bayesian inference and assess whether using the previous task’s posterior as a prior for a new task can prevent catastrophic forgetting in Bayesian neural networks. Our first contribution is to perform sequential Bayesian inference using Hamiltonian Monte Carlo. We propagate the posterior as a prior for new tasks by approximating the posterior via fitting a density estimator on Hamiltonian Monte Carlo samples. We find that this approach fails to prevent catastrophic forgetting, demonstrating the difficulty in performing sequential Bayesian inference in neural networks. From there, we study simple analytical examples of sequential Bayesian inference and CL and highlight the issue of model misspecification, which can lead to sub-optimal continual learning performance despite exact inference. Furthermore, we discuss how task data imbalances can cause forgetting. From these limitations, we argue that we need probabilistic models of the continual learning generative process rather than relying on sequential Bayesian inference over Bayesian neural network weights. Our final contribution is to propose a simple baseline called Prototypical Bayesian Continual Learning, which is competitive with the best performing Bayesian continual learning methods on class incremental continual learning computer vision benchmarks. Full article

(This article belongs to the Special Issue Information Theory for Data Science)

► Show Figures

Figure 1

20 pages, 445 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Improvement of Error Correction in Nonequilibrium Information Dynamics

by Qian Zeng, Ran Li and Jin Wang

Entropy 2023, 25(6), 881; https://doi.org/10.3390/e25060881 - 31 May 2023

Viewed by 1877

Abstract

Errors are inevitable in information processing and transfer. While error correction is widely studied in engineering, the underlying physics is not fully understood. Due to the complexity and energy exchange involved, information transmission should be considered as a nonequilibrium process. In this study, [...] Read more.

Errors are inevitable in information processing and transfer. While error correction is widely studied in engineering, the underlying physics is not fully understood. Due to the complexity and energy exchange involved, information transmission should be considered as a nonequilibrium process. In this study, we investigate the effects of nonequilibrium dynamics on error correction using a memoryless channel model. Our findings suggest that error correction improves as nonequilibrium increases, and the thermodynamic cost can be utilized to improve the correction quality. Our results inspire new approaches to error correction that incorporate nonequilibrium dynamics and thermodynamics, and highlight the importance of the nonequilibrium effects in error correction design, particularly in biological systems. Full article

(This article belongs to the Collection Disorder and Biological Physics)

► Show Figures

Figure 1

15 pages, 775 KiB

Open AccessFeature PaperEditor’s ChoiceReview

Self-Organization and Genomic Causality in Models of Morphogenesis

by Ute Deichmann

Entropy 2023, 25(6), 873; https://doi.org/10.3390/e25060873 - 30 May 2023

Cited by 5 | Viewed by 2034

Abstract

The debate about what causes the generation of form and structure in embryological development goes back to antiquity. Most recently, it has focused on the divergent views as to whether the generation of patterns and form in development is a largely self-organized process [...] Read more.

The debate about what causes the generation of form and structure in embryological development goes back to antiquity. Most recently, it has focused on the divergent views as to whether the generation of patterns and form in development is a largely self-organized process or is mainly determined by the genome, in particular, complex developmental gene regulatory processes. This paper presents and analyzes pertinent models of pattern formation and form generation in a developing organism in the past and the present, with a special emphasis on Alan Turing’s 1952 reaction–diffusion model. I first draw attention to the fact that Turing’s paper remained, at first, without a noticeable impact on the community of biologists because purely physical–chemical models were unable to explain embryological development and often also simple repetitive patterns. I then show that from the year 2000 and onwards, Turing’s 1952 paper was increasingly cited also by biologists. The model was updated to include gene products and now seemed able to account for the generation of biological patterns, though discrepancies between models and biological reality remained. I then point out Eric Davidson’s successful theory of early embryogenesis based on gene-regulatory network analysis and its mathematical modeling that not only was able to provide a mechanistic and causal explanation for gene regulatory events controlling developmental cell fate specification but, unlike reaction–diffusion models, also addressed the effects of evolution and organisms’ longstanding developmental and species stability. The paper concludes with an outlook on further developments of the gene regulatory network model. Full article

(This article belongs to the Special Issue Mathematical Modeling in Systems Biology)

► Show Figures

Figure 1

20 pages, 2936 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Bandit Algorithm Driven by a Classical Random Walk and a Quantum Walk

by Tomoki Yamagami, Etsuo Segawa, Takatomo Mihana, André Röhm, Ryoichi Horisaki and Makoto Naruse

Entropy 2023, 25(6), 843; https://doi.org/10.3390/e25060843 - 25 May 2023

Cited by 3 | Viewed by 2842

Abstract

Quantum walks (QWs) have a property that classical random walks (RWs) do not possess—the coexistence of linear spreading and localization—and this property is utilized to implement various kinds of applications. This paper proposes RW- and QW-based algorithms for multi-armed-bandit (MAB) problems. We show [...] Read more.

Quantum walks (QWs) have a property that classical random walks (RWs) do not possess—the coexistence of linear spreading and localization—and this property is utilized to implement various kinds of applications. This paper proposes RW- and QW-based algorithms for multi-armed-bandit (MAB) problems. We show that, under some settings, the QW-based model realizes higher performance than the corresponding RW-based one by associating the two operations that make MAB problems difficult—exploration and exploitation—with these two behaviors of QWs. Full article

(This article belongs to the Special Issue Recent Advances in Quantum Information Processing)

► Show Figures

Figure 1

35 pages, 1600 KiB

Open AccessFeature PaperEditor’s ChoiceArticle

Local Phase Transitions in a Model of Multiplex Networks with Heterogeneous Degrees and Inter-Layer Coupling

by Nedim Bayrakdar, Valerio Gemmetto and Diego Garlaschelli

Entropy 2023, 25(5), 828; https://doi.org/10.3390/e25050828 - 22 May 2023

Viewed by 1763

Abstract

Multilayer networks represent multiple types of connections between the same set of nodes. Clearly, a multilayer description of a system adds value only if the multiplex does not merely consist of independent layers. In real-world multiplexes, it is expected that the observed inter-layer [...] Read more.

Multilayer networks represent multiple types of connections between the same set of nodes. Clearly, a multilayer description of a system adds value only if the multiplex does not merely consist of independent layers. In real-world multiplexes, it is expected that the observed inter-layer overlap may result partly from spurious correlations arising from the heterogeneity of nodes, and partly from true inter-layer dependencies. It is therefore important to consider rigorous ways to disentangle these two effects. In this paper, we introduce an unbiased maximum entropy model of multiplexes with controllable intra-layer node degrees and controllable inter-layer overlap. The model can be mapped to a generalized Ising model, where the combination of node heterogeneity and inter-layer coupling leads to the possibility of local phase transitions. In particular, we find that node heterogeneity favors the splitting of critical points characterizing different pairs of nodes, leading to link-specific phase transitions that may, in turn, increase the overlap. By quantifying how the overlap can be increased by increasing either the intra-layer node heterogeneity (spurious correlation) or the strength of the inter-layer coupling (true correlation), the model allows us to disentangle the two effects. As an application, we show that the empirical overlap observed in the International Trade Multiplex genuinely requires a nonzero inter-layer coupling in its modeling, as it is not merely a spurious result of the correlation between node degrees across different layers. Full article

(This article belongs to the Special Issue Recent Trends and Developments in Econophysics)

► Show Figures

Figure 1

49 pages, 10680 KiB

Open AccessEditor’s ChoiceArticle

Multivariate Time Series Information Bottleneck

by Denis Ullmann, Olga Taran and Slava Voloshynovskiy

Entropy 2023, 25(5), 831; https://doi.org/10.3390/e25050831 - 22 May 2023

Cited by 2 | Viewed by 3799

Abstract

Time series (TS) and multiple time series (MTS) predictions have historically paved the way for distinct families of deep learning models. The temporal dimension, distinguished by its evolutionary sequential aspect, is usually modeled by decomposition into the trio of “trend, seasonality, noise”, by [...] Read more.

Time series (TS) and multiple time series (MTS) predictions have historically paved the way for distinct families of deep learning models. The temporal dimension, distinguished by its evolutionary sequential aspect, is usually modeled by decomposition into the trio of “trend, seasonality, noise”, by attempts to copy the functioning of human synapses, and more recently, by transformer models with self-attention on the temporal dimension. These models may find applications in finance and e-commerce, where any increase in performance of less than

1 %

has large monetary repercussions, they also have potential applications in natural language processing (NLP), medicine, and physics. To the best of our knowledge, the information bottleneck (IB) framework has not received significant attention in the context of TS or MTS analyses. One can demonstrate that a compression of the temporal dimension is key in the context of MTS. We propose a new approach with partial convolution, where a time sequence is encoded into a two-dimensional representation resembling images. Accordingly, we use the recent advances made in image extension to predict an unseen part of an image from a given one. We show that our model compares well with traditional TS models, has information–theoretical foundations, and can be easily extended to more dimensions than only time and space. An evaluation of our multiple time series–information bottleneck (MTS-IB) model proves its efficiency in electricity production, road traffic, and astronomical data representing solar activity, as recorded by NASA’s interface region imaging spectrograph (IRIS) satellite. Full article

(This article belongs to the Special Issue Information-Theoretic Methods in Deep Learning: Theory and Applications)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Editor’s Choice Articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI