You seem to have javascript disabled. Please note that many of the page functionalities won't work as expected without javascript enabled.

Search for Articles:

Title / Keyword

Author / Affiliation / Email

Journal

Article Type

Advanced Search

Section

Special Issue

Volume

Issue

Number

Page

Logical OperatorOperator

Search Text

Search Type

Journal Description

Entropy

Entropy is an international and interdisciplinary peer-reviewed open access journal of entropy and information studies, published monthly online by MDPI. The International Society for the Study of Information (IS4SI) and Spanish Society of Biomedical Engineering (SEIB) are affiliated with Entropy and their members receive a discount on the article processing charge.

Open Access— free for readers, with article processing charges (APC) paid by authors or their institutions.
High Visibility: indexed within Scopus, SCIE (Web of Science), Inspec, PubMed, PMC, Astrophysics Data System, and other databases.
Journal Rank: JCR - Q2 (Physics, Multidisciplinary) / CiteScore - Q1 (Mathematical Physics)
Rapid Publication: manuscripts are peer-reviewed and a first decision is provided to authors approximately 22.3 days after submission; acceptance to publication is undertaken in 2.7 days (median values for papers published in this journal in the second half of 2024).
Recognition of Reviewers: reviewers who provide timely, thorough peer-review reports receive vouchers entitling them to a discount on the APC of their next publication in any MDPI journal, in appreciation of the work done.
Testimonials: See what our editors and authors say about Entropy.
Companion journals for Entropy include: Foundations, Thermo and Complexities.

Impact Factor: 2.1 (2023); 5-Year Impact Factor: 2.2 (2023)

Imprint Information Journal Flyer Open Access ISSN: 1099-4300

Latest Articles

16 pages, 2258 KiB

Open AccessArticle

Lightweight Pre-Trained Korean Language Model Based on Knowledge Distillation and Low-Rank Factorization

by Jin-Hwan Kim and Young-Seok Choi

Entropy 2025, 27(4), 379; https://doi.org/10.3390/e27040379 - 2 Apr 2025

Natural Language Processing (NLP) stands as a forefront of artificial intelligence research, empowering computational systems to comprehend and process human language as used in everyday contexts. Language models (LMs) underpin this field, striving to capture the intricacies of linguistic structure and semantics by [...] Read more.

Natural Language Processing (NLP) stands as a forefront of artificial intelligence research, empowering computational systems to comprehend and process human language as used in everyday contexts. Language models (LMs) underpin this field, striving to capture the intricacies of linguistic structure and semantics by assigning probabilities to sequences of words. The trend towards large language models (LLMs) has shown significant performance improvements with increasing model size. However, the deployment of LLMs on resource-limited devices such as mobile and edge devices remains a challenge. This issue is particularly pronounced in languages other than English, including Korean, where pre-trained models are relatively scarce. Addressing this gap, we introduce a novel lightweight pre-trained Korean language model that leverages knowledge distillation and low-rank factorization techniques. Our approach distills knowledge from a 432 MB (approximately 110 M parameters) teacher model into student models of substantially reduced sizes (e.g., 53 MB ≈ 14 M parameters, 35 MB ≈ 13 M parameters, 30 MB ≈ 11 M parameters, and 18 MB ≈ 4 M parameters). The smaller student models further employ low-rank factorization to minimize the parameter count within the Transformer’s feed-forward network (FFN) and embedding layer. We evaluate the efficacy of our lightweight model across six established Korean NLP tasks. Notably, our most compact model, KR-ELECTRA-Small-KD, attains over 97.387% of the teacher model’s performance despite an 8.15× reduction in size. Remarkably, on the NSMC sentiment classification benchmark, KR-ELECTRA-Small-KD surpasses the teacher model with an accuracy of 89.720%. These findings underscore the potential of our model as an efficient solution for NLP applications in resource-constrained settings. Full article

(This article belongs to the Special Issue Information Processing in Complex Biological Systems)

► Show Figures

Figure 1

47 pages, 2484 KiB

Open AccessArticle

Exploring ISAC: Information-Theoretic Insights

by Mehrasa Ahmadipour, Michèle Wigger and Shlomo Shamai

Entropy 2025, 27(4), 378; https://doi.org/10.3390/e27040378 - 2 Apr 2025

This article reviews results from the literature illustrating the bottlenecks and tradeoffs of integrated sensing and communication (ISAC) through the lens of information theory, thus offering a distinct perspective compared to recent works that focus on signal processing, wireless communications, or other related [...] Read more.

This article reviews results from the literature illustrating the bottlenecks and tradeoffs of integrated sensing and communication (ISAC) through the lens of information theory, thus offering a distinct perspective compared to recent works that focus on signal processing, wireless communications, or other related overviews. Different models and scenarios are considered and compared. For example, scenarios where radar sensing is performed at the communication and radar transmitter (mono-static ISAC) and scenarios where the radar receiver differs from the radar transmitter (called bi-static radar). Similarly, we discuss ISAC bottlenecks and tradeoffs both in slowly-varying environments where the main sensing target is described by a single parameter and accordingly, sensing performance is described by detection error probabilities, as well as in fast-varying environments, where the sensing targets are described by vectors and thus vector-valued performance measures such as average distortions like mean-squared errors are used to determine sensing performances. This overview article further also considers limitations and opportunities in network ISAC environments, such as collaborative or interactive sensing, and the influence of secrecy and privacy requirements on ISAC systems, a line of research that has received growing interest over the last few years. For all these scenarios, we provide and discuss precise models and their limitations and provide either bounds or full characterizations of the fundamental information-theoretic performance limits of these systems. Further extensions as well as important open research directions are also discussed. Full article

(This article belongs to the Special Issue Integrated Sensing and Communications)

21 pages, 1375 KiB

Open AccessArticle

Bit-Level Construction for Multiplicative-Repetition-Based Non-Binary Polar Codes

by Rongchi Xu, Peiyao Chen, Ling Liu, Min Zhu and Baoming Bai

Entropy 2025, 27(4), 377; https://doi.org/10.3390/e27040377 - 2 Apr 2025

In this paper, we discuss non-binary polar codes using a

2 \times 2

matrix over a Galois field GF(

2^{q}

) as the kernel. Conventional construction of non-binary polar codes divides the synthesized channels into frozen channels and information channels. Each information [...] Read more.

In this paper, we discuss non-binary polar codes using a

2 \times 2

matrix over a Galois field GF(

2^{q}

) as the kernel. Conventional construction of non-binary polar codes divides the synthesized channels into frozen channels and information channels. Each information channel carries one symbol, i.e., q bits. However, there are many middle channels with insufficient polarization, which cannot carry one symbol of q bits but only i bits,

1 \leq i < q, i \in Z

, at finite block length. In this paper, we consider bit-level construction for multiplicative repetition (MR)-based non-binary polar codes and propose a bit-level construction based on the two following methods. We first calculate the error probability and channel capacity lower bound of each synthesized channel based on the channel degradation method, and then determine both the number and index of the carried bits for each synthesized channel according to the symbol error probability and capacity. To reduce complexity, we also introduce a Monte-Carlo method. We compute the error probability of each synthesized channel carrying i information bits and select the optimal construction that can minimize the union bound of the error probability. Finally, an improved construction-based probabilistic shaping method for MR-based non-binary polar codes is considered. Simulation results show that the proposed construction significantly improved the decoding performance compared with the conventional construction scheme. Full article

(This article belongs to the Special Issue Advances in Modern Channel Coding)

► Show Figures

Figure 1

26 pages, 464 KiB

Open AccessArticle

A Family of Optimal Linear Functional-Repair Regenerating Storage Codes

by Henk D. L. Hollmann

Entropy 2025, 27(4), 376; https://doi.org/10.3390/e27040376 - 1 Apr 2025

We construct a family of linear optimal functional-repair regenerating storage codes with parameters [...] Read more.

We construct a family of linear optimal functional-repair regenerating storage codes with parameters

{m, (n, k), (r, α, β)} = {(2 r - α + 1) α / 2, (r + 1, r), (r, α, 1)}

for any integers (

r, α

) with

1 \leq α \leq r

over any field when

α \in {1, r - 1, r}

and over any finite field (

F_{q}

) with

q \geq r - 1

otherwise. These storage codes are Minimum-Storage Regenerating (MSR) when

α = 1

and Minimum-Bandwidth Regenerating (MBR) when

α = r

and represent extremal points of the (convex) attainable cut-set region different from the MSR and MBR points in all other cases. It is known that when

2 \leq α \leq r - 1

, these parameters cannot be realized by exact-repair storage codes. Each of these codes comes with an explicit and relatively simple repair method, and repair can even be realized as help by transfer (HBT) if desired. The coding states of codes from this family can be described geometrically as configurations of

r + 1

subspaces of dimension

α

in an m-dimensional vector space with restricted sub-span dimensions. A few “small” codes with these parameters are known: one for

(r, α) = (3, 2)

dating from 2013 and one for

(r, α) = (4, 3)

dating from 2024. Apart from these, our codes are the first examples of explicit, relatively simple, optimal functional-repair storage codes over a small, finite field with an explicit repair method and with parameters representing an extremal point of the attainable cut-set region distinct from the MSR and MBR points. Full article

(This article belongs to the Special Issue Discrete Math in Coding Theory)

► Show Figures

Figure 1

20 pages, 3249 KiB

Open AccessArticle

Correlations and Kappa Distributions: Numerical Experiment and Physical Understanding

by David J. McComas, George Livadiotis and Nicholas V. Sarlis

Entropy 2025, 27(4), 375; https://doi.org/10.3390/e27040375 - 31 Mar 2025

Kappa distributions, their statistical framework, and their thermodynamic origin describe systems with correlations among their particle energies, residing in stationary states out of classical thermal equilibrium/space plasmas, from solar wind to the outer heliosphere, are such systems. We show how correlations from long-range [...] Read more.

Kappa distributions, their statistical framework, and their thermodynamic origin describe systems with correlations among their particle energies, residing in stationary states out of classical thermal equilibrium/space plasmas, from solar wind to the outer heliosphere, are such systems. We show how correlations from long-range interactions compete with collisions to define the specific shape of particle velocity distributions, using a simple numerical experiment with collisions and a variable amount of correlation among the particles. When the correlations are turned off, collisions drive any initial distribution to evolve toward equilibrium and a Maxwell–Boltzmann (MB) distribution. However, when some correlation is introduced, the distribution evolves toward a different stationary state defined by a kappa distribution with some finite value of the thermodynamic kappa

κ

(where

κ \to \infty

corresponds to a MB distribution). Furthermore, the stronger the correlations, the lower the

κ

value. This simple numerical experiment illuminates the role of correlations in forming stationary state particle distributions, which are described by kappa distributions, as well as the physical interpretation of correlations from long-range interactions and how they are related to the thermodynamic kappa. Full article

(This article belongs to the Collection Foundations of Statistical Mechanics)

► Show Figures

Figure 1

12 pages, 10081 KiB

Open AccessArticle

Transient Thermal Energy Harvesting at a Single Temperature Using Nonlinearity

by Tamzeed B. Amin, James M. Mangum, Md R. Kabir, Syed M. Rahman, Ashaduzzaman, Pradeep Kumar, Luis L. Bonilla and Paul M. Thibado

Entropy 2025, 27(4), 374; https://doi.org/10.3390/e27040374 - 31 Mar 2025

The authors present an in-depth theoretical study of two nonlinear circuits capable of transient thermal energy harvesting at one temperature. The first circuit has a storage capacitor and diode connected in series. The second circuit has three storage capacitors, and two diodes arranged [...] Read more.

The authors present an in-depth theoretical study of two nonlinear circuits capable of transient thermal energy harvesting at one temperature. The first circuit has a storage capacitor and diode connected in series. The second circuit has three storage capacitors, and two diodes arranged for full wave rectification. The authors solve both Ito–Langevin and Fokker–Planck equations for both circuits using a large parameter space including capacitance values and diode quality. Surprisingly, using diodes one can harvest thermal energy at a single temperature by charging capacitors. However, this is a transient phenomenon. In equilibrium, the capacitor charge is zero, and this solution alone satisfies the second law of thermodynamics. The authors found that higher quality diodes provide more stored charge and longer lifetimes. Harvesting thermal energy from the ambient environment using diode nonlinearity requires capacitors to be charged but then disconnected from the circuit before they have time to discharge. Full article

(This article belongs to the Section Thermodynamics)

30 pages, 1467 KiB

Open AccessArticle

Rate–Distortion–Perception Trade-Off in Information Theory, Generative Models, and Intelligent Communications

by Xueyan Niu, Bo Bai, Nian Guo, Weixi Zhang and Wei Han

Entropy 2025, 27(4), 373; https://doi.org/10.3390/e27040373 - 31 Mar 2025

Traditional rate–distortion (RD) theory examines the trade-off between the average length of the compressed representation of a source and the additive distortions of its reconstruction. The rate–distortion–perception (RDP) framework, which integrates the perceptual dimension into the RD paradigm, has garnered significant attention due [...] Read more.

Traditional rate–distortion (RD) theory examines the trade-off between the average length of the compressed representation of a source and the additive distortions of its reconstruction. The rate–distortion–perception (RDP) framework, which integrates the perceptual dimension into the RD paradigm, has garnered significant attention due to recent advancements in machine learning, where perceptual fidelity is assessed by the divergence between input and reconstruction distributions. In communication systems where downstream tasks involve generative modeling, high perceptual fidelity is essential, despite distortion constraints. However, while zero distortion implies perfect realism, the converse is not true, highlighting an imbalance in the significance of distortion and perceptual constraints. This article clarifies that incorporating perceptual constraints does not decrease the necessary rate; instead, under certain conditions, additional rate is required, even with the aid of common and private randomness, which are key elements in generative models. Consequently, we project an increase in expected traffic in intelligent communication networks with the consideration of perceptual quality. Nevertheless, a modest increase in rate can enable generative models to significantly enhance the perceptual quality of reconstructions. By exploring the synergies between generative modeling and communication through the lens of information-theoretic results, this article demonstrates the benefits of intelligent communication systems and advocates for the application of the RDP framework in advancing compression and semantic communication research. Full article

(This article belongs to the Special Issue Semantic Information Theory)

► Show Figures

Figure 1

14 pages, 267 KiB

Open AccessArticle

Intrinsic Motivation as Constrained Entropy Maximization

by Alex B. Kiefer

Entropy 2025, 27(4), 372; https://doi.org/10.3390/e27040372 - 31 Mar 2025

“Intrinsic motivation” refers to the capacity for intelligent systems to be motivated endogenously, i.e., by features of agential architecture itself rather than by learned associations between action and reward. This paper views active inference, empowerment, and other formal accounts of intrinsic motivation as [...] Read more.

“Intrinsic motivation” refers to the capacity for intelligent systems to be motivated endogenously, i.e., by features of agential architecture itself rather than by learned associations between action and reward. This paper views active inference, empowerment, and other formal accounts of intrinsic motivation as variations on the theme of constrained maximum entropy inference, providing a general perspective on intrinsic motivation complementary to existing frameworks. The connection between free energy and empowerment noted in previous literature is further explored, and it is argued that the maximum-occupancy approach in practice incorporates an implicit model-evidence constraint. Full article

(This article belongs to the Special Issue From Functional Imaging to Free Energy—Dedicated to Professor Karl Friston on the Occasion of His 65th Birthday)

29 pages, 1945 KiB

Open AccessArticle

Latent Abstractions in Generative Diffusion Models

by Giulio Franzese, Mattia Martini, Giulio Corallo, Paolo Papotti and Pietro Michiardi

Entropy 2025, 27(4), 371; https://doi.org/10.3390/e27040371 - 31 Mar 2025

In this work, we study how diffusion-based generative models produce high-dimensional data, such as images, by relying on latent abstractions that guide the generative process. We introduce a novel theoretical framework extending Nonlinear Filtering (NLF), offering a new perspective on SDE-based generative models. [...] Read more.

In this work, we study how diffusion-based generative models produce high-dimensional data, such as images, by relying on latent abstractions that guide the generative process. We introduce a novel theoretical framework extending Nonlinear Filtering (NLF), offering a new perspective on SDE-based generative models. Our theory is based on a new formulation of joint (state and measurement) dynamics and an information-theoretic measure of state influence on the measurement process. We show that diffusion models can be interpreted as a system of SDE, describing a non-linear filter where unobservable latent abstractions steer the dynamics of an observable measurement process. Additionally, we present an empirical study validating our theory and supporting previous findings on the emergence of latent abstractions at different generative stages. Full article

(This article belongs to the Special Issue The Statistical Physics of Generative Diffusion Models)

► Show Figures

Figure 1

14 pages, 265 KiB

Open AccessArticle

Successive Refinement for Lossy Compression of Individual Sequences

by Neri Merhav

Entropy 2025, 27(4), 370; https://doi.org/10.3390/e27040370 - 31 Mar 2025

We consider the problem of successive-refinement coding for lossy compression of individual sequences, namely, compression in two stages, where in the first stage, a coarse description at a relatively low rate is sent from the encoder to the decoder, and in the second [...] Read more.

We consider the problem of successive-refinement coding for lossy compression of individual sequences, namely, compression in two stages, where in the first stage, a coarse description at a relatively low rate is sent from the encoder to the decoder, and in the second stage, an additional coding rate is allocated in order to refine the description and thereby improve the reproduction. Our main result is in establishing outer bounds (converse theorems) for the rate region where we limit the encoders to be finite-state machines in the spirit of Ziv and Lempel’s 1978 model. The matching achievability scheme is conceptually straightforward. We also consider the more general multiple description coding problem on a similar footing and propose achievability schemes that are analogous to the well-known El Gamal–Cover and the Zhang–Berger achievability schemes of memoryless sources and additive distortion measures. Full article

(This article belongs to the Collection Feature Papers in Information Theory)

23 pages, 1113 KiB

Open AccessArticle

Feature-Driven Semantic Communication for Efficient Image Transmission

by Ji Zhang, Ying Zhang, Baofeng Ji, Anmin Chen, Aoxue Liu and Hengzhou Xu

Entropy 2025, 27(4), 369; https://doi.org/10.3390/e27040369 - 31 Mar 2025

Semantic communication is an emerging approach that enhances transmission efficiency by conveying the semantic content of information more effectively. It has garnered significant attention in recent years. However, existing semantic communication systems for image transmission typically adopt direct transmission of features or uniformly [...] Read more.

Semantic communication is an emerging approach that enhances transmission efficiency by conveying the semantic content of information more effectively. It has garnered significant attention in recent years. However, existing semantic communication systems for image transmission typically adopt direct transmission of features or uniformly compress features before transmission. They have not yet considered the differential impact of features on image recovery at the receiver end and the issue of bandwidth limitations during actual transmission. This paper shows that non-uniform processing of features leads to better image recovery under bandwidth constraints compared to uniform processing. Based on this, we propose a semantic communication system for image transmission, which introduces non-uniform quantization techniques. In the feature transmission stage, the system performs varying levels of quantization based on the differences in feature performance at the receiver, thereby reducing the bandwidth requirement. Inspired by quantitative quantization techniques, we design a non-uniform quantization algorithm capable of dynamic bit allocation. This algorithm, under bandwidth constraints, dynamically adjusts the quantization precision of features based on their contribution to the completion of tasks at the receiver end, ensuring the quality and accuracy of the transmitted data even under limited bandwidth conditions. Experimental results show that the proposed system reduces bandwidth usage while ensuring image reconstruction quality. Full article

(This article belongs to the Special Issue Semantic Information Theory)

► Show Figures

Figure 1

19 pages, 6626 KiB

Open AccessArticle

Action Recognition with 3D Residual Attention and Cross Entropy

by Yuhao Ouyang and Xiangqian Li

Entropy 2025, 27(4), 368; https://doi.org/10.3390/e27040368 - 31 Mar 2025

This study proposes a three-dimensional (3D) residual attention network (3DRFNet) for human activity recognition by learning spatiotemporal representations from motion pictures. Core innovation integrates the attention mechanism into the 3D ResNet framework to emphasize key features and suppress irrelevant ones. In each 3D [...] Read more.

This study proposes a three-dimensional (3D) residual attention network (3DRFNet) for human activity recognition by learning spatiotemporal representations from motion pictures. Core innovation integrates the attention mechanism into the 3D ResNet framework to emphasize key features and suppress irrelevant ones. In each 3D ResNet block, channel and spatial attention mechanisms generate attention maps for tensor segments, which are then multiplied by the input feature mapping to emphasize key features. Additionally, the integration of Fast Fourier Convolution (FFC) enhances the network’s capability to effectively capture temporal and spatial features. Simultaneously, we used the cross-entropy loss function to describe the difference between the predicted value and GT to guide the model’s backpropagation. Subsequent experimental results have demonstrated that 3DRFNet achieved SOTA performance in human action recognition. 3DRFNet achieved accuracies of 91.7% and 98.7% on the HMDB-51 and UCF-101 datasets, respectively, which highlighted 3DRFNet’s advantages in recognition accuracy and robustness, particularly in effectively capturing key behavioral features in videos using both attention mechanisms. Full article

(This article belongs to the Special Issue Application of Information Theory to Computer Vision and Image Processing II)

► Show Figures

Figure 1

23 pages, 8280 KiB

Open AccessArticle

From Task Distributions to Expected Paths Lengths Distributions: Value Function Initialization in Sparse Reward Environments for Lifelong Reinforcement Learning

by Soumia Mehimeh and Xianglong Tang

Entropy 2025, 27(4), 367; https://doi.org/10.3390/e27040367 - 30 Mar 2025

This paper studies value function transfer within reinforcement learning frameworks, focusing on tasks continuously assigned to an agent through a probabilistic distribution. Specifically, we focus on environments characterized by sparse rewards with a terminal goal. Initially, we propose and theoretically demonstrate that the [...] Read more.

This paper studies value function transfer within reinforcement learning frameworks, focusing on tasks continuously assigned to an agent through a probabilistic distribution. Specifically, we focus on environments characterized by sparse rewards with a terminal goal. Initially, we propose and theoretically demonstrate that the distribution of the computed value function from such environments, whether in cases where the goals or the dynamics are changing across tasks, can be reformulated as the distribution of the number of steps to the goal generated by their optimal policies, which we name the expected optimal path length. To test our propositions, we hypothesize that the distribution of the expected optimal path lengths resulting from the task distribution is normal. This claim leads us to propose that if the distribution is normal, then the distribution of the value function follows a log-normal pattern. Leveraging this insight, we introduce “LogQInit” as a novel value function transfer method, based on the properties of log-normality. Finally, we run experiments on a scenario of goals and dynamics distributions, validate our proposition by providing an a dequate analysis of the results, and demonstrate that LogQInit outperforms existing methods of value function initialization, policy transfer, and reward shaping. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

20 pages, 342 KiB

Open AccessArticle

Generalized Orthogonal de Bruijn and Kautz Sequences

by Yuan-Pon Chen, Jin Sima and Olgica Milenkovic

Entropy 2025, 27(4), 366; https://doi.org/10.3390/e27040366 - 30 Mar 2025

A de Bruijn sequence of order k over a finite alphabet is a cyclic sequence with the property that it contains every possible k-sequence as a substring exactly once. Orthogonal de Bruijn sequences are the collections of de Bruijn sequences of the [...] Read more.

A de Bruijn sequence of order k over a finite alphabet is a cyclic sequence with the property that it contains every possible k-sequence as a substring exactly once. Orthogonal de Bruijn sequences are the collections of de Bruijn sequences of the same order, k, that satisfy the joint constraint that every

(k + 1)

-sequence appears as a substring in, at most, one of the sequences in the collection. Both de Bruijn and orthogonal de Bruijn sequences have found numerous applications in synthetic biology, although the latter remain largely unexplored in the coding theory literature. Here, we study three relevant practical generalizations of orthogonal de Bruijn sequences, where we relax either the constraint that every

(k + 1)

-sequence appears exactly once or the sequences themselves are de Bruijn rather than balanced de Bruijn sequences. We also provide lower and upper bounds on the number of fixed-weight orthogonal de Bruijn sequences. The paper concludes with parallel results for orthogonal nonbinary Kautz sequences, which satisfy similar constraints as de Bruijn sequences, except for being only required to cover all subsequences of length k whose maximum run length equals one. Full article

(This article belongs to the Special Issue Coding and Algorithms for DNA-Based Data Storage Systems)

► Show Figures

Figure 1

33 pages, 2471 KiB

Open AccessArticle

Exploring the Evolution-Coupling Hypothesis: Do Enzymes’ Performance Gains Correlate with Increased Dissipation?

by Davor Juretić

Entropy 2025, 27(4), 365; https://doi.org/10.3390/e27040365 - 29 Mar 2025

The research literature presents divergent opinions regarding the role of dissipation in living systems, with views ranging from it being useless to it being essential for driving life. The implications of universal thermodynamic evolution are often overlooked or considered controversial. A higher rate [...] Read more.

The research literature presents divergent opinions regarding the role of dissipation in living systems, with views ranging from it being useless to it being essential for driving life. The implications of universal thermodynamic evolution are often overlooked or considered controversial. A higher rate of entropy production indicates faster thermodynamic evolution. We calculated enzyme-associated dissipation under steady-state conditions using minimalistic models of enzyme kinetics when all microscopic rate constants are known. We found that dissipation is roughly proportional to the turnover number, and a log-log power-law relationship exists between dissipation and the catalytic efficiency of enzymes. “Perfect” specialized enzymes exhibit the highest dissipation levels and represent the pinnacle of biological evolution. The examples that we analyzed suggested two key points: (a) more evolved enzymes excel in free-energy dissipation, and (b) the proposed evolutionary trajectory from generalist to specialized enzymes should involve increased dissipation for the latter. Introducing stochastic noise in the kinetics of individual enzymes may lead to optimal performance parameters that exceed the observed values. Our findings indicate that biological evolution has opened new channels for dissipation through specialized enzymes. We also discuss the implications of our results concerning scaling laws and the seamless coupling between thermodynamic and biological evolution in living systems immersed in out-of-equilibrium environments. Full article

► Show Figures

Figure 1

17 pages, 2459 KiB

Open AccessArticle

Entropy Profiles for Li-Ion Batteries—Effects of Chemistries and Degradation

by Julia Wind and Preben J. S. Vie

Entropy 2025, 27(4), 364; https://doi.org/10.3390/e27040364 - 29 Mar 2025

This paper presents entropy measurements for a large set of commercial Li-ion cells. We present entropy data on full cells with a variety of common Li-ion cell electrode chemistries; graphite, hard carbon, lithium-titanium-oxide (LTO), lithium cobalt-oxide (LCO), nickel manganese cobalt oxides (NMC), nickel [...] Read more.

This paper presents entropy measurements for a large set of commercial Li-ion cells. We present entropy data on full cells with a variety of common Li-ion cell electrode chemistries; graphite, hard carbon, lithium-titanium-oxide (LTO), lithium cobalt-oxide (LCO), nickel manganese cobalt oxides (NMC), nickel cobalt aluminium oxide (NCA), lithium iron-phosphate (LFP), as well as electrodes with mixes of these. All data were collected using an accelerated potentiometric method in steps of approximately 5% State-of-Charge (SoC) across the full SoC window. We observe that the entropy profiles depend on the chemistry of the Li-ion cells, but that they also vary between different commercial cells with the same chemistry. Entropy contributions are quantified with respect to both, their means, positive and negative contributions as well as their SoC variation. In addition, we present how different cyclic ageing temperatures change the entropy profiles for a selected commercial Li-ion cell through ageing. A clear difference in entropy profiles is observed after a capacity loss of 20%. This difference can be attributed to different ageing mechanisms within the Li-ion cells, leading to changes in the balancing of electrodes, as well as changes in the electrode materials. Full article

(This article belongs to the Special Issue The Entropy Production—as Cornerstone in Applied Nonequilibrium Thermodynamics—Dedicated to Professor Signe Kjelstrup on the Occasion of Her 75th Birthday)

► Show Figures

Figure 1

20 pages, 4435 KiB

Open AccessArticle

OMAL: A Multi-Label Active Learning Approach from Data Streams

by Qiao Fang, Chen Xiang, Jicong Duan, Benallal Soufiyan, Changbin Shao, Xibei Yang, Sen Xu and Hualong Yu

Entropy 2025, 27(4), 363; https://doi.org/10.3390/e27040363 - 29 Mar 2025

With the rapid growth of digital computing, communication, and storage devices applied in various real-world scenarios, more and more data have been collected and stored to drive the development of machine learning techniques. It is also noted that the data that emerge in [...] Read more.

With the rapid growth of digital computing, communication, and storage devices applied in various real-world scenarios, more and more data have been collected and stored to drive the development of machine learning techniques. It is also noted that the data that emerge in real-world applications tend to become more complex. In this study, we regard a complex data type, i.e., multi-label data, acquired with a time constraint in a dynamic online scenario. Under such conditions, constructing a learning model has to face two challenges: it requires dynamically adapting the variances in label correlations and imbalanced data distributions and it requires more labeling consumptions. To solve these two issues, we propose a novel online multi-label active learning (OMAL) algorithm that considers simultaneously adopting uncertainty (using the average entropy of prediction probabilities) and diversity (using the average cosine distance between feature vectors) as an active query strategy. Specifically, to focus on label correlations, we use a classifier chain (CC) as the multi-label learning model and design a label co-occurrence ranking strategy to arrange label sequence in CC. To adapt the naturally imbalanced distribution of the multi-label data, we select weight extreme learning machine (WELM) as the basic binary-class classifier in CC. The experimental results on ten benchmark multi-label datasets that were transformed into streams show that our proposed method is superior to several popular static multi-label active learning algorithms in terms of both the Macro-F1 and Micro-F1 metrics, indicating its specifical adaptions in the dynamic data stream environment. Full article

(This article belongs to the Section Signal and Data Analysis)

► Show Figures

Figure 1

26 pages, 615 KiB

Open AccessArticle

Vector Flows That Compute the Capacity of Discrete Memoryless Channels

by Guglielmo Beretta and Marcello Pelillo

Entropy 2025, 27(4), 362; https://doi.org/10.3390/e27040362 - 29 Mar 2025

One of the fundamental problems of information theory, since its foundation by C. Shannon, has been the computation of the capacity of a discrete memoryless channel, a quantity expressing the maximum rate at which information can travel through the channel. In this paper, [...] Read more.

One of the fundamental problems of information theory, since its foundation by C. Shannon, has been the computation of the capacity of a discrete memoryless channel, a quantity expressing the maximum rate at which information can travel through the channel. In this paper, we investigate the properties of a novel approach to computing the capacity, based on a continuous-time dynamical system. Interestingly, the proposed dynamical system can be regarded as a continuous-time version of the classical Blahut–Arimoto algorithm, and we can prove that the former shares with the latter an exponential rate of convergence if certain conditions are met. Moreover, a circuit design is presented to implement the dynamics, hence enabling analog computation to estimate the capacity. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

18 pages, 958 KiB

Open AccessArticle

Lower Limit of Percolation Threshold on Square Lattice with Complex Neighborhoods

by Antoni Piotr Ciepłucha, Marcin Utnicki, Maciej Wołoszyn and Krzysztof Malarz

Entropy 2025, 27(4), 361; https://doi.org/10.3390/e27040361 - 29 Mar 2025

In this paper, the 60-year-old concept of long-range interaction in percolation problems introduced by Dalton, Domb and Sykes is reconsidered. With Monte Carlo simulation—based on the Newman–Ziff algorithm and the finite-size scaling hypothesis—we estimate 64 percolation thresholds for a random site percolation problem [...] Read more.

In this paper, the 60-year-old concept of long-range interaction in percolation problems introduced by Dalton, Domb and Sykes is reconsidered. With Monte Carlo simulation—based on the Newman–Ziff algorithm and the finite-size scaling hypothesis—we estimate 64 percolation thresholds for a random site percolation problem on a square lattice with neighborhoods that contain sites from the seventh coordination zone. The percolation thresholds obtained range from

0.27013

(for the neighborhood that contains only sites from the seventh coordination zone) to

0.11535

(for the neighborhood that contains all sites from the first to the seventh coordination zone). Similarly to neighborhoods with smaller ranges, the power-law dependence of the percolation threshold on the effective coordination number with an exponent close to

- 1 / 2

is observed. Finally, we empirically determine the limit of the percolation threshold on square lattices with complex neighborhoods. This limit scales with the inverse square of the mean radius of the neighborhood. The boundary of this limit is touched for threshold values associated with extended (compact) neighborhoods. Full article

(This article belongs to the Special Issue Percolation in the 21st Century)

► Show Figures

Figure 1

20 pages, 3634 KiB

Open AccessArticle

LOMDP: Maximizing Desired Opinions in Social Networks by Considering User Expression Intentions

by Xuan Wang, Bin Wu and Tong Wu

Entropy 2025, 27(4), 360; https://doi.org/10.3390/e27040360 - 29 Mar 2025

To address the problem of maximizing desired opinions in social networks, we present the Limited Opinion Maximization with Dynamic Propagation Optimization framework, which is grounded in information entropy theory. Innovatively, we introduce the concept of node expression capacity, which quantifies the uncertainty of [...] Read more.

To address the problem of maximizing desired opinions in social networks, we present the Limited Opinion Maximization with Dynamic Propagation Optimization framework, which is grounded in information entropy theory. Innovatively, we introduce the concept of node expression capacity, which quantifies the uncertainty of users’ expression intentions via entropy and effectively identifies the impact of silent nodes on the propagation process. Based on this, in terms of seed node selection, we develop the Limited Opinion Maximization algorithm for multi-stage seed selection, which dynamically optimizes the seed distribution among communities through a multi-stage seeding approach. In terms of node opinion changes, we establish the LODP dynamic opinion propagation model, reconstructing the node opinion update mechanism and explicitly modeling the entropy-increasing effect of silent nodes on the information propagation path. The experimental results on four datasets show that LOMDP outperforms six baseline algorithms. Our research effectively resolves the problem of maximizing desired opinions and offers insights into the dynamics of information propagation in social networks from the perspective of entropy and information theory. Full article

(This article belongs to the Special Issue Spreading Dynamics in Complex Networks)

► Show Figures

Figure 1

More Articles...

Submit to Entropy Review for Entropy

Journal Menu

Journal Browser

► Journal Browser

Highly Accessed Articles

View More...

Latest Books

More Books and Reprints...

E-Mail Alert

News

2 April 2025
MDPI INSIGHTS: The CEO's Letter #21 - Annual Report, Swiss Consortium, IWD, ICARS, Serbia

17 March 2025
Meet Us at the Conference on Lasers and Electro-Optics 2025, 6–8 May 2025, California, USA

13 March 2025
Entropy Webinar | Delay-Doppler Domain Communications for Future Wireless Networks, 14 April 2025

More News & Announcements...

Topics

Propose a Topic

Topic in AI, Buildings, Computers, Drones, Entropy, Symmetry

Applications of Machine Learning in Large-Scale Optimization and High-Dimensional Learning Topic Editors: Jeng-Shyang Pan, Junzo Watada, Vaclav Snasel, Pei Hu
Deadline: 30 April 2025

Topic in Axioms, Computation, Entropy, MCA, Mathematics, Symmetry

Numerical Methods for Partial Differential Equations Topic Editors: Pengzhan Huang, Yinnian He
Deadline: 30 June 2025

Topic in Atmosphere, Earth, Encyclopedia, Entropy, Fractal Fract, MAKE, Meteorology

Revisiting Butterfly Effect, Multiscale Dynamics, and Predictability Using Ai-Enhanced Modeling Framework (AEMF) and Chaos Theory Topic Editors: Bo-Wen Shen, Roger A. Pielke Sr., Xubin Zeng
Deadline: 31 July 2025

Topic in Entropy, Algorithms, Computation, Fractal Fract

Computational Complex Networks Topic Editors: Alexandre G. Evsukoff, Yilun Shang
Deadline: 30 September 2025

More Topics

Conferences

Announce Your Conference

14 April 2025 Entropy Webinar | Delay-Doppler Domain Communications for Future Wireless Networks, 14 April 2025

10–14 June 2025 The 1st International Online Conference of the Journal Philosophies: Intelligent Inquiry into Intelligence—Contributing to the 2025 IS4SI Summit

22–25 April 2025 The 16th International Conference on Complex Networks (COMPLENET 2025)

More Conferences...

Special Issues

Propose a Special Issue

Special Issue in Entropy

Quantum Walks for Quantum Technologies Guest Editors: Luca Razzoli, Paolo Bordone
Deadline: 10 April 2025

Special Issue in Entropy

Spreading Dynamics in Complex Networks Guest Editor: Lazaros Gallos
Deadline: 15 April 2025

Special Issue in Entropy

Entanglement Entropy and Quantum Phase Transition Guest Editors: Longwen Zhou, Dajian Zhang
Deadline: 15 April 2025

Special Issue in Entropy

Causal Inference in Recommender Systems Guest Editors: Fuli Feng, Xu Chen
Deadline: 15 April 2025

More Special Issues

Topical Collections

Topical Collection in Entropy

Graphs and Networks from an Algorithmic Information Perspective Collection Editors: Narsis A. Kiani, Hector Zenil, Jesper Tegnér

Topical Collection in Entropy

Wavelets, Fractals and Information Theory Collection Editor: Carlo Cattani

Topical Collection in Entropy

Disorder and Biological Physics Collection Editors: Hong Qian, Felix Ritort, Lamberto Rondoni

Topical Collection in Entropy

Feature Papers in Information Theory Collection Editors: Raúl Alcaraz, Luca Faes, Leandro Pardo, Boris Ryabko

More Topical Collections

Back to TopTop