Entropy

20 pages, 3249 KiB

Open AccessArticle

Correlations and Kappa Distributions: Numerical Experiment and Physical Understanding

by David J. McComas, George Livadiotis and Nicholas V. Sarlis

Entropy 2025, 27(4), 375; https://doi.org/10.3390/e27040375 (registering DOI) - 31 Mar 2025

Kappa distributions, their statistical framework, and their thermodynamic origin describe systems with correlations among their particle energies, residing in stationary states out of classical thermal equilibrium/space plasmas, from solar wind to the outer heliosphere, are such systems. We show how correlations from long-range [...] Read more.

Kappa distributions, their statistical framework, and their thermodynamic origin describe systems with correlations among their particle energies, residing in stationary states out of classical thermal equilibrium/space plasmas, from solar wind to the outer heliosphere, are such systems. We show how correlations from long-range interactions compete with collisions to define the specific shape of particle velocity distributions, using a simple numerical experiment with collisions and a variable amount of correlation among the particles. When the correlations are turned off, collisions drive any initial distribution to evolve toward equilibrium and a Maxwell–Boltzmann (MB) distribution. However, when some correlation is introduced, the distribution evolves toward a different stationary state defined by a kappa distribution with some finite value of the thermodynamic kappa

κ

(where

κ \to \infty

corresponds to a MB distribution). Furthermore, the stronger the correlations, the lower the

κ

value. This simple numerical experiment illuminates the role of correlations in forming stationary state particle distributions, which are described by kappa distributions, as well as the physical interpretation of correlations from long-range interactions and how they are related to the thermodynamic kappa. Full article

(This article belongs to the Collection Foundations of Statistical Mechanics)

► Show Figures

Figure 1

12 pages, 10081 KiB

Open AccessArticle

Transient Thermal Energy Harvesting at a Single Temperature Using Nonlinearity

by Tamzeed B. Amin, James M. Mangum, Md R. Kabir, Syed M. Rahman, Ashaduzzaman, Pradeep Kumar, Luis L. Bonilla and Paul M. Thibado

Entropy 2025, 27(4), 374; https://doi.org/10.3390/e27040374 (registering DOI) - 31 Mar 2025

Abstract

The authors present an in-depth theoretical study of two nonlinear circuits capable of transient thermal energy harvesting at one temperature. The first circuit has a storage capacitor and diode connected in series. The second circuit has three storage capacitors, and two diodes arranged [...] Read more.

The authors present an in-depth theoretical study of two nonlinear circuits capable of transient thermal energy harvesting at one temperature. The first circuit has a storage capacitor and diode connected in series. The second circuit has three storage capacitors, and two diodes arranged for full wave rectification. The authors solve both Ito–Langevin and Fokker–Planck equations for both circuits using a large parameter space including capacitance values and diode quality. Surprisingly, using diodes one can harvest thermal energy at a single temperature by charging capacitors. However, this is a transient phenomenon. In equilibrium, the capacitor charge is zero, and this solution alone satisfies the second law of thermodynamics. The authors found that higher quality diodes provide more stored charge and longer lifetimes. Harvesting thermal energy from the ambient environment using diode nonlinearity requires capacitors to be charged but then disconnected from the circuit before they have time to discharge. Full article

(This article belongs to the Section Thermodynamics)

30 pages, 1467 KiB

Open AccessArticle

Rate–Distortion–Perception Trade-Off in Information Theory, Generative Models, and Intelligent Communications

by Xueyan Niu, Bo Bai, Nian Guo, Weixi Zhang and Wei Han

Entropy 2025, 27(4), 373; https://doi.org/10.3390/e27040373 (registering DOI) - 31 Mar 2025

Abstract

Traditional rate–distortion (RD) theory examines the trade-off between the average length of the compressed representation of a source and the additive distortions of its reconstruction. The rate–distortion–perception (RDP) framework, which integrates the perceptual dimension into the RD paradigm, has garnered significant attention due [...] Read more.

Traditional rate–distortion (RD) theory examines the trade-off between the average length of the compressed representation of a source and the additive distortions of its reconstruction. The rate–distortion–perception (RDP) framework, which integrates the perceptual dimension into the RD paradigm, has garnered significant attention due to recent advancements in machine learning, where perceptual fidelity is assessed by the divergence between input and reconstruction distributions. In communication systems where downstream tasks involve generative modeling, high perceptual fidelity is essential, despite distortion constraints. However, while zero distortion implies perfect realism, the converse is not true, highlighting an imbalance in the significance of distortion and perceptual constraints. This article clarifies that incorporating perceptual constraints does not decrease the necessary rate; instead, under certain conditions, additional rate is required, even with the aid of common and private randomness, which are key elements in generative models. Consequently, we project an increase in expected traffic in intelligent communication networks with the consideration of perceptual quality. Nevertheless, a modest increase in rate can enable generative models to significantly enhance the perceptual quality of reconstructions. By exploring the synergies between generative modeling and communication through the lens of information-theoretic results, this article demonstrates the benefits of intelligent communication systems and advocates for the application of the RDP framework in advancing compression and semantic communication research. Full article

(This article belongs to the Special Issue Semantic Information Theory)

► Show Figures

Figure 1

14 pages, 265 KiB

Open AccessArticle

Intrinsic Motivation as Constrained Entropy Maximization

by Alex B. Kiefer

Entropy 2025, 27(4), 372; https://doi.org/10.3390/e27040372 (registering DOI) - 31 Mar 2025

Abstract

“Intrinsic motivation” refers to the capacity for intelligent systems to be motivated endogenously, i.e., by features of agential architecture itself rather than by learned associations between action and reward. This paper views active inference, empowerment, and other formal accounts of intrinsic motivation as [...] Read more.

“Intrinsic motivation” refers to the capacity for intelligent systems to be motivated endogenously, i.e., by features of agential architecture itself rather than by learned associations between action and reward. This paper views active inference, empowerment, and other formal accounts of intrinsic motivation as variations on the theme of constrained maximum entropy inference, providing a general perspective on intrinsic motivation complementary to existing frameworks. The connection between free energy and empowerment noted in previous literature is further explored, and it is argued that the maximum-occupancy approach in practice incorporates an implicit model-evidence constraint. Full article

(This article belongs to the Special Issue From Functional Imaging to Free Energy—Dedicated to Professor Karl Friston on the Occasion of His 65th Birthday)

29 pages, 1945 KiB

Open AccessArticle

Latent Abstractions in Generative Diffusion Models

by Giulio Franzese, Mattia Martini, Giulio Corallo, Paolo Papotti and Pietro Michiardi

Entropy 2025, 27(4), 371; https://doi.org/10.3390/e27040371 (registering DOI) - 31 Mar 2025

Abstract

In this work, we study how diffusion-based generative models produce high-dimensional data, such as images, by relying on latent abstractions that guide the generative process. We introduce a novel theoretical framework extending Nonlinear Filtering (NLF), offering a new perspective on SDE-based generative models. [...] Read more.

In this work, we study how diffusion-based generative models produce high-dimensional data, such as images, by relying on latent abstractions that guide the generative process. We introduce a novel theoretical framework extending Nonlinear Filtering (NLF), offering a new perspective on SDE-based generative models. Our theory is based on a new formulation of joint (state and measurement) dynamics and an information-theoretic measure of state influence on the measurement process. We show that diffusion models can be interpreted as a system of SDE, describing a non-linear filter where unobservable latent abstractions steer the dynamics of an observable measurement process. Additionally, we present an empirical study validating our theory and supporting previous findings on the emergence of latent abstractions at different generative stages. Full article

(This article belongs to the Special Issue The Statistical Physics of Generative Diffusion Models)

► Show Figures

Figure 1

14 pages, 265 KiB

Open AccessArticle

Successive Refinement for Lossy Compression of Individual Sequences

by Neri Merhav

Entropy 2025, 27(4), 370; https://doi.org/10.3390/e27040370 - 31 Mar 2025

Abstract

We consider the problem of successive-refinement coding for lossy compression of individual sequences, namely, compression in two stages, where in the first stage, a coarse description at a relatively low rate is sent from the encoder to the decoder, and in the second [...] Read more.

We consider the problem of successive-refinement coding for lossy compression of individual sequences, namely, compression in two stages, where in the first stage, a coarse description at a relatively low rate is sent from the encoder to the decoder, and in the second stage, an additional coding rate is allocated in order to refine the description and thereby improve the reproduction. Our main result is in establishing outer bounds (converse theorems) for the rate region where we limit the encoders to be finite-state machines in the spirit of Ziv and Lempel’s 1978 model. The matching achievability scheme is conceptually straightforward. We also consider the more general multiple description coding problem on a similar footing and propose achievability schemes that are analogous to the well-known El Gamal–Cover and the Zhang–Berger achievability schemes of memoryless sources and additive distortion measures. Full article

(This article belongs to the Collection Feature Papers in Information Theory)

23 pages, 1113 KiB

Open AccessArticle

Feature-Driven Semantic Communication for Efficient Image Transmission

by Ji Zhang, Ying Zhang, Baofeng Ji, Anmin Chen, Aoxue Liu and Hengzhou Xu

Entropy 2025, 27(4), 369; https://doi.org/10.3390/e27040369 - 31 Mar 2025

Abstract

Semantic communication is an emerging approach that enhances transmission efficiency by conveying the semantic content of information more effectively. It has garnered significant attention in recent years. However, existing semantic communication systems for image transmission typically adopt direct transmission of features or uniformly [...] Read more.

Semantic communication is an emerging approach that enhances transmission efficiency by conveying the semantic content of information more effectively. It has garnered significant attention in recent years. However, existing semantic communication systems for image transmission typically adopt direct transmission of features or uniformly compress features before transmission. They have not yet considered the differential impact of features on image recovery at the receiver end and the issue of bandwidth limitations during actual transmission. This paper shows that non-uniform processing of features leads to better image recovery under bandwidth constraints compared to uniform processing. Based on this, we propose a semantic communication system for image transmission, which introduces non-uniform quantization techniques. In the feature transmission stage, the system performs varying levels of quantization based on the differences in feature performance at the receiver, thereby reducing the bandwidth requirement. Inspired by quantitative quantization techniques, we design a non-uniform quantization algorithm capable of dynamic bit allocation. This algorithm, under bandwidth constraints, dynamically adjusts the quantization precision of features based on their contribution to the completion of tasks at the receiver end, ensuring the quality and accuracy of the transmitted data even under limited bandwidth conditions. Experimental results show that the proposed system reduces bandwidth usage while ensuring image reconstruction quality. Full article

(This article belongs to the Special Issue Semantic Information Theory)

► Show Figures

Figure 1

19 pages, 6626 KiB

Open AccessArticle

Action Recognition with 3D Residual Attention and Cross Entropy

by Yuhao Ouyang and Xiangqian Li

Entropy 2025, 27(4), 368; https://doi.org/10.3390/e27040368 - 31 Mar 2025

Abstract

This study proposes a three-dimensional (3D) residual attention network (3DRFNet) for human activity recognition by learning spatiotemporal representations from motion pictures. Core innovation integrates the attention mechanism into the 3D ResNet framework to emphasize key features and suppress irrelevant ones. In each 3D [...] Read more.

This study proposes a three-dimensional (3D) residual attention network (3DRFNet) for human activity recognition by learning spatiotemporal representations from motion pictures. Core innovation integrates the attention mechanism into the 3D ResNet framework to emphasize key features and suppress irrelevant ones. In each 3D ResNet block, channel and spatial attention mechanisms generate attention maps for tensor segments, which are then multiplied by the input feature mapping to emphasize key features. Additionally, the integration of Fast Fourier Convolution (FFC) enhances the network’s capability to effectively capture temporal and spatial features. Simultaneously, we used the cross-entropy loss function to describe the difference between the predicted value and GT to guide the model’s backpropagation. Subsequent experimental results have demonstrated that 3DRFNet achieved SOTA performance in human action recognition. 3DRFNet achieved accuracies of 91.7% and 98.7% on the HMDB-51 and UCF-101 datasets, respectively, which highlighted 3DRFNet’s advantages in recognition accuracy and robustness, particularly in effectively capturing key behavioral features in videos using both attention mechanisms. Full article

(This article belongs to the Special Issue Application of Information Theory to Computer Vision and Image Processing II)

► Show Figures

Figure 1

23 pages, 8280 KiB

Open AccessArticle

From Task Distributions to Expected Paths Lengths Distributions: Value Function Initialization in Sparse Reward Environments for Lifelong Reinforcement Learning

by Soumia Mehimeh and Xianglong Tang

Entropy 2025, 27(4), 367; https://doi.org/10.3390/e27040367 (registering DOI) - 30 Mar 2025

Abstract

This paper studies value function transfer within reinforcement learning frameworks, focusing on tasks continuously assigned to an agent through a probabilistic distribution. Specifically, we focus on environments characterized by sparse rewards with a terminal goal. Initially, we propose and theoretically demonstrate that the [...] Read more.

This paper studies value function transfer within reinforcement learning frameworks, focusing on tasks continuously assigned to an agent through a probabilistic distribution. Specifically, we focus on environments characterized by sparse rewards with a terminal goal. Initially, we propose and theoretically demonstrate that the distribution of the computed value function from such environments, whether in cases where the goals or the dynamics are changing across tasks, can be reformulated as the distribution of the number of steps to the goal generated by their optimal policies, which we name the expected optimal path length. To test our propositions, we hypothesize that the distribution of the expected optimal path lengths resulting from the task distribution is normal. This claim leads us to propose that if the distribution is normal, then the distribution of the value function follows a log-normal pattern. Leveraging this insight, we introduce “LogQInit” as a novel value function transfer method, based on the properties of log-normality. Finally, we run experiments on a scenario of goals and dynamics distributions, validate our proposition by providing an a dequate analysis of the results, and demonstrate that LogQInit outperforms existing methods of value function initialization, policy transfer, and reward shaping. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

20 pages, 342 KiB

Open AccessArticle

Generalized Orthogonal de Bruijn and Kautz Sequences

by Yuan-Pon Chen, Jin Sima and Olgica Milenkovic

Entropy 2025, 27(4), 366; https://doi.org/10.3390/e27040366 (registering DOI) - 30 Mar 2025

Abstract

A de Bruijn sequence of order k over a finite alphabet is a cyclic sequence with the property that it contains every possible k-sequence as a substring exactly once. Orthogonal de Bruijn sequences are the collections of de Bruijn sequences of the [...] Read more.

A de Bruijn sequence of order k over a finite alphabet is a cyclic sequence with the property that it contains every possible k-sequence as a substring exactly once. Orthogonal de Bruijn sequences are the collections of de Bruijn sequences of the same order, k, that satisfy the joint constraint that every

(k + 1)

-sequence appears as a substring in, at most, one of the sequences in the collection. Both de Bruijn and orthogonal de Bruijn sequences have found numerous applications in synthetic biology, although the latter remain largely unexplored in the coding theory literature. Here, we study three relevant practical generalizations of orthogonal de Bruijn sequences, where we relax either the constraint that every

(k + 1)

-sequence appears exactly once or the sequences themselves are de Bruijn rather than balanced de Bruijn sequences. We also provide lower and upper bounds on the number of fixed-weight orthogonal de Bruijn sequences. The paper concludes with parallel results for orthogonal nonbinary Kautz sequences, which satisfy similar constraints as de Bruijn sequences, except for being only required to cover all subsequences of length k whose maximum run length equals one. Full article

(This article belongs to the Special Issue Coding and Algorithms for DNA-Based Data Storage Systems)

► Show Figures

Figure 1

33 pages, 2471 KiB

Open AccessArticle

Exploring the Evolution-Coupling Hypothesis: Do Enzymes’ Performance Gains Correlate with Increased Dissipation?

by Davor Juretić

Entropy 2025, 27(4), 365; https://doi.org/10.3390/e27040365 - 29 Mar 2025

Abstract

The research literature presents divergent opinions regarding the role of dissipation in living systems, with views ranging from it being useless to it being essential for driving life. The implications of universal thermodynamic evolution are often overlooked or considered controversial. A higher rate [...] Read more.

The research literature presents divergent opinions regarding the role of dissipation in living systems, with views ranging from it being useless to it being essential for driving life. The implications of universal thermodynamic evolution are often overlooked or considered controversial. A higher rate of entropy production indicates faster thermodynamic evolution. We calculated enzyme-associated dissipation under steady-state conditions using minimalistic models of enzyme kinetics when all microscopic rate constants are known. We found that dissipation is roughly proportional to the turnover number, and a log-log power-law relationship exists between dissipation and the catalytic efficiency of enzymes. “Perfect” specialized enzymes exhibit the highest dissipation levels and represent the pinnacle of biological evolution. The examples that we analyzed suggested two key points: (a) more evolved enzymes excel in free-energy dissipation, and (b) the proposed evolutionary trajectory from generalist to specialized enzymes should involve increased dissipation for the latter. Introducing stochastic noise in the kinetics of individual enzymes may lead to optimal performance parameters that exceed the observed values. Our findings indicate that biological evolution has opened new channels for dissipation through specialized enzymes. We also discuss the implications of our results concerning scaling laws and the seamless coupling between thermodynamic and biological evolution in living systems immersed in out-of-equilibrium environments. Full article

► Show Figures

Figure 1

17 pages, 2459 KiB

Open AccessArticle

Entropy Profiles for Li-Ion Batteries—Effects of Chemistries and Degradation

by Julia Wind and Preben J. S. Vie

Entropy 2025, 27(4), 364; https://doi.org/10.3390/e27040364 - 29 Mar 2025

Abstract

This paper presents entropy measurements for a large set of commercial Li-ion cells. We present entropy data on full cells with a variety of common Li-ion cell electrode chemistries; graphite, hard carbon, lithium-titanium-oxide (LTO), lithium cobalt-oxide (LCO), nickel manganese cobalt oxides (NMC), nickel [...] Read more.

This paper presents entropy measurements for a large set of commercial Li-ion cells. We present entropy data on full cells with a variety of common Li-ion cell electrode chemistries; graphite, hard carbon, lithium-titanium-oxide (LTO), lithium cobalt-oxide (LCO), nickel manganese cobalt oxides (NMC), nickel cobalt aluminium oxide (NCA), lithium iron-phosphate (LFP), as well as electrodes with mixes of these. All data were collected using an accelerated potentiometric method in steps of approximately 5% State-of-Charge (SoC) across the full SoC window. We observe that the entropy profiles depend on the chemistry of the Li-ion cells, but that they also vary between different commercial cells with the same chemistry. Entropy contributions are quantified with respect to both, their means, positive and negative contributions as well as their SoC variation. In addition, we present how different cyclic ageing temperatures change the entropy profiles for a selected commercial Li-ion cell through ageing. A clear difference in entropy profiles is observed after a capacity loss of 20%. This difference can be attributed to different ageing mechanisms within the Li-ion cells, leading to changes in the balancing of electrodes, as well as changes in the electrode materials. Full article

(This article belongs to the Special Issue The Entropy Production—as Cornerstone in Applied Nonequilibrium Thermodynamics—Dedicated to Professor Signe Kjelstrup on the Occasion of Her 75th Birthday)

► Show Figures

Figure 1

20 pages, 4435 KiB

Open AccessArticle

OMAL: A Multi-Label Active Learning Approach from Data Streams

by Qiao Fang, Chen Xiang, Jicong Duan, Benallal Soufiyan, Changbin Shao, Xibei Yang, Sen Xu and Hualong Yu

Entropy 2025, 27(4), 363; https://doi.org/10.3390/e27040363 - 29 Mar 2025

Abstract

With the rapid growth of digital computing, communication, and storage devices applied in various real-world scenarios, more and more data have been collected and stored to drive the development of machine learning techniques. It is also noted that the data that emerge in [...] Read more.

With the rapid growth of digital computing, communication, and storage devices applied in various real-world scenarios, more and more data have been collected and stored to drive the development of machine learning techniques. It is also noted that the data that emerge in real-world applications tend to become more complex. In this study, we regard a complex data type, i.e., multi-label data, acquired with a time constraint in a dynamic online scenario. Under such conditions, constructing a learning model has to face two challenges: it requires dynamically adapting the variances in label correlations and imbalanced data distributions and it requires more labeling consumptions. To solve these two issues, we propose a novel online multi-label active learning (OMAL) algorithm that considers simultaneously adopting uncertainty (using the average entropy of prediction probabilities) and diversity (using the average cosine distance between feature vectors) as an active query strategy. Specifically, to focus on label correlations, we use a classifier chain (CC) as the multi-label learning model and design a label co-occurrence ranking strategy to arrange label sequence in CC. To adapt the naturally imbalanced distribution of the multi-label data, we select weight extreme learning machine (WELM) as the basic binary-class classifier in CC. The experimental results on ten benchmark multi-label datasets that were transformed into streams show that our proposed method is superior to several popular static multi-label active learning algorithms in terms of both the Macro-F1 and Micro-F1 metrics, indicating its specifical adaptions in the dynamic data stream environment. Full article

(This article belongs to the Section Signal and Data Analysis)

► Show Figures

Figure 1

26 pages, 615 KiB

Open AccessArticle

Vector Flows That Compute the Capacity of Discrete Memoryless Channels

by Guglielmo Beretta and Marcello Pelillo

Entropy 2025, 27(4), 362; https://doi.org/10.3390/e27040362 - 29 Mar 2025

Abstract

One of the fundamental problems of information theory, since its foundation by C. Shannon, has been the computation of the capacity of a discrete memoryless channel, a quantity expressing the maximum rate at which information can travel through the channel. In this paper, [...] Read more.

One of the fundamental problems of information theory, since its foundation by C. Shannon, has been the computation of the capacity of a discrete memoryless channel, a quantity expressing the maximum rate at which information can travel through the channel. In this paper, we investigate the properties of a novel approach to computing the capacity, based on a continuous-time dynamical system. Interestingly, the proposed dynamical system can be regarded as a continuous-time version of the classical Blahut–Arimoto algorithm, and we can prove that the former shares with the latter an exponential rate of convergence if certain conditions are met. Moreover, a circuit design is presented to implement the dynamics, hence enabling analog computation to estimate the capacity. Full article

(This article belongs to the Section Information Theory, Probability and Statistics)

► Show Figures

Figure 1

18 pages, 962 KiB

Open AccessArticle

Lower Limit of Percolation Threshold on Square Lattice with Complex Neighborhoods

by Antoni Piotr Ciepłucha, Marcin Utnicki, Maciej Wołoszyn and Krzysztof Malarz

Entropy 2025, 27(4), 361; https://doi.org/10.3390/e27040361 - 29 Mar 2025

Abstract

In this paper, the 60-year-old concept of long-range interaction in percolation problems introduced by Dalton, Domb and Sykes is reconsidered. With Monte Carlo simulation—based on the Newman–Ziff algorithm and the finite-size scaling hypothesis—we estimate 64 percolation thresholds for a random site percolation problem [...] Read more.

In this paper, the 60-year-old concept of long-range interaction in percolation problems introduced by Dalton, Domb and Sykes is reconsidered. With Monte Carlo simulation—based on the Newman–Ziff algorithm and the finite-size scaling hypothesis—we estimate 64 percolation thresholds for a random site percolation problem on a square lattice with neighborhoods that contain sites from the seventh coordination zone. The percolation thresholds obtained range from

0.27013

(for the neighborhood that contains only sites from the seventh coordination zone) to

0.11535

(for the neighborhood that contains all sites from the first to the seventh coordination zone). Similarly to neighborhoods with smaller ranges, the power-law dependence of the percolation threshold on the effective coordination number with an exponent close to

- 1 / 2

is observed. Finally, we empirically determine the limit of the percolation threshold on square lattices with complex neighborhoods. This limit scales with the inverse square of the mean radius of the neighborhood. The boundary of this limit is touched for threshold values associated with extended (compact) neighborhoods. Full article

(This article belongs to the Special Issue Percolation in the 21st Century)

20 pages, 3634 KiB

Open AccessArticle

LOMDP: Maximizing Desired Opinions in Social Networks by Considering User Expression Intentions

by Xuan Wang, Bin Wu and Tong Wu

Entropy 2025, 27(4), 360; https://doi.org/10.3390/e27040360 - 29 Mar 2025

Abstract

To address the problem of maximizing desired opinions in social networks, we present the Limited Opinion Maximization with Dynamic Propagation Optimization framework, which is grounded in information entropy theory. Innovatively, we introduce the concept of node expression capacity, which quantifies the uncertainty of [...] Read more.

To address the problem of maximizing desired opinions in social networks, we present the Limited Opinion Maximization with Dynamic Propagation Optimization framework, which is grounded in information entropy theory. Innovatively, we introduce the concept of node expression capacity, which quantifies the uncertainty of users’ expression intentions via entropy and effectively identifies the impact of silent nodes on the propagation process. Based on this, in terms of seed node selection, we develop the Limited Opinion Maximization algorithm for multi-stage seed selection, which dynamically optimizes the seed distribution among communities through a multi-stage seeding approach. In terms of node opinion changes, we establish the LODP dynamic opinion propagation model, reconstructing the node opinion update mechanism and explicitly modeling the entropy-increasing effect of silent nodes on the information propagation path. The experimental results on four datasets show that LOMDP outperforms six baseline algorithms. Our research effectively resolves the problem of maximizing desired opinions and offers insights into the dynamics of information propagation in social networks from the perspective of entropy and information theory. Full article

(This article belongs to the Special Issue Spreading Dynamics in Complex Networks)

► Show Figures

Figure 1

26 pages, 879 KiB

Open AccessArticle

Design and Implementation of Low-Complexity Multiple Symbol Detection Algorithm Using Hybrid Stochastic Computing in Aircraft Wireless Communications

by Yukai Liu, Rongke Liu, Kairui Tian, Zheng Lu and Ling Zhao

Entropy 2025, 27(4), 359; https://doi.org/10.3390/e27040359 - 28 Mar 2025

Abstract

The Multiple Symbol Detection (MSD) algorithm can effectively lower the demodulation threshold in Frequency Modulation (FM) technology, which is widely used in aircraft wireless communications due to its insensitivity to large Doppler shifts. However, the high computational complexity of the MSD algorithm leads [...] Read more.

The Multiple Symbol Detection (MSD) algorithm can effectively lower the demodulation threshold in Frequency Modulation (FM) technology, which is widely used in aircraft wireless communications due to its insensitivity to large Doppler shifts. However, the high computational complexity of the MSD algorithm leads to considerable hardware resource overhead. In this paper, we propose a novel MSD architecture based on hybrid stochastic computing (SC), which allows for accurate signal detection while maintaining low hardware complexity. Given that the correlation calculation dominates the computational load in the MSD algorithm, we develop an SC-based, low-complexity unit to perform complex correlation operations using simple hardware circuits, significantly reducing the hardware overhead. Particularly, we integrate a flexible and scalable stochastic adder in the SC-based correlation calculation, which incorporates an adjustable scaling factor to enable high distinguishability in all possible correlation results. Additionally, for the symbol decision process of the MSD algorithm, we design a binary computing-based pipeline architecture to execute the computing process serially, which leverages the inherent low update rate of SC-based correlation results to further reduce the overall resource overhead. Experimental results show that, compared to an 8-bit quantization MSD implementation, our proposed hybrid SC-based MSD architecture achieves a comparable bit error rate while reducing the hardware resources to 69%, 45%, and 36% of those required for the three-, five-, and seven-symbol MSD algorithms, respectively. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

► Show Figures

Figure 1

21 pages, 737 KiB

Open AccessArticle

A Model for the Formation of Beliefs and Social Norms Based on the Satisfaction Problem (SAT)

by Bastien Chopard, Franck Raynaud and Julien Stalhandske

Entropy 2025, 27(4), 358; https://doi.org/10.3390/e27040358 - 28 Mar 2025

Abstract

We propose a numerical representation of beliefs in social systems based on the so-called SAT problem in computer science. The main idea is that a belief system is a set of true/false values associated with claims or propositions. Each individual assigns these values [...] Read more.

We propose a numerical representation of beliefs in social systems based on the so-called SAT problem in computer science. The main idea is that a belief system is a set of true/false values associated with claims or propositions. Each individual assigns these values according to its cognitive system in order to minimize logical contradictions, thus trying to solve a satisfaction problem. Social interactions between agents that disagree on a proposition can be introduced in order to see how, in the long term, social norms and competing belief systems build up in a population. Among other metrics, entropy is used to characterize the diversity of belief systems. Full article

(This article belongs to the Special Issue Entropy-Based Applications in Sociophysics II)

► Show Figures

Figure 1

25 pages, 1002 KiB

Open AccessArticle

InfoMat: Leveraging Information Theory to Visualize and Understand Sequential Data

by Dor Tsur and Haim Permuter

Entropy 2025, 27(4), 357; https://doi.org/10.3390/e27040357 - 28 Mar 2025

Abstract

Despite the widespread use of information measures in analyzing probabilistic systems, effective visualization tools for understanding complex dependencies in sequential data are scarce. In this work, we introduce the information matrix (InfoMat), a novel and intuitive matrix representation of information transfer in sequential [...] Read more.

Despite the widespread use of information measures in analyzing probabilistic systems, effective visualization tools for understanding complex dependencies in sequential data are scarce. In this work, we introduce the information matrix (InfoMat), a novel and intuitive matrix representation of information transfer in sequential systems. InfoMat provides a structured visual perspective on mutual information decompositions, enabling the discovery of new relationships between sequential information measures and enhancing interpretability in time series data analytics. We demonstrate how InfoMat captures key sequential information measures, such as directed information and transfer entropy. To facilitate its application in real-world datasets, we propose both an efficient Gaussian mutual information estimator and a neural InfoMat estimator based on masked autoregressive flows to model more complex dependencies. These estimators make InfoMat a valuable tool for uncovering hidden patterns in data analytics applications, encompassing neuroscience, finance, communication systems, and machine learning. We further illustrate the utility of InfoMat in visualizing information flow in real-world sequential physiological data analysis and in visualizing information flow in communication channels under various coding schemes. By mapping visual patterns in InfoMat to various modes of dependence structures, we provide a data-driven framework for analyzing causal relationships and temporal interactions. InfoMat thus serves as both a theoretical and empirical tool for data-driven decision making, bridging the gap between information theory and applied data analytics. Full article

(This article belongs to the Special Issue Information-Theoretic Methods in Data Analytics)

► Show Figures

Figure 1

Journal Description

Entropy

Latest Articles

Journal Menu

Journal Browser

Highly Accessed Articles

Latest Books

E-Mail Alert

News

Topics

Conferences

Special Issues

Topical Collections

Further Information

Guidelines

MDPI Initiatives

Follow MDPI