Next Issue
Previous Issue

E-Mail Alert

Add your e-mail address to receive forthcoming issues of this journal:

Journal Browser

Journal Browser

Table of Contents

Entropy, Volume 19, Issue 5 (May 2017)

  • Issues are regarded as officially published after their release is announced to the table of contents alert mailing list.
  • You may sign up for e-mail alerts to receive table of contents of newly released issues.
  • PDF is the official format for papers published in both, html and pdf forms. To view the papers in pdf format, click on the "PDF Full-text" link, and use the free Adobe Readerexternal link to open them.
Cover Story (view full-size image) Any system can be described at a multitude of spatial and temporal scales. One long-standing [...] Read more.
View options order results:
result details:
Displaying articles 1-54
Export citation of selected articles as:
Open AccessArticle Lyapunov Spectra of Coulombic and Gravitational Periodic Systems
Entropy 2017, 19(5), 238; https://doi.org/10.3390/e19050238
Received: 2 April 2017 / Revised: 15 May 2017 / Accepted: 15 May 2017 / Published: 20 May 2017
Cited by 1 | PDF Full-text (545 KB) | HTML Full-text | XML Full-text
Abstract
An open question in nonlinear dynamics is the relation between the Kolmogorov entropy and the largest Lyapunov exponent of a given orbit. Both have been shown to have diagnostic capability for phase transitions in thermodynamic systems. For systems with long-range interactions, the choice
[...] Read more.
An open question in nonlinear dynamics is the relation between the Kolmogorov entropy and the largest Lyapunov exponent of a given orbit. Both have been shown to have diagnostic capability for phase transitions in thermodynamic systems. For systems with long-range interactions, the choice of boundary plays a critical role and appropriate boundary conditions must be invoked. In this work, we compute Lyapunov spectra for Coulombic and gravitational versions of the one-dimensional systems of parallel sheets with periodic boundary conditions. Exact expressions for time evolution of the tangent-space vectors are derived and are utilized toward computing Lypaunov characteristic exponents using an event-driven algorithm. The results indicate that the energy dependence of the largest Lyapunov exponent emulates that of Kolmogorov entropy for each system for a given system size. Our approach forms an effective and approximation-free instrument for studying the dynamical properties exhibited by the Coulombic and gravitational systems and finds applications in investigating indications of thermodynamic transitions in small as well as large versions of the spatially periodic systems. When a phase transition exists, we find that the largest Lyapunov exponent serves as a precursor of the transition that becomes more pronounced as the system size increases. Full article
(This article belongs to the Special Issue Thermodynamics and Statistical Mechanics of Small Systems)
Figures

Figure 1

Open AccessArticle Can a Robot Have Free Will?
Entropy 2017, 19(5), 237; https://doi.org/10.3390/e19050237
Received: 27 February 2017 / Revised: 28 April 2017 / Accepted: 15 May 2017 / Published: 20 May 2017
Cited by 1 | PDF Full-text (552 KB) | HTML Full-text | XML Full-text
Abstract
Using insights from cybernetics and an information-based understanding of biological systems, a precise, scientifically inspired, definition of free-will is offered and the essential requirements for an agent to possess it in principle are set out. These are: (a) there must be a self
[...] Read more.
Using insights from cybernetics and an information-based understanding of biological systems, a precise, scientifically inspired, definition of free-will is offered and the essential requirements for an agent to possess it in principle are set out. These are: (a) there must be a self to self-determine; (b) there must be a non-zero probability of more than one option being enacted; (c) there must be an internal means of choosing among options (which is not merely random, since randomness is not a choice). For (a) to be fulfilled, the agent of self-determination must be organisationally closed (a “Kantian whole”). For (c) to be fulfilled: (d) options must be generated from an internal model of the self which can calculate future states contingent on possible responses; (e) choosing among these options requires their evaluation using an internally generated goal defined on an objective function representing the overall “master function” of the agent and (f) for “deep free-will”, at least two nested levels of choice and goal (d–e) must be enacted by the agent. The agent must also be able to enact its choice in physical reality. The only systems known to meet all these criteria are living organisms, not just humans, but a wide range of organisms. The main impediment to free-will in present-day artificial robots, is their lack of being a Kantian whole. Consciousness does not seem to be a requirement and the minimum complexity for a free-will system may be quite low and include relatively simple life-forms that are at least able to learn. Full article
(This article belongs to the Special Issue Complexity, Criticality and Computation (C³)) Printed Edition available
Figures

Figure 1

Open AccessArticle Entropy in Investigation of Vasovagal Syndrome in Passive Head Up Tilt Test
Entropy 2017, 19(5), 236; https://doi.org/10.3390/e19050236
Received: 1 March 2017 / Revised: 15 May 2017 / Accepted: 16 May 2017 / Published: 20 May 2017
PDF Full-text (1984 KB) | HTML Full-text | XML Full-text
Abstract
This paper presents an application of Approximate Entropy (ApEn) and Sample Entropy (SampEn) in the analysis of heart rhythm, blood pressure and stroke volume for the diagnosis of vasovagal syndrome. The analyzed biosignals were recorded during positive passive tilt tests—HUTT(+). Signal changes and
[...] Read more.
This paper presents an application of Approximate Entropy (ApEn) and Sample Entropy (SampEn) in the analysis of heart rhythm, blood pressure and stroke volume for the diagnosis of vasovagal syndrome. The analyzed biosignals were recorded during positive passive tilt tests—HUTT(+). Signal changes and their entropy were compared in three main phases of the test: supine position, tilt, and pre-syncope, with special focus on the latter, which was analyzed in a sliding window of each signal. In some cases, ApEn and SampEn were equally useful for the assessment of signal complexity (p < 0.05 in corresponding calculations). The complexity of the signals was found to decrease in the pre-syncope phase (SampEn (RRI): 1.20–0.34, SampEn (sBP): 1.29–0.57, SampEn (dBP): 1.19–0.48, SampEn (SV): 1.62–0.91). The pattern of the SampEn (SV) decrease differs from the pattern of the SampEn (sBP), SampEn (dBP) and SampEn (RRI) decrease. For all signals, the lowest entropy values in the pre-syncope phase were observed at the moment when loss of consciousness occurred. Full article
(This article belongs to the Special Issue Entropy and Cardiac Physics II)
Figures

Figure 1

Open AccessFeature PaperArticle On Linear Coding over Finite Rings and Applications to Computing
Entropy 2017, 19(5), 233; https://doi.org/10.3390/e19050233
Received: 6 January 2017 / Revised: 24 April 2017 / Accepted: 15 May 2017 / Published: 20 May 2017
PDF Full-text (436 KB) | HTML Full-text | XML Full-text
Abstract
This paper presents a coding theorem for linear coding over finite rings, in the setting of the Slepian–Wolf source coding problem. This theorem covers corresponding achievability theorems of Elias (IRE Conv. Rec. 1955, 3, 37–46) and Csiszár (IEEE Trans. Inf. Theory
[...] Read more.
This paper presents a coding theorem for linear coding over finite rings, in the setting of the Slepian–Wolf source coding problem. This theorem covers corresponding achievability theorems of Elias (IRE Conv. Rec. 1955, 3, 37–46) and Csiszár (IEEE Trans. Inf. Theory 1982, 28, 585–592) for linear coding over finite fields as special cases. In addition, it is shown that, for any set of finite correlated discrete memoryless sources, there always exists a sequence of linear encoders over some finite non-field rings which achieves the data compression limit, the Slepian–Wolf region. Hence, the optimality problem regarding linear coding over finite non-field rings for data compression is closed with positive confirmation with respect to existence. For application, we address the problem of source coding for computing, where the decoder is interested in recovering a discrete function of the data generated and independently encoded by several correlated i.i.d. random sources. We propose linear coding over finite rings as an alternative solution to this problem. Results in Körner–Marton (IEEE Trans. Inf. Theory 1979, 25, 219–221) and Ahlswede–Han (IEEE Trans. Inf. Theory 1983, 29, 396–411, Theorem 10) are generalized to cases for encoding (pseudo) nomographic functions (over rings). Since a discrete function with a finite domain always admits a nomographic presentation, we conclude that both generalizations universally apply for encoding all discrete functions of finite domains. Based on these, we demonstrate that linear coding over finite rings strictly outperforms its field counterpart in terms of achieving better coding rates and reducing the required alphabet sizes of the encoders for encoding infinitely many discrete functions. Full article
(This article belongs to the Special Issue Network Information Theory)
Figures

Figure 1

Open AccessArticle The Particle as a Statistical Ensemble of Events in Stueckelberg–Horwitz–Piron Electrodynamics
Entropy 2017, 19(5), 234; https://doi.org/10.3390/e19050234
Received: 8 March 2017 / Revised: 15 May 2017 / Accepted: 17 May 2017 / Published: 19 May 2017
Cited by 1 | PDF Full-text (263 KB) | HTML Full-text | XML Full-text
Abstract
In classical Maxwell electrodynamics, charged particles following deterministic trajectories are described by currents that induce fields, mediating interactions with other particles. Statistical methods are used when needed to treat complex particle and/or field configurations. In Stueckelberg–Horwitz–Piron (SHP) electrodynamics, the classical trajectories are traced
[...] Read more.
In classical Maxwell electrodynamics, charged particles following deterministic trajectories are described by currents that induce fields, mediating interactions with other particles. Statistical methods are used when needed to treat complex particle and/or field configurations. In Stueckelberg–Horwitz–Piron (SHP) electrodynamics, the classical trajectories are traced out dynamically, through the evolution of a 4D spacetime event x μ ( τ ) as τ grows monotonically. Stueckelberg proposed to formalize the distinction between coordinate time x 0 = c t (measured by laboratory clocks) and chronology τ (the temporal ordering of event occurrence) in order to describe antiparticles and resolve problems of irreversibility such as grandfather paradoxes. Consequently, in SHP theory, the elementary object is not a particle (a 4D curve in spacetime) but rather an event (a single point along the dynamically evolving curve). Following standard deterministic methods in classical relativistic field theory, one is led to Maxwell-like field equations that are τ -dependent and sourced by a current that represents a statistical ensemble of instantaneous events distributed along the trajectory. The width λ of this distribution defines a correlation time for the interactions and a mass spectrum for the photons emitted by particles. As λ becomes very large, the photon mass goes to zero and the field equations become τ -independent Maxwell’s equations. Maxwell theory thus emerges as an equilibrium limit of SHP, in which λ is larger than any other relevant time scale. Thus, statistical mechanics is a fundamental ingredient in SHP electrodynamics, and its insights are required to give meaning to the concept of a particle. Full article
(This article belongs to the Section Statistical Mechanics)
Open AccessArticle A Kullback–Leibler View of Maximum Entropy and Maximum Log-Probability Methods
Entropy 2017, 19(5), 232; https://doi.org/10.3390/e19050232
Received: 2 March 2017 / Revised: 30 April 2017 / Accepted: 15 May 2017 / Published: 19 May 2017
Cited by 1 | PDF Full-text (940 KB) | HTML Full-text | XML Full-text
Abstract
Entropy methods enable a convenient general approach to providing a probability distribution with partial information. The minimum cross-entropy principle selects the distribution that minimizes the Kullback–Leibler divergence subject to the given constraints. This general principle encompasses a wide variety of distributions, and generalizes
[...] Read more.
Entropy methods enable a convenient general approach to providing a probability distribution with partial information. The minimum cross-entropy principle selects the distribution that minimizes the Kullback–Leibler divergence subject to the given constraints. This general principle encompasses a wide variety of distributions, and generalizes other methods that have been proposed independently. There remains, however, some confusion about the breadth of entropy methods in the literature. In particular, the asymmetry of the Kullback–Leibler divergence provides two important special cases when the target distribution is uniform: the maximum entropy method and the maximum log-probability method. This paper compares the performance of both methods under a variety of conditions. We also examine a generalized maximum log-probability method as a further demonstration of the generality of the entropy approach. Full article
(This article belongs to the Section Information Theory)
Figures

Figure 1

Open AccessArticle A Novel Faults Diagnosis Method for Rolling Element Bearings Based on EWT and Ambiguity Correlation Classifiers
Entropy 2017, 19(5), 231; https://doi.org/10.3390/e19050231
Received: 20 March 2017 / Revised: 6 May 2017 / Accepted: 15 May 2017 / Published: 18 May 2017
Cited by 3 | PDF Full-text (3529 KB) | HTML Full-text | XML Full-text
Abstract
According to non-stationary characteristic of the acoustic emission signal of rolling element bearings, a novel fault diagnosis method based on empirical wavelet transform (EWT) and ambiguity correlation classification (ACC) is proposed. In the proposed method, the acoustic emission signal acquired from a one-channel
[...] Read more.
According to non-stationary characteristic of the acoustic emission signal of rolling element bearings, a novel fault diagnosis method based on empirical wavelet transform (EWT) and ambiguity correlation classification (ACC) is proposed. In the proposed method, the acoustic emission signal acquired from a one-channel sensor is firstly decomposed using the EWT method, and then the mutual information of decomposed components and the original signal is computed and used to extract the noiseless component in order to obtain the reconstructed signal. Afterwards, the ambiguity correlation classifier, which has the advantages of ambiguity functions in the processing of the non-stationary signal, and the combining of correlation coefficients, is applied. Finally, multiple datasets of reconstructed signals for different operative conditions are fed to the ambiguity correlation classifier for training and testing. The proposed method was verified by experiments, and experimental results have shown that the proposed method can effectively diagnose three different operative conditions of rolling element bearings with higher detection rates than support vector machine and back-propagation (BP) neural network algorithms. Full article
(This article belongs to the Section Information Theory)
Figures

Graphical abstract

Open AccessArticle Specific and Complete Local Integration of Patterns in Bayesian Networks
Entropy 2017, 19(5), 230; https://doi.org/10.3390/e19050230
Received: 19 March 2017 / Revised: 11 May 2017 / Accepted: 12 May 2017 / Published: 18 May 2017
Cited by 1 | PDF Full-text (1119 KB) | HTML Full-text | XML Full-text
Abstract
We present a first formal analysis of specific and complete local integration. Complete local integration was previously proposed as a criterion for detecting entities or wholes in distributed dynamical systems. Such entities in turn were conceived to form the basis of a theory
[...] Read more.
We present a first formal analysis of specific and complete local integration. Complete local integration was previously proposed as a criterion for detecting entities or wholes in distributed dynamical systems. Such entities in turn were conceived to form the basis of a theory of emergence of agents within dynamical systems. Here, we give a more thorough account of the underlying formal measures. The main contribution is the disintegration theorem which reveals a special role of completely locally integrated patterns (what we call ι-entities) within the trajectories they occur in. Apart from proving this theorem we introduce the disintegration hierarchy and its refinement-free version as a way to structure the patterns in a trajectory. Furthermore, we construct the least upper bound and provide a candidate for the greatest lower bound of specific local integration. Finally, we calculate the ι -entities in small example systems as a first sanity check and find that ι -entities largely fulfil simple expectations. Full article
(This article belongs to the Special Issue Complexity, Criticality and Computation (C³)) Printed Edition available
Figures

Figure 1

Open AccessArticle Ion Hopping and Constrained Li Diffusion Pathways in the Superionic State of Antifluorite Li2O
Entropy 2017, 19(5), 227; https://doi.org/10.3390/e19050227
Received: 21 March 2017 / Revised: 25 April 2017 / Accepted: 15 May 2017 / Published: 18 May 2017
Cited by 2 | PDF Full-text (2600 KB) | HTML Full-text | XML Full-text
Abstract
Li2O belongs to the family of antifluorites that show superionic behavior at high temperatures. While some of the superionic characteristics of Li2O are well-known, the mechanistic details of ionic conduction processes are somewhat nebulous. In this work, we first
[...] Read more.
Li2O belongs to the family of antifluorites that show superionic behavior at high temperatures. While some of the superionic characteristics of Li2O are well-known, the mechanistic details of ionic conduction processes are somewhat nebulous. In this work, we first establish an onset of superionic conduction that is emblematic of a gradual disordering process among the Li ions at a characteristic temperature Tα (~1000 K) using reported neutron diffraction data and atomistic simulations. In the superionic state, the Li ions are observed to portray dynamic disorder by hopping between the tetrahedral lattice sites. We then show that string-like ionic diffusion pathways are established among the Li ions in the superionic state. The diffusivity of these dynamical string-like structures, which have a finite lifetime, shows a remarkable correlation to the bulk diffusivity of the system. Full article
(This article belongs to the Special Issue Understanding Molecular Dynamics via Stochastic Processes)
Figures

Figure 1

Open AccessArticle Investigation of the Intra- and Inter-Limb Muscle Coordination of Hands-and-Knees Crawling in Human Adults by Means of Muscle Synergy Analysis
Entropy 2017, 19(5), 229; https://doi.org/10.3390/e19050229
Received: 27 March 2017 / Revised: 25 April 2017 / Accepted: 15 May 2017 / Published: 17 May 2017
Cited by 1 | PDF Full-text (3987 KB) | HTML Full-text | XML Full-text
Abstract
To investigate the intra- and inter-limb muscle coordination mechanism of human hands-and-knees crawling by means of muscle synergy analysis, surface electromyographic (sEMG) signals of 20 human adults were collected bilaterally from 32 limb related muscles during crawling with hands and knees at different
[...] Read more.
To investigate the intra- and inter-limb muscle coordination mechanism of human hands-and-knees crawling by means of muscle synergy analysis, surface electromyographic (sEMG) signals of 20 human adults were collected bilaterally from 32 limb related muscles during crawling with hands and knees at different speeds. The nonnegative matrix factorization (NMF) algorithm was exerted on each limb to extract muscle synergies. The results showed that intra-limb coordination was relatively stable during human hands-and-knees crawling. Two synergies, one relating to the stance phase and the other relating to the swing phase, could be extracted from each limb during a crawling cycle. Synergy structures during different speeds kept good consistency, but the recruitment levels, durations, and phases of muscle synergies were adjusted to adapt the change of crawling speed. Furthermore, the ipsilateral phase lag (IPL) value which was used to depict the inter-limb coordination changed with crawling speed for most subjects, and subjects using the no-limb-pairing mode at low speed tended to adopt the trot-like mode or pace-like mode at high speed. The research results could be well explained by the two-level central pattern generator (CPG) model consisting of a half-center rhythm generator (RG) and a pattern formation (PF) circuit. This study sheds light on the underlying control mechanism of human crawling. Full article
(This article belongs to the Special Issue Information Theory Applied to Physiological Signals)
Figures

Figure 1

Open AccessArticle Face Verification with Multi-Task and Multi-Scale Feature Fusion
Entropy 2017, 19(5), 228; https://doi.org/10.3390/e19050228
Received: 18 March 2017 / Revised: 5 May 2017 / Accepted: 13 May 2017 / Published: 17 May 2017
Cited by 2 | PDF Full-text (2416 KB) | HTML Full-text | XML Full-text
Abstract
Face verification for unrestricted faces in the wild is a challenging task. This paper proposes a method based on two deep convolutional neural networks (CNN) for face verification. In this work, we explore using identification signals to supervise one CNN and the combination
[...] Read more.
Face verification for unrestricted faces in the wild is a challenging task. This paper proposes a method based on two deep convolutional neural networks (CNN) for face verification. In this work, we explore using identification signals to supervise one CNN and the combination of semi-verification and identification to train the other one. In order to estimate semi-verification loss at a low computation cost, a circle, which is composed of all faces, is used for selecting face pairs from pairwise samples. In the process of face normalization, we propose using different landmarks of faces to solve the problems caused by poses. In addition, the final face representation is formed by the concatenating feature of each deep CNN after principal component analysis (PCA) reduction. Furthermore, each feature is a combination of multi-scale representations through making use of auxiliary classifiers. For the final verification, we only adopt the face representation of one region and one resolution of a face jointing Joint Bayesian classifier. Experiments show that our method can extract effective face representation with a small training dataset and our algorithm achieves 99.71% verification accuracy on Labeled Faces in the Wild (LFW) dataset. Full article
(This article belongs to the Special Issue Information Theory in Machine Learning and Data Science)
Figures

Figure 1

Open AccessArticle Information Entropy and Measures of Market Risk
Entropy 2017, 19(5), 226; https://doi.org/10.3390/e19050226
Received: 29 March 2017 / Revised: 8 May 2017 / Accepted: 11 May 2017 / Published: 16 May 2017
Cited by 1 | PDF Full-text (1818 KB) | HTML Full-text | XML Full-text
Abstract
In this paper we investigate the relationship between the information entropy of the distribution of intraday returns and intraday and daily measures of market risk. Using data on the EUR/JPY exchange rate, we find a negative relationship between entropy and intraday Value-at-Risk, and
[...] Read more.
In this paper we investigate the relationship between the information entropy of the distribution of intraday returns and intraday and daily measures of market risk. Using data on the EUR/JPY exchange rate, we find a negative relationship between entropy and intraday Value-at-Risk, and also between entropy and intraday Expected Shortfall. This relationship is then used to forecast daily Value-at-Risk, using the entropy of the distribution of intraday returns as a predictor. Full article
Figures

Figure 1

Open AccessArticle Entropy Information of Cardiorespiratory Dynamics in Neonates during Sleep
Entropy 2017, 19(5), 225; https://doi.org/10.3390/e19050225
Received: 30 March 2017 / Revised: 11 May 2017 / Accepted: 12 May 2017 / Published: 15 May 2017
Cited by 1 | PDF Full-text (724 KB) | HTML Full-text | XML Full-text
Abstract
Sleep is a central activity in human adults and characterizes most of the newborn infant life. During sleep, autonomic control acts to modulate heart rate variability (HRV) and respiration. Mechanisms underlying cardiorespiratory interactions in different sleep states have been studied but are not
[...] Read more.
Sleep is a central activity in human adults and characterizes most of the newborn infant life. During sleep, autonomic control acts to modulate heart rate variability (HRV) and respiration. Mechanisms underlying cardiorespiratory interactions in different sleep states have been studied but are not yet fully understood. Signal processing approaches have focused on cardiorespiratory analysis to elucidate this co-regulation. This manuscript proposes to analyze heart rate (HR), respiratory variability and their interrelationship in newborn infants to characterize cardiorespiratory interactions in different sleep states (active vs. quiet). We are searching for indices that could detect regulation alteration or malfunction, potentially leading to infant distress. We have analyzed inter-beat (RR) interval series and respiration in a population of 151 newborns, and followed up with 33 at 1 month of age. RR interval series were obtained by recognizing peaks of the QRS complex in the electrocardiogram (ECG), corresponding to the ventricles depolarization. Univariate time domain, frequency domain and entropy measures were applied. In addition, Transfer Entropy was considered as a bivariate approach able to quantify the bidirectional information flow from one signal (respiration) to another (RR series). Results confirm the validity of the proposed approach. Overall, HRV is higher in active sleep, while high frequency (HF) power characterizes more quiet sleep. Entropy analysis provides higher indices for SampEn and Quadratic Sample entropy (QSE) in quiet sleep. Transfer Entropy values were higher in quiet sleep and point to a major influence of respiration on the RR series. At 1 month of age, time domain parameters show an increase in HR and a decrease in variability. No entropy differences were found across ages. The parameters employed in this study help to quantify the potential for infants to adapt their cardiorespiratory responses as they mature. Thus, they could be useful as early markers of risk for infant cardiorespiratory vulnerabilities. Full article
(This article belongs to the Special Issue Entropy and Sleep Disorders)
Figures

Figure 1

Open AccessArticle Classification of Fractal Signals Using Two-Parameter Non-Extensive Wavelet Entropy
Entropy 2017, 19(5), 224; https://doi.org/10.3390/e19050224
Received: 3 March 2017 / Revised: 28 April 2017 / Accepted: 9 May 2017 / Published: 15 May 2017
PDF Full-text (1217 KB) | HTML Full-text | XML Full-text
Abstract
This article proposes a methodology for the classification of fractal signals as stationary or nonstationary. The methodology is based on the theoretical behavior of two-parameter wavelet entropy of fractal signals. The wavelet (q,q)-entropy is a wavelet-based extension
[...] Read more.
This article proposes a methodology for the classification of fractal signals as stationary or nonstationary. The methodology is based on the theoretical behavior of two-parameter wavelet entropy of fractal signals. The wavelet ( q , q ) -entropy is a wavelet-based extension of the ( q , q ) -entropy of Borges and is based on the entropy planes for various q and q ; it is theoretically shown that it constitutes an efficient and effective technique for fractal signal classification. Moreover, the second parameter q provides further analysis flexibility and robustness in the sense that different ( q , q ) pairs can analyze the same phenomena and increase the range of dispersion of entropies. A comparison study against the standard signal summation conversion technique shows that the proposed methodology is not only comparable in accuracy but also more computationally efficient. The application of the proposed methodology to physiological and financial time series is also presented along with the classification of these as stationary or nonstationary. Full article
(This article belongs to the Special Issue Wavelets, Fractals and Information Theory II)
Figures

Figure 1

Open AccessArticle Prediction and Evaluation of Zero Order Entropy Changes in Grammar-Based Codes
Entropy 2017, 19(5), 223; https://doi.org/10.3390/e19050223
Received: 30 January 2017 / Revised: 9 May 2017 / Accepted: 10 May 2017 / Published: 13 May 2017
PDF Full-text (398 KB) | HTML Full-text | XML Full-text
Abstract
The change of zero order entropy is studied over different strategies of grammar production rule selection. The two major rules are distinguished: transformations leaving the message size intact and substitution functions changing the message size. Relations for zero order entropy changes were derived
[...] Read more.
The change of zero order entropy is studied over different strategies of grammar production rule selection. The two major rules are distinguished: transformations leaving the message size intact and substitution functions changing the message size. Relations for zero order entropy changes were derived for both cases and conditions under which the entropy decreases were described. In this article, several different greedy strategies reducing zero order entropy, as well as message sizes are summarized, and the new strategy MinEnt is proposed. The resulting evolution of the zero order entropy is compared with a strategy of selecting the most frequent digram used in the Re-Pair algorithm. Full article
(This article belongs to the Section Information Theory)
Figures

Figure 1

Back to Top