Search Results (34)

In this work, we investigate the possibility of using varentropy, an information measure previously proposed for discrete probability distribution, as a measure of the probabilistic uncertainty of continuous probability distribution. We show that varentropy allows avoiding negative values and some undesirable features of informational entropy encountered while using the Boltzmann–Shannon formula (and others) for continuous probability distributions. Full article

We investigate discrete-time deterministic systems whose trajectories are indexed by the positive cone of a countable linearly ordered group G and evolve on a σ-finite measure space $(\Omega ,\mathcal{B},m)$. The paper operates at two levels. At the measure-theoretic level, the ambient space $(\Omega ,m)$ may be non-atomic and infinite, where strong recurrence, characterized by the absence of weakly wandering sets of positive measure, governs structural properties such as syndeticity of return-time sets, invariance under measure equivalence, and inheritance to subgroups. At the combinatorial level, the dynamics is compressed onto the finite effective state space ${\Omega }_{\mathrm{eff}}$, the set of states actually visited by a trajectory, on which a canonical atomic probability measure μ is constructed via a deliberate support-switch from m. At this level, the entropy results additionally require G to be amenable, where Følner sequences are employed to show that μ is the limiting empirical distribution along almost every trajectory. When G acts transitively on ${\Omega }_{\mathrm{eff}}$, the measure μ is necessarily uniform and the stationary Shannon entropy $H\left(X\right)=-{\sum }_x\mu \left(\left\{x\right\}\right)\log \mu \left(\left\{x\right\}\right)$ achieves its maximum $\log |{\Omega }_{\mathrm{eff}}|$ from structural constraints alone. A generalized orbit-decomposition formula covers the non-transitive case. Results at both levels are illustrated through cyclic shifts, rational rotations, Sturmian shifts, and ${\mathbb{Z}}^2$-actions. Full article

Entropy-based image thresholding is one of the most widely used segmentation techniques in image processing. The Tsallis and Masi entropies are information measures that can capture long-range interactions in various physical systems. On the other hand, Shannon entropy is more appropriate for short-range correlations. In this paper, we have improved a thresholding technique based on Tsallis and Shannon formulas by using Masi entropy. Specifically, we replace the Tsallis information measure with Masi’s one, obtaining better results than the original methodology. As the proposed method depends on an entropic parameter, we designed a thresholding algorithm that incorporates a simulated annealing procedure for parameter optimization. Then, we compared our results with thresholding methods that use just Masi (or Tsallis), or a combination of them, Shannon, Sine, and Hill entropies. The comparison is enriched with a kernel version of a support vector machine, as well as a discussion of our proposal in relation to deep learning approaches. Quantitative measures of segmentation accuracy demonstrated the superior performance of our method in infrared, nondestructive testing (NDT), as well as RGB images from the BSDS500 dataset. Full article

One among the most intriguing results coming from the application of statistical mechanics to the study of the brain is the understanding that it, as a dynamical system, is inherently out of equilibrium. In the realm of non-equilibrium statistical mechanics and stochastic processes, the standard observable computed to determine whether a system is at equilibrium or not is the entropy produced along the dynamics. For this reason, we present here a detailed calculation of the entropy production in the Amari model, a coarse-grained model of the brain neural network, consisting of an integro-differential equation for the neural activity field, when stochasticity is added to the original dynamics. Since the way to add stochasticity is always to some extent arbitrary, particularly for coarse-grained models, there is no general prescription to do so. We precisely investigate the interplay between noise properties and the original model features, discussing in which cases the stationary state is in thermal equilibrium and which cases it is out of equilibrium, providing explicit and simple formulae. Following the derivation for the particular case considered, we also show how the entropy production rate is related to the variation in time of the Shannon entropy of the system. Full article

Fissure swarms and dike swarms in Iceland constitute the main parts of volcanic systems that are 40–150 km long, 5–20 km wide, extend to depths of 10–20 km, and contain 2 × 10¹⁴ outcrop-scale (≥0.1 m) and 10^22–23 down to grain-scale (≥1 mm) fractures, suggesting that statistical physics is an appropriate method of analysis. Length-size distributions of 565 outcrop-scale Holocene fissures (tension fractures and normal faults) and 1041 Neogene dikes show good to excellent fits with negative power laws and exponential laws. Here, the Helmholtz free energy is used to represent the energy supplied to the swarms and to derive the Gibbs–Shannon entropy formula. The calculated entropies of 12 sets and subsets of fissures and 3 sets and subsets of dikes all show strong positive correlations with sets/subsets length ranges and scaling exponents. Statistical physics considerations suggest that, at a given time, the probability of the overall state of stress in a crustal segment being heterogeneous is much greater than the state of stress being homogeneous and favourable to the propagation of a fissure or a dike. In a heterogeneous stress field, most fissures/dikes become arrested after a short propagation—which is a formal explanation of the observed statistical size-length distributions. As the size of the stress-homogenised rock volume increases larger fissures/dikes can form, increasing the length range of the distribution (and its entropy) which may, potentially, transform from an exponential distribution into a power-law distribution. Full article

The main aim of this study is probabilistic computer simulation of the effective physical parameters of fluids containing nanoparticles. A deterministic model following the rule of mixtures and some semi-empirical formulas are employed to calculate effective density, heat conductivity, heat capacity, as well as viscosity for the given nanofluid. This models is randomized here using the Monte-Carlo simulation apparatus for estimation of the Shannon entropy of all these physical parameters, which is the crucial novelty of this study. The volume fraction of the nanoparticles is assumed for this purpose as the Gaussian uncertainty source with the given first two moments. The basic probabilistic characteristics of the nanofluids’ homogenized parameters have also been determined here for some validation of Shannon entropy variations in addition to the statistical disorder of the nanoparticle fraction. These research findings contribute to advancing nanofluidic and microfluidic research, offering robust tools for uncertainty analysis and enhancing the reliability of physical parameter predictions in applications requiring high numerical and/or experimental precision. Full article

Classic formulae for entropy and cross-entropy contain operations ${\displaystyle \frac{x}{0}}$ and ${log}_{2}x$ that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as $0{log}_{2}0$ and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols $\pm \infty$ for signed infinite values and a symbol named ⊥ for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback–Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally. Full article

All characterizations of the Shannon entropy include the so-called chain rule, a formula on a hierarchically structured probability distribution, which is based on at least two elementary distributions. We show that the chain rule can be split into two natural components, the well-known additivity of the entropy in case of cross-products and a variant of the chain rule that involves only a single elementary distribution. The latter is given as a proportionality relation and, hence, allows a vague interpretation as self-similarity, hence intrinsic property of the Shannon entropy. Analogous characterizations are given for the Rényi entropy and its limits, the min-entropy and the Hartley entropy. Full article

A closed chain of oscillators can be considered a model for ring-shaped ecosystems, such as atolls or the coastal zones of inland reservoirs. We use the logistic map, which is often referred to as an archetypical example of how complex dynamics can arise from very simple nonlinear equations, as a model for a separate oscillator in the chain. We present an original algorithm that allows us to find solutions to the spatiotemporal logistic equation quite efficiently or to state with certainty that there are no such solutions. Based on the Shannon formula, we propose formulas for estimating the spatial and temporal entropy, which allow us to classify our solutions as regular or irregular. We show that regular solutions can occur within the Malthus parameter region that corresponds to the irregular dynamics of a solitary logistic map. Full article

Shannon entropy quantifying bi-colored Ramsey complete graphs is introduced and calculated for complete graphs containing up to six vertices. Complete graphs in which vertices are connected with two types of links, labeled as α-links and β-links, are considered. Shannon entropy is introduced according to the classical Shannon formula considering the fractions of monochromatic convex $\alpha$-colored polygons with n α-sides or edges, and the fraction of monochromatic $\beta$-colored convex polygons with m β-sides in the given complete graph. The introduced Shannon entropy is insensitive to the exact shape of the polygons, but it is sensitive to the distribution of monochromatic polygons in a given complete graph. The introduced Shannon entropies $S_{\alpha }$ and $S_{\beta }$ are interpreted as follows: $S_{\alpha }$ is interpreted as an average uncertainty to find the green $\alpha -$polygon in the given graph; $S_{\beta }$ is, in turn, an average uncertainty to find the red $\beta -$polygon in the same graph. The re-shaping of the Ramsey theorem in terms of the Shannon entropy is suggested. Generalization for multi-colored complete graphs is proposed. Various measures quantifying the Shannon entropy of the entire complete bi-colored graphs are suggested. Physical interpretations of the suggested Shannon entropies are discussed. Full article

Soil researchers are interested in a gaining better understanding of the soil system state by analyzing its properties and their dynamics in time as well as in relation to land use change. Tilled, abandoned, and forest soils were assessed regarding attribute–response relationships for the bulk density (BD), total porosity (TP), volumetric moisture (θv), and penetration resistance (PR) with the use of the interquartile ratio (IRI) integrated into a resilience formula and Shannon entropy indices. The IRI results differentiated soil properties according to agrotechnics (wheel track vs. between wheels) and the state of the system (tilled vs. abandoned vineyard). Entropy (En) indicated a high level of uncertainty for PR. The linear regression applied to the pairs of BD-TP, TP-θv, and PR-θv showed better results for the IRI weight (IRI_weight) compared to the entropy weight (En_weight) for the soil between the wheels. The soil of the abandoned vineyard showed a faster tendency toward resilience that was more pronounced in the tilled wheel tracks than in the area between the wheels. The IRI can thus be an alternative to entropy in the evaluation of the response of some soil properties according to their use. When integrated into a resilience formula, the IRI can estimate the dynamics of soil properties for abandoned land compared to reference soil. Full article

Nowadays, the systems for visual information processing are significantly extending their application field. Moreover, an unsolved problem for such systems is that the classification procedure has often-conflicting requirements for performance and classification reliability. Therefore, the goal of the article is to develop the wavelet method for classifying the systems for visual information processing by evaluating the performance and informativeness of the adopted classification solutions. This method of classification uses the Haar wavelet functions with training and calculates the ranges of changes in the coefficients of the separating surfaces. The authors proposed to select the ranges of changes in these coefficients by employing the Shannon entropy formula for measuring the information content. A case study proved that such a method will significantly increase the speed of detecting the intervals of coefficient values. In addition, this enables us to justify the choice of the width of the ranges for the change of coefficients, solving the contradiction between the performance and reliability of the classifier. Full article

The measurement of uncertainty across the lifetimes of engineering systems has drawn more attention in recent years. It is a helpful metric for assessing how predictable a system’s lifetime is. In these circumstances, Renyi entropy, a Shannon entropy extension, is particularly appealing. In this paper, we develop the system signature to give an explicit formula for the Renyi entropy of the residual lifetime of a coherent system when all system components have lived to a time t. In addition, several findings are studied for the aforementioned entropy, including the bounds and order characteristics. It is possible to compare the residual lifespan predictability of two coherent systems with known signatures using the findings of this study. Full article

This paper considers the appearance of indications of useful acoustic signals in the signal/noise mixture. Various information characteristics (information entropy, Jensen–Shannon divergence, spectral information divergence and statistical complexity) are investigated in the context of solving this problem. Both time and frequency domains are studied for the calculation of information entropy. The effectiveness of statistical complexity is shown in comparison with other information metrics for different signal-to-noise ratios. Two different approaches for statistical complexity calculations are also compared. In addition, analytical formulas for complexity and disequilibrium are obtained using entropy variation in the case of signal spectral distribution. The connection between the statistical complexity criterion and the Neyman–Pearson approach for hypothesis testing is discussed. The effectiveness of the proposed approach is shown for different types of acoustic signals and noise models, including colored noises, and different signal-to-noise ratios, especially when the estimation of additional noise characteristics is impossible. Full article

Mathematical generalizations of the additive Boltzmann–Gibbs–Shannon entropy formula have been numerous since the 1960s. In this paper we seek an interpretation of the Rényi and Tsallis q-entropy formulas single parameter in terms of physical properties of a finite capacity heat-bath and fluctuations of temperature. Ideal gases of non-interacting particles are used as a demonstrating example. Full article