In the first part of the paper we work out the consequences of the fact that Jaynes’ Maximum Entropy Principle, when translated in mathematical terms, is a constrained extremum problem for an entropy function
There are two entropy-based methods to deal with linear inverse problems, which we shall call the ordinary method of maximum entropy (OME) and the method of maximum entropy in the mean (MEM). Not only doesMEM use OME as a stepping stone, it also allo...
Evidence from climate science suggests that a principle of maximum thermodynamic entropy production can be used to make predictions about some physical systems. I discuss the general form of this principle and an inherent problem with it, currently u...
In its modern formulation, the Maximum Entropy Principle was promoted by E.T. Jaynes, starting in the mid-fifties. The principle dictates that one should look for a distribution, consistent with available information, which maximizes the entropy. How...
The entropy of the observable universe has been calculated as Suni ~ 10104 k and is dominated by the entropy of supermassive black holes. Irreversible processes in the universe can only happen if there is an entropy gap ΔS between the entropy of the...
A new variance formula is developed using the generalized inverse of an increasing function. Based on the variance formula, a new entropy formula for any uncertain variable is provided. Most of the entropy formulas in the literature are special cases...
We discuss a one-parameter family of generalized cross entropy between two distributions with the power index, called the projective power entropy. The cross entropy is essentially reduced to the Tsallis entropy if two distributions are taken to be e...
The determination of the probability distribution function (PDF) of uncertain input and model parameters in engineering application codes is an issue of importance for uncertainty quantification methods. One of the approaches that can be used for the...
The maximum entropy principle is effective in solving decision problems, especially when it is not possible to obtain sufficient information to induce a decision. Among others, the concept of maximum entropy is successfully used to obtain the maximum...
Drug discovery applies multidisciplinary approaches either experimentally, computationally or both ways to identify lead compounds to treat various diseases. While conventional approaches have yielded many US Food and Drug Administration (FDA)-appr...
Maximum entropy is a powerful concept that entails a sharp separation between relevant and irrelevant variables. It is typically invoked in inference, once an assumption is made on what the relevant variables are, in order to estimate a model from da...
Any given density matrix can be represented as an infinite number of ensembles of pure states. This leads to the natural question of how to uniquely select one out of the many, apparently equally-suitable, possibilities. Following Jaynes’ infor...
Objective Bayesian epistemology invokes three norms: the strengths of our beliefs should be probabilities; they should be calibrated to our evidence of physical probabilities; and they should otherwise equivocate sufficiently between the basic propos...
In this article, we provide initial findings regarding the problem of solving likelihood equations by means of a maximum entropy (ME) approach. Unlike standard procedures that require equating the score function of the maximum likelihood problem at z...
In a paper published in this journal, I addressed the following problem: under which conditions will two scientists, observing the same system and sharing the same initial information, reach the same probabilistic description upon the application of...
The spiking activity of neuronal networks follows laws that are not time-reversal symmetric; the notion of pre-synaptic and post-synaptic neurons, stimulus correlations and noise correlations have a clear time order. Therefore, a biologically realist...
Maximum entropy models have become popular statistical models in neuroscience and other areas in biology and can be useful tools for obtaining estimates of mutual information in biological systems. However, maximum entropy models fit to small data se...
This article proposes the application of the maximum-entropy principle (MEP) to agency contracting (where a principal hires an agent to make decisions on their behalf) in situations where the principal and agent only have partial knowledge on the pro...
In a compact group the Haar probability measure plays the role of uniform distribution. The entropy and rate distortion theory for this uniform distribution is studied. New results and simplified proofs on convergence of convolutions on compact group...
The principle of maximum entropy production (MEP) is the subject of considerable academic study, but has yet to become remarkable for its practical applications. A tale is told of an instance in which a spin-off from consideration of an MEP-constrain...
Recently we used the maximum entropy principle for finding the price density in a multi agent insurance market. The result is similar to what the Buhlmann had obtained by maximizing the utility function. Here we begin with the price density that is d...
Understanding how ensembles of neurons collectively interact will be a key step in developing a mechanistic theory of cognitive processes. Recent progress in multineuron recording and analysis techniques has generated tremendous excitement over the p...
In this letter, we elaborate on some of the issues raised by a recent paper by Neapolitan and Jiang concerning the maximum entropy (ME) principle and alternative principles for estimating probabilities consistent with known, measured constraint infor...
Presently, computer experiments with multiple levels of accuracy are widely applied in science and engineering. This paper introduces a class of nested maximum entropy designs for such computer experiments. A multi-layer DETMAX algorithm is proposed...
Asymptotic behavior of qualitative variation statistics, including entropy measures, can be modeled well by normal distributions. In this study, we test the normality of various qualitative variation measures in general. We find that almost all indic...
Maximum entropy models are often used to describe supply and demand behavior in urban transportation and land use systems. However, they have been criticized for not representing behavioral rules of system agents and because their parameters seems to...
For over five decades, the mathematical procedure termed “maximum entropy” (M-E) has been used to deconvolve structure in spectra, optical and otherwise, although quantitative measures of performance remain unknown. Here, we examine this...
In the maximum entropy theory of ecology (METE), the form of a function describing the distribution of abundances over species and metabolic rates over individuals in an ecosystem is inferred using the maximum entropy inference procedure. Favretti sh...
Traditional mean–variance portfolio optimization proves inadequate for cryptocurrency markets, where extreme volatility, fat-tailed return distributions, and unstable correlation structures undermine the validity of variance as a comprehensive...
Maximum entropy method has been successfully used for underdetermined systems. Network design problem, with routing and topology subproblems, is an underdetermined system and a good candidate for maximum entropy method application. Wireless ad-hoc ne...
A new approach for estimating the aggregate hierarchical logit model is presented. Though usually derived from random utility theory assuming correlated stochastic errors, the model can also be derived as a solution to a maximum entropy problem. Unde...
The method of Generalized Maximum Entropy (GME), proposed in Golan, Judge and Miller (1996), is an information-theoretic approach that is robust to multicolinearity problem. It uses an objective function that is the sum of the entropies for coefficie...
The probability density function (pdf) valid for the Gaussian case is often applied for describing the convolutional noise pdf in the blind adaptive deconvolution problem, although it is known that it can be applied only at the latter stages of the d...
In this paper, we consider a special nonlinear expectation problem on the special parameter space and give a necessary and sufficient condition for the existence of the solution. Meanwhile, we generalize the necessary and sufficient condition to the...
We develop ideas proposed by Van der Straeten to extend maximum entropy principles to Markov chains. We focus in particular on the convergence of such estimates in order to explain how our approach makes possible the estimation of transition probabil...
We propose a continuous maximum entropy method to investigate the robustoptimal portfolio selection problem for the market with transaction costs and dividends.This robust model aims to maximize the worst-case portfolio return in the case that allof...
Conventionally, the maximum likelihood (ML) criterion is applied to train a deep belief network (DBN). We present a maximum entropy (ME) learning algorithm for DBNs, designed specifically to handle limited training data. Maximizing only the entropy o...
A distribution that maximizes an entropy can be found by applying two different principles. On the one hand, Jaynes (1957a,b) formulated the maximum entropy principle (MaxEnt) as the search for a distribution maximizing a given entropy under some giv...
Science aims at identifying suitable models that best describe a population based on a set of features. Lacking information about the relationships among features there is no justification to a priori fix a certain model. Ideally, we want to incorpor...
We investigate the statistical properties of maximum entropy density estimation, both for the complete data case and the incomplete data case. We show that under certain assumptions, the generalization error can be bounded in terms of the complexity...
We consider the maximum entropy Markov chain inference approach to characterize the collective statistics of neuronal spike trains, focusing on the statistical properties of the inferred model. To find the maximum entropy Markov chain, we use the the...
It is well-known that the fatigue lives of materials and structures have a considerable amount of scatter and they are commonly suggested to be considered in engineering design. In order to reduce the introduction of subjective uncertainties and obta...
We study Maximum Entropy density estimation on continuous domains under finitely many moment constraints, formulated as the minimization of the Kullback–Leibler divergence with respect to a reference measure. To model uncertainty in empirical m...
Herein we study the problem of recovering a density operator from a set of compatible marginals, motivated by limitations of physical observations. Given that the set of compatible density operators is not singular, we adopt Jaynes’ principle a...
The effective integration of geochemical data with multisource geoscience data is a necessary condition for mapping mineral prospects. In the present study, based on the maximum entropy principle, a maximum entropy model (MaxEnt model) was establishe...
We deal with absolutely continuous probability distributions with finite all-positive integer-order moments. It is well known that any such distribution is either uniquely determined by its moments (M-determinate), or it is non-unique (M-indeterminat...
We show that the Maximum Entropy principle (E.T. Jaynes, [8]) has a natural description in terms of Morse Families of a Lagrangian submanifold. This geometric approach becomes useful when dealing with the M.E.P. with nonlinear constraints. Examples a...
Within the framework of evidence theory, maximum entropy is regarded as a measure of total uncertainty that satisfies a comprehensive set of mathematical properties and behavioral requirements. However, its practical applicability is severely questio...
In recent years, inverse reinforcement learning algorithms have garnered substantial attention and demonstrated remarkable success across various control domains, including autonomous driving, intelligent gaming, robotic manipulation, and automated i...
There are two components in this work that allow for solutions of the turbulent channel flow problem: One is the Galilean-transformed Navier-Stokes equation which gives a theoretical expression for the Reynolds stress (u′v′); and the seco...
of 46