Skip to Content

2,627 Results Found

  • Article
  • Open Access
24 Citations
6,466 Views
13 Pages

27 December 2017

In the first part of the paper we work out the consequences of the fact that Jaynes’ Maximum Entropy Principle, when translated in mathematical terms, is a constrained extremum problem for an entropy function H ( p ) expressing the uncertaint...

(This article belongs to the Special Issue Entropy and Its Applications across Disciplines)
  • Article
  • Open Access
2 Citations
6,662 Views
11 Pages

24 February 2014

There are two entropy-based methods to deal with linear inverse problems, which we shall call the ordinary method of maximum entropy (OME) and the method of maximum entropy in the mean (MEM). Not only doesMEM use OME as a stepping stone, it also allo...

(This article belongs to the Special Issue Maximum Entropy and Its Application)
  • Article
  • Open Access
22 Citations
13,262 Views
20 Pages

18 January 2010

Evidence from climate science suggests that a principle of maximum thermodynamic entropy production can be used to make predictions about some physical systems. I discuss the general form of this principle and an inherent problem with it, currently u...

(This article belongs to the Special Issue What Is Maximum Entropy Production and How Should We Apply It?)
  • Article
  • Open Access
112 Citations
18,743 Views
36 Pages

Maximum Entropy Fundamentals

  • Peter Harremoës and
  • Flemming Topsøe

30 September 2001

In its modern formulation, the Maximum Entropy Principle was promoted by E.T. Jaynes, starting in the mid-fifties. The principle dictates that one should look for a distribution, consistent with available information, which maximizes the entropy. How...

  • Proceeding Paper
  • Open Access
1 Citations
11,171 Views
8 Pages

17 November 2019

The entropy of the observable universe has been calculated as Suni ~ 10104 k and is dominated by the entropy of supermassive black holes. Irreversible processes in the universe can only happen if there is an entropy gap ΔS between the entropy of the...

(This article belongs to the Proceedings of The 5th International Electronic Conference on Entropy and Its Applications)
  • Article
  • Open Access
1 Citations
2,298 Views
16 Pages

11 August 2023

A new variance formula is developed using the generalized inverse of an increasing function. Based on the variance formula, a new entropy formula for any uncertain variable is provided. Most of the entropy formulas in the literature are special cases...

(This article belongs to the Section Entropy Reviews)
  • Article
  • Open Access
24 Citations
10,218 Views
19 Pages

26 September 2011

We discuss a one-parameter family of generalized cross entropy between two distributions with the power index, called the projective power entropy. The cross entropy is essentially reduced to the Tsallis entropy if two distributions are taken to be e...

(This article belongs to the Special Issue Tsallis Entropy)
  • Article
  • Open Access
34 Citations
8,967 Views
37 Pages

12 September 2017

The determination of the probability distribution function (PDF) of uncertain input and model parameters in engineering application codes is an issue of importance for uncertainty quantification methods. One of the approaches that can be used for the...

(This article belongs to the Special Issue Maximum Entropy and Bayesian Methods)
  • Review
  • Open Access
8 Citations
8,387 Views
15 Pages

Maximum Entropy in Drug Discovery

  • Chih-Yuan Tseng and
  • Jack Tuszynski

7 July 2014

Drug discovery applies multidisciplinary approaches either experimentally, computationally or both ways to identify lead compounds to treat various diseases. While conventional approaches have yielded many US Food and Drug Administration (FDA)-appr...

(This article belongs to the Special Issue Maximum Entropy and Its Application)
  • Feature Paper
  • Article
  • Open Access
9 Citations
6,901 Views
16 Pages

On Maximum Entropy and Inference

  • Luigi Gresele and
  • Matteo Marsili

28 November 2017

Maximum entropy is a powerful concept that entails a sharp separation between relevant and irrelevant variables. It is typically invoked in inference, once an assumption is made on what the relevant variables are, in order to estimate a model from da...

(This article belongs to the Special Issue Entropy and Its Applications across Disciplines)
  • Feature Paper
  • Article
  • Open Access
12 Citations
3,905 Views
20 Pages

Maximum Geometric Quantum Entropy

  • Fabio Anza and
  • James P. Crutchfield

1 March 2024

Any given density matrix can be represented as an infinite number of ensembles of pure states. This leads to the natural question of how to uniquely select one out of the many, apparently equally-suitable, possibilities. Following Jaynes’ infor...

(This article belongs to the Special Issue On the Role of Geometric and Entropic Arguments in Physics: From Classical Thermodynamics to Quantum Mechanics)
  • Article
  • Open Access
28 Citations
8,214 Views
64 Pages

4 September 2013

Objective Bayesian epistemology invokes three norms: the strengths of our beliefs should be probabilities; they should be calibrated to our evidence of physical probabilities; and they should otherwise equivocate sufficiently between the basic propos...

(This article belongs to the Special Issue Maximum Entropy and Bayes Theorem)
  • Article
  • Open Access
5 Citations
5,453 Views
15 Pages

A Maximum Entropy Procedure to Solve Likelihood Equations

  • Antonio Calcagnì,
  • Livio Finos,
  • Gianmarco Altoé and
  • Massimiliano Pastore

15 June 2019

In this article, we provide initial findings regarding the problem of solving likelihood equations by means of a maximum entropy (ME) approach. Unlike standard procedures that require equating the score function of the maximum likelihood problem at z...

(This article belongs to the Special Issue Information-Theoretical Methods in Data Mining)
  • Reply
  • Open Access
9 Citations
4,922 Views
5 Pages

24 April 2018

In a paper published in this journal, I addressed the following problem: under which conditions will two scientists, observing the same system and sharing the same initial information, reach the same probabilistic description upon the application of...

(This article belongs to the Special Issue Entropy and Its Applications across Disciplines)
  • Article
  • Open Access
20 Citations
7,892 Views
22 Pages

9 January 2018

The spiking activity of neuronal networks follows laws that are not time-reversal symmetric; the notion of pre-synaptic and post-synaptic neurons, stimulus correlations and noise correlations have a clear time order. Therefore, a biologically realist...

(This article belongs to the Special Issue Information Theory in Neuroscience)
  • Article
  • Open Access
5 Citations
10,578 Views
21 Pages

Estimation Bias in Maximum Entropy Models

  • Jakob H. Macke,
  • Iain Murray and
  • Peter E. Latham

2 August 2013

Maximum entropy models have become popular statistical models in neuroscience and other areas in biology and can be useful tools for obtaining estimates of mutual information in biological systems. However, maximum entropy models fit to small data se...

(This article belongs to the Special Issue Estimating Information-Theoretic Quantities from Data)
  • Article
  • Open Access
2 Citations
2,374 Views
17 Pages

Agency Contracts under Maximum-Entropy

  • Oscar Gutiérrez and
  • Vicente Salas-Fumás

26 July 2021

This article proposes the application of the maximum-entropy principle (MEP) to agency contracting (where a principal hires an agent to make decisions on their behalf) in situations where the principal and agent only have partial knowledge on the pro...

(This article belongs to the Section Multidisciplinary Applications)
  • Article
  • Open Access
17 Citations
9,246 Views
4 Pages

30 November 2009

The principle of maximum entropy production (MEP) is the subject of considerable academic study, but has yet to become remarkable for its practical applications. A tale is told of an instance in which a spin-off from consideration of an MEP-constrain...

(This article belongs to the Special Issue What Is Maximum Entropy Production and How Should We Apply It?)
  • Review
  • Open Access
50 Citations
14,185 Views
18 Pages

Maximum Entropy Approaches to Living Neural Networks

  • Fang-Chin Yeh,
  • Aonan Tang,
  • Jon P. Hobbs,
  • Pawel Hottowy and ,
  • Wladyslaw Dabrowski,
  • Alexander Sher,
  • Alan Litke and
  • John M. Beggs
  • + 3 authors+ 0 author

13 January 2010

Understanding how ensembles of neurons collectively interact will be a key step in developing a mechanistic theory of cognitive processes. Recent progress in multineuron recording and analysis techniques has generated tremendous excitement over the p...

  • Letter
  • Open Access
5,624 Views
6 Pages

On an Objective Basis for the Maximum Entropy Principle

  • David J. Miller and
  • Hossein Soleimani

19 January 2015

In this letter, we elaborate on some of the issues raised by a recent paper by Neapolitan and Jiang concerning the maximum entropy (ME) principle and alternative principles for estimating probabilities consistent with known, measured constraint infor...

(This article belongs to the Section Information Theory, Probability and Statistics)
  • Article
  • Open Access
1 Citations
2,082 Views
12 Pages

Nested Maximum Entropy Designs for Computer Experiments

  • Weiyan Mu,
  • Chengxin Liu and
  • Shifeng Xiong

18 August 2023

Presently, computer experiments with multiple levels of accuracy are widely applied in science and engineering. This paper introduces a class of nested maximum entropy designs for such computer experiments. A multi-layer DETMAX algorithm is proposed...

(This article belongs to the Special Issue Optimal Experimental Design and Statistical Modeling)
  • Article
  • Open Access
7 Citations
6,081 Views
11 Pages

4 May 2017

Asymptotic behavior of qualitative variation statistics, including entropy measures, can be modeled well by normal distributions. In this study, we test the normality of various qualitative variation measures in general. We find that almost all indic...

(This article belongs to the Special Issue Maximum Entropy and Its Application II)
  • Article
  • Open Access
14 Citations
8,153 Views
8 Pages

29 September 2010

Maximum entropy models are often used to describe supply and demand behavior in urban transportation and land use systems. However, they have been criticized for not representing behavioral rules of system agents and because their parameters seems to...

  • Feature Paper
  • Article
  • Open Access
9 Citations
6,294 Views
11 Pages

Decoding ‘Maximum Entropy’ Deconvolution

  • Long V. Le,
  • Tae Jung Kim,
  • Young Dong Kim and
  • David E. Aspnes

2 September 2022

For over five decades, the mathematical procedure termed “maximum entropy” (M-E) has been used to deconvolve structure in spectra, optical and otherwise, although quantitative measures of performance remain unknown. Here, we examine this...

(This article belongs to the Special Issue Concepts of Entropy and Their Applications III)
  • Comment
  • Open Access
4 Citations
4,770 Views
4 Pages

14 April 2018

In the maximum entropy theory of ecology (METE), the form of a function describing the distribution of abundances over species and metabolic rates over individuals in an ecosystem is inferred using the maximum entropy inference procedure. Favretti sh...

(This article belongs to the Special Issue Entropy and Its Applications across Disciplines)
  • Article
  • Open Access
1,933 Views
18 Pages

2 March 2026

Traditional mean–variance portfolio optimization proves inadequate for cryptocurrency markets, where extreme volatility, fat-tailed return distributions, and unstable correlation structures undermine the validity of variance as a comprehensive...

  • Article
  • Open Access
8 Citations
7,616 Views
21 Pages

A Maximum Entropy Estimator for the Aggregate Hierarchical Logit Model

  • Pedro Donoso,
  • Louis De Grange and
  • Felipe González

2 August 2011

A new approach for estimating the aggregate hierarchical logit model is presented. Though usually derived from random utility theory assuming correlated stochastic errors, the model can also be derived as a solution to a maximum entropy problem. Unde...

  • Article
  • Open Access
10 Citations
6,410 Views
11 Pages

21 January 2013

Maximum entropy method has been successfully used for underdetermined systems. Network design problem, with routing and topology subproblems, is an underdetermined system and a good candidate for maximum entropy method application. Wireless ad-hoc ne...

(This article belongs to the Special Issue Information Theory Applied to Communications and Networking)
  • Article
  • Open Access
3 Citations
3,112 Views
17 Pages

28 April 2021

The probability density function (pdf) valid for the Gaussian case is often applied for describing the convolutional noise pdf in the blind adaptive deconvolution problem, although it is known that it can be applied only at the latter stages of the d...

(This article belongs to the Special Issue Information Entropy Algorithms for Image, Video, and Signal Processing)
  • Article
  • Open Access
30 Citations
10,082 Views
14 Pages

26 November 2009

The method of Generalized Maximum Entropy (GME), proposed in Golan, Judge and Miller (1996), is an information-theoretic approach that is robust to multicolinearity problem. It uses an objective function that is the sum of the entropies for coefficie...

(This article belongs to the Special Issue Maximum Entropy)
  • Article
  • Open Access
3 Citations
3,082 Views
10 Pages

4 January 2019

In this paper, we consider a special nonlinear expectation problem on the special parameter space and give a necessary and sufficient condition for the existence of the solution. Meanwhile, we generalize the necessary and sufficient condition to the...

  • Article
  • Open Access
8 Citations
5,844 Views
14 Pages

Maximum Entropy Rate Reconstruction of Markov Dynamics

  • Gregor Chliamovitch,
  • Alexandre Dupuis and
  • Bastien Chopard

8 June 2015

We develop ideas proposed by Van der Straeten to extend maximum entropy principles to Markov chains. We focus in particular on the convergence of such estimates in order to explain how our approach makes possible the estimation of transition probabil...

(This article belongs to the Special Issue Information Processing in Complex Systems)
  • Article
  • Open Access
16 Citations
7,672 Views
15 Pages

A Maximum Entropy Method for a Robust Portfolio Problem

  • Yingying Xu,
  • Zhuwu Wu,
  • Long Jiang and
  • Xuefeng Song

20 June 2014

We propose a continuous maximum entropy method to investigate the robustoptimal portfolio selection problem for the market with transaction costs and dividends.This robust model aims to maximize the worst-case portfolio return in the case that allof...

(This article belongs to the Special Issue Maximum Entropy and Its Application)
  • Article
  • Open Access
15 Citations
16,809 Views
18 Pages

Maximum Entropy Learning with Deep Belief Networks

  • Payton Lin,
  • Szu-Wei Fu,
  • Syu-Siang Wang,
  • Ying-Hui Lai and
  • Yu Tsao

8 July 2016

Conventionally, the maximum likelihood (ML) criterion is applied to train a deep belief network (DBN). We present a maximum entropy (ME) learning algorithm for DBNs, designed specifically to handle limited training data. Maximizing only the entropy o...

(This article belongs to the Special Issue Information Theoretic Learning)
  • Article
  • Open Access
401 Views
28 Pages

25 September 2026

Jaynes’ maximum entropy (MaxEnt) principle selects, from all probability distributions consistent with a set of constraints, the one that maximizes the Shannon entropy. The principle is well-defined for classical, real-valued probabilities, but...

(This article belongs to the Special Issue 70 Years of Maximum Entropy—from Jaynes’ Foundations to Modern Inference and Data Driven Science)
  • Proceeding Paper
  • Open Access
1,905 Views
9 Pages

Model Selection in the World of Maximum Entropy

  • Orestis Loukas and
  • Ho-Ryun Chung

Science aims at identifying suitable models that best describe a population based on a set of features. Lacking information about the relationships among features there is no justification to a priori fix a certain model. Ideally, we want to incorpor...

(This article belongs to the Proceedings of The 41st International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering)
  • Article
  • Open Access
9 Citations
4,214 Views
33 Pages

12 January 2020

A distribution that maximizes an entropy can be found by applying two different principles. On the one hand, Jaynes (1957a,b) formulated the maximum entropy principle (MaxEnt) as the search for a distribution maximizing a given entropy under some giv...

  • Article
  • Open Access
6 Citations
7,239 Views
25 Pages

9 December 2013

We investigate the statistical properties of maximum entropy density estimation, both for the complete data case and the incomplete data case. We show that under certain assumptions, the generalization error can be bounded in terms of the complexity...

(This article belongs to the Special Issue Maximum Entropy and Bayes Theorem)
  • Article
  • Open Access
6 Citations
6,952 Views
27 Pages

3 August 2018

We consider the maximum entropy Markov chain inference approach to characterize the collective statistics of neuronal spike trains, focusing on the statistical properties of the inferred model. To find the maximum entropy Markov chain, we use the the...

(This article belongs to the Special Issue Maximum Entropy and Bayesian Methods)
  • Article
  • Open Access
37 Citations
9,655 Views
19 Pages

30 March 2016

It is well-known that the fatigue lives of materials and structures have a considerable amount of scatter and they are commonly suggested to be considered in engineering design. In order to reduce the introduction of subjective uncertainties and obta...

(This article belongs to the Section Information Theory, Probability and Statistics)
  • Feature Paper
  • Article
  • Open Access
1 Citations
1,107 Views
14 Pages

2 March 2026

We study Maximum Entropy density estimation on continuous domains under finitely many moment constraints, formulated as the minimization of the Kullback–Leibler divergence with respect to a reference measure. To model uncertainty in empirical m...

(This article belongs to the Section Information Theory, Probability and Statistics)
  • Article
  • Open Access
4 Citations
4,041 Views
24 Pages

19 January 2021

Herein we study the problem of recovering a density operator from a set of compatible marginals, motivated by limitations of physical observations. Given that the set of compatible density operators is not singular, we adopt Jaynes’ principle a...

(This article belongs to the Special Issue Quantum Computing Algorithms and Computational Complexity)
  • Article
  • Open Access
24 Citations
5,708 Views
23 Pages

Application of a Maximum Entropy Model for Mineral Prospectivity Maps

  • Binbin Li,
  • Bingli Liu,
  • Ke Guo,
  • Cheng Li and
  • Bin Wang

15 September 2019

The effective integration of geochemical data with multisource geoscience data is a necessary condition for mapping mineral prospects. In the present study, based on the maximum entropy principle, a maximum entropy model (MaxEnt model) was establishe...

(This article belongs to the Special Issue Geological Modelling, Volume II)
  • Feature Paper
  • Article
  • Open Access
12 Citations
2,043 Views
10 Pages

Maximum Entropy Criterion for Moment Indeterminacy of Probability Densities

  • Jordan M. Stoyanov,
  • Aldo Tagliani and
  • Pier Luigi Novi Inverardi

30 January 2024

We deal with absolutely continuous probability distributions with finite all-positive integer-order moments. It is well known that any such distribution is either uniquely determined by its moments (M-determinate), or it is non-unique (M-indeterminat...

(This article belongs to the Section Information Theory, Probability and Statistics)
  • Article
  • Open Access
3 Citations
7,587 Views
15 Pages

12 January 2005

We show that the Maximum Entropy principle (E.T. Jaynes, [8]) has a natural description in terms of Morse Families of a Lagrangian submanifold. This geometric approach becomes useful when dealing with the M.E.P. with nonlinear constraints. Examples a...

  • Article
  • Open Access
3 Citations
883 Views
20 Pages

A Comparison of Algorithms to Achieve the Maximum Entropy in the Theory of Evidence

  • Joaquín Abellán,
  • Aina López-Gay,
  • Maria Isabel A. Benítez and
  • Francisco Javier G. Castellano

21 February 2026

Within the framework of evidence theory, maximum entropy is regarded as a measure of total uncertainty that satisfies a comprehensive set of mathematical properties and behavioral requirements. However, its practical applicability is severely questio...

(This article belongs to the Special Issue Entropy, Statistical Evidence, and Scientific Inference: Evidence Functions in Theory and Applications, 2nd Edition)
  • Article
  • Open Access
6 Citations
5,371 Views
35 Pages

2 October 2025

In recent years, inverse reinforcement learning algorithms have garnered substantial attention and demonstrated remarkable success across various control domains, including autonomous driving, intelligent gaming, robotic manipulation, and automated i...

(This article belongs to the Section B: Mathematics)

of 53