Next Article in Journal
New Riemannian Priors on the Univariate Normal Model
Previous Article in Journal
Human Brain Networks: Spiking Neuron Models, Multistability, Synchronization, Thermodynamics, Maximum Entropy Production, and Anesthetic Cascade Mechanisms
Previous Article in Special Issue
Hierarchical Geometry Verification via Maximum Entropy Saliency in Image Retrieval
Open AccessArticle

A Note of Caution on Maximizing Entropy

1
Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL 60611, USA
2
Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA 15232, USA
*
Author to whom correspondence should be addressed.
Entropy 2014, 16(7), 4004-4014; https://doi.org/10.3390/e16074004
Received: 4 May 2014 / Revised: 10 July 2014 / Accepted: 15 July 2014 / Published: 17 July 2014
(This article belongs to the Special Issue Maximum Entropy and Its Application)
The Principle of Maximum Entropy is often used to update probabilities due to evidence instead of performing Bayesian updating using Bayes’ Theorem, and its use often has efficacious results. However, in some circumstances the results seem unacceptable and unintuitive. This paper discusses some of these cases, and discusses how to identify some of the situations in which this principle should not be used. The paper starts by reviewing three approaches to probability, namely the classical approach, the limiting frequency approach, and the Bayesian approach. It then introduces maximum entropy and shows its relationship to the three approaches. Next, through examples, it shows that maximizing entropy sometimes can stand in direct opposition to Bayesian updating based on reasonable prior beliefs. The paper concludes that if we take the Bayesian approach that probability is about reasonable belief based on all available information, then we can resolve the conflict between the maximum entropy approach and the Bayesian approach that is demonstrated in the examples. View Full-Text
Keywords: maximum entropy; classical approach; limiting frequency; Bayesian; subjective probability maximum entropy; classical approach; limiting frequency; Bayesian; subjective probability
MDPI and ACS Style

Neapolitan, R.E.; Jiang, X. A Note of Caution on Maximizing Entropy. Entropy 2014, 16, 4004-4014. https://doi.org/10.3390/e16074004

AMA Style

Neapolitan RE, Jiang X. A Note of Caution on Maximizing Entropy. Entropy. 2014; 16(7):4004-4014. https://doi.org/10.3390/e16074004

Chicago/Turabian Style

Neapolitan, Richard E.; Jiang, Xia. 2014. "A Note of Caution on Maximizing Entropy" Entropy 16, no. 7: 4004-4014. https://doi.org/10.3390/e16074004

Find Other Styles

Article Access Map by Country/Region

1
Only visits after 24 November 2015 are recorded.
Search more from Scilit
 
Search
Back to TopTop