Next Article in Journal
Simulation of Entropy Generation under Stall Conditions in a Centrifugal Fan
Next Article in Special Issue
A Maximum Entropy Fixed-Point Route Choice Model for Route Correlation
Previous Article in Journal
Hybrid Quantum-Classical Protocol for Storage and Retrieval of Discrete-Valued Information
Previous Article in Special Issue
A Maximum Entropy Method for a Robust Portfolio Problem
Article Menu

Export Article

Open AccessArticle
Entropy 2014, 16(7), 3552-3572; doi:10.3390/e16073552

Duality of Maximum Entropy and Minimum Divergence

The Institute of Statistical Mathematics and The Graduate University of Advanced Studies, Tachikawa Tokyo 190-8562, Japan
The Institute of Statistical Mathematics, Tachikawa Tokyo 190-8562, Japan
Department of Electrical and Electronics Engineering, University of Fukui, Fukui 910-8507, Japan
Author to whom correspondence should be addressed.
Received: 28 April 2014 / Revised: 19 June 2014 / Accepted: 24 June 2014 / Published: 26 June 2014
(This article belongs to the Special Issue Maximum Entropy and Its Application)
View Full-Text   |   Download PDF [286 KB, uploaded 24 February 2015]


We discuss a special class of generalized divergence measures by the use of generator functions. Any divergence measure in the class is separated into the difference between cross and diagonal entropy. The diagonal entropy measure in the class associates with a model of maximum entropy distributions; the divergence measure leads to statistical estimation via minimization, for arbitrarily giving a statistical model. The dualistic relationship between the maximum entropy model and the minimum divergence estimation is explored in the framework of information geometry. The model of maximum entropy distributions is characterized to be totally geodesic with respect to the linear connection associated with the divergence. A natural extension for the classical theory for the maximum likelihood method under the maximum entropy model in terms of the Boltzmann-Gibbs-Shannon entropy is given. We discuss the duality in detail for Tsallis entropy as a typical example. View Full-Text
Keywords: β-divergence; dual connections; information geometry; MaxEnt; multivariate t-distribution; power exponential family; sufficiency β-divergence; dual connections; information geometry; MaxEnt; multivariate t-distribution; power exponential family; sufficiency
This is an open access article distributed under the Creative Commons Attribution License (CC BY 3.0).

Scifeed alert for new publications

Never miss any articles matching your research from any publisher
  • Get alerts for new papers matching your research
  • Find out the new papers from selected authors
  • Updated daily for 49'000+ journals and 6000+ publishers
  • Define your Scifeed now

SciFeed Share & Cite This Article

MDPI and ACS Style

Eguchi, S.; Komori, O.; Ohara, A. Duality of Maximum Entropy and Minimum Divergence. Entropy 2014, 16, 3552-3572.

Show more citation formats Show less citations formats

Related Articles

Article Metrics

Article Access Statistics



[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top