Next Article in Journal
The Rose Model of Water: Linking Theory and Simulation
Previous Article in Journal
AMGAA: Attention-Guided Multi-Target Generative Adversarial Attack for Vision Transformers
Previous Article in Special Issue
On the Application of Information Geometry to the Manifold Induced by the Parameters of the Mean Square Error of Probability Functions
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Entropy
Volume 28
Issue 6
10.3390/e28060681
entropy-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
This is an early access version, the complete PDF, HTML, and XML versions will be available soon.
Article

Reconsideration of Information-Theoretic Principles—Perspective from the Dual Probability Distribution

by
Yoshikazu Ohtaki
1,
Tomomi Nakamura
1,
Hiroshi-H. Hasegawa
1,* and
Tatsuaki Wada
2
1
Department of Mathematics and Informatics, Ibaraki University, Mito 310-8512, Japan
2
Region of Electrical and Electronic Systems Engineering, Ibaraki University, Hitachi 316-8511, Japan
*
Author to whom correspondence should be addressed.
Entropy 2026, 28(6), 681; https://doi.org/10.3390/e28060681 (registering DOI)
Submission received: 30 April 2026 / Revised: 1 June 2026 / Accepted: 4 June 2026 / Published: 12 June 2026
(This article belongs to the Special Issue SUURI of Information Geometry: Dedicated to SUURI Engineer Professor Shun’ichi Amari on the Occasion of His 90th Birthday)
Versions Notes

Abstract

We reconsider information-theoretic principles, such as the maximum entropy principle/minimum Massieu potential principle, from the perspective of the dual probability distribution. This is introduced through Sanov’s Lemma for the multinomial distribution. The dual correspondence becomes asymptotically manifest. The Massieu potential is rewritten as the Kullback–Leibler divergence between the dual probability distribution and the dual reference distribution. Similarly, the dual potential is rewritten as the cumulant generating function with respect to the dual reference distribution. This perspective gives us new insight into information-theoretic principles. As the dual probability distribution naturally arises in data sampling, we anticipate that this new perspective will play a significant role in data analysis.
Keywords: divergence; Pythagorean theorem; Legendre transformation; Massieu potential; maximum entropy principle; minimum Massieu potential principle; Sanov’s Lemma; large deviation principle; Robbins’ bound; information geometry divergence; Pythagorean theorem; Legendre transformation; Massieu potential; maximum entropy principle; minimum Massieu potential principle; Sanov’s Lemma; large deviation principle; Robbins’ bound; information geometry

Share and Cite

MDPI and ACS Style

Ohtaki, Y.; Nakamura, T.; Hasegawa, H.-H.; Wada, T. Reconsideration of Information-Theoretic Principles—Perspective from the Dual Probability Distribution. Entropy 2026, 28, 681. https://doi.org/10.3390/e28060681

AMA Style

Ohtaki Y, Nakamura T, Hasegawa H-H, Wada T. Reconsideration of Information-Theoretic Principles—Perspective from the Dual Probability Distribution. Entropy. 2026; 28(6):681. https://doi.org/10.3390/e28060681

Chicago/Turabian Style

Ohtaki, Yoshikazu, Tomomi Nakamura, Hiroshi-H. Hasegawa, and Tatsuaki Wada. 2026. "Reconsideration of Information-Theoretic Principles—Perspective from the Dual Probability Distribution" Entropy 28, no. 6: 681. https://doi.org/10.3390/e28060681

APA Style

Ohtaki, Y., Nakamura, T., Hasegawa, H.-H., & Wada, T. (2026). Reconsideration of Information-Theoretic Principles—Perspective from the Dual Probability Distribution. Entropy, 28(6), 681. https://doi.org/10.3390/e28060681

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop