Next Article in Journal
New Hyperbolic Function Solutions for Some Nonlinear Partial Differential Equation Arising in Mathematical Physics
Next Article in Special Issue
On Monotone Embedding in Information Geometry
Previous Article in Journal
Sliding-Mode Synchronization Control for Uncertain Fractional-Order Chaotic Systems with Time Delay
Previous Article in Special Issue
Entropy, Information Theory, Information Geometry and Bayesian Inference in Data, Signal and Image Processing and Inverse Problems
Article Menu

Export Article

Open AccessArticle
Entropy 2015, 17(6), 4215-4254; doi:10.3390/e17064215

Natural Gradient Flow in the Mixture Geometry of a Discrete Exponential Family

1
Department of Electrical and Electronic Engineering, Shinshu University, Nagano, Japan
2
Inria Saclay, Île-de-France, Orsay Cedex, France
3
De Castro Statistics, Collegio Carlo Alberto, Moncalieri, Italy
This paper is an extended version of our paper published in the Proceedings of MaxEnt 2014 Conference on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Amboise, France, 21–26 September 2014.
*
Author to whom correspondence should be addressed.
Academic Editors: Frédéric Barbaresco and Ali Mohammad-Djafari
Received: 31 January 2015 / Revised: 21 May 2015 / Accepted: 2 June 2015 / Published: 18 June 2015
(This article belongs to the Special Issue Information, Entropy and Their Geometric Structures)
View Full-Text   |   Download PDF [1656 KB, uploaded 18 June 2015]   |  

Abstract

In this paper, we study Amari’s natural gradient flows of real functions defined on the densities belonging to an exponential family on a finite sample space. Our main example is the minimization of the expected value of a real function defined on the sample space. In such a case, the natural gradient flow converges to densities with reduced support that belong to the border of the exponential family. We have suggested in previous works to use the natural gradient evaluated in the mixture geometry. Here, we show that in some cases, the differential equation can be extended to a bigger domain in such a way that the densities at the border of the exponential family are actually internal points in the extended problem. The extension is based on the algebraic concept of an exponential variety. We study in full detail a toy example and obtain positive partial results in the important case of a binary sample space. View Full-Text
Keywords: information geometry; stochastic relaxation; natural gradient flow; expectation parameters; toric models information geometry; stochastic relaxation; natural gradient flow; expectation parameters; toric models
Figures

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).

Scifeed alert for new publications

Never miss any articles matching your research from any publisher
  • Get alerts for new papers matching your research
  • Find out the new papers from selected authors
  • Updated daily for 49'000+ journals and 6000+ publishers
  • Define your Scifeed now

SciFeed Share & Cite This Article

MDPI and ACS Style

Malagò, L.; Pistone, G. Natural Gradient Flow in the Mixture Geometry of a Discrete Exponential Family. Entropy 2015, 17, 4215-4254.

Show more citation formats Show less citations formats

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top