Next Article in Journal
A Concentrated, Nonlinear Information-Theoretic Estimator for the Sample Selection Model
Previous Article in Journal
Fairness Is an Emergent Self-Organized Property of the Free Market for Labor
Article Menu

Article Versions

Export Article

Open AccessArticle
Entropy 2010, 12(6), 1532-1568; doi:10.3390/e12061532

Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities

1
Riken Brain Science Institute, Laboratory for Advanced Brain Signal Processing, Wako-shi, Japan
2
Systems Research Institute, Polish Academy of Science, Poland
3
Riken Brain Science Institute, Laboratory for Mathematical Neuroscience, Wako-shi, Japan
*
Author to whom correspondence should be addressed.
Received: 26 April 2010 / Accepted: 1 June 2010 / Published: 14 June 2010
Download PDF [523 KB, uploaded 24 February 2015]

Abstract

In this paper, we extend and overview wide families of Alpha-, Beta- and Gamma-divergences and discuss their fundamental properties. In literature usually only one single asymmetric (Alpha, Beta or Gamma) divergence is considered. We show in this paper that there exist families of such divergences with the same consistent properties. Moreover, we establish links and correspondences among these divergences by applying suitable nonlinear transformations. For example, we can generate the Beta-divergences directly from Alpha-divergences and vice versa. Furthermore, we show that a new wide class of Gamma-divergences can be generated not only from the family of Beta-divergences but also from a family of Alpha-divergences. The paper bridges these divergences and shows also their links to Tsallis and Rényi entropies. Most of these divergences have a natural information theoretic interpretation.
Keywords: Similarity measures; generalized divergences; extended Itakura–Saito like divergences; Csiszár–Morimoto and Bregman divergences; Tsallis and Rényi entropies Similarity measures; generalized divergences; extended Itakura–Saito like divergences; Csiszár–Morimoto and Bregman divergences; Tsallis and Rényi entropies
This is an open access article distributed under the Creative Commons Attribution License (CC BY 3.0).

Scifeed alert for new publications

Never miss any articles matching your research from any publisher
  • Get alerts for new papers matching your research
  • Find out the new papers from selected authors
  • Updated daily for 49'000+ journals and 6000+ publishers
  • Define your Scifeed now

SciFeed Share & Cite This Article

MDPI and ACS Style

Cichocki, A.; Amari, S.-I. Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities. Entropy 2010, 12, 1532-1568.

Show more citation formats Show less citations formats

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top