Open AccessThis article is
- freely available
Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities
Riken Brain Science Institute, Laboratory for Advanced Brain Signal Processing, Wako-shi, Japan
Systems Research Institute, Polish Academy of Science, Poland
Riken Brain Science Institute, Laboratory for Mathematical Neuroscience, Wako-shi, Japan
* Author to whom correspondence should be addressed.
Received: 26 April 2010; Accepted: 1 June 2010 / Published: 14 June 2010
Abstract: In this paper, we extend and overview wide families of Alpha-, Beta- and Gamma-divergences and discuss their fundamental properties. In literature usually only one single asymmetric (Alpha, Beta or Gamma) divergence is considered. We show in this paper that there exist families of such divergences with the same consistent properties. Moreover, we establish links and correspondences among these divergences by applying suitable nonlinear transformations. For example, we can generate the Beta-divergences directly from Alpha-divergences and vice versa. Furthermore, we show that a new wide class of Gamma-divergences can be generated not only from the family of Beta-divergences but also from a family of Alpha-divergences. The paper bridges these divergences and shows also their links to Tsallis and Rényi entropies. Most of these divergences have a natural information theoretic interpretation.
Keywords: Similarity measures; generalized divergences; extended Itakura–Saito like divergences; Csiszár–Morimoto and Bregman divergences; Tsallis and Rényi entropies
Citations to this Article
Cite This Article
MDPI and ACS Style
Cichocki, A.; Amari, S.-I. Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities. Entropy 2010, 12, 1532-1568.
Cichocki A, Amari S-I. Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities. Entropy. 2010; 12(6):1532-1568.
Cichocki, Andrzej; Amari, Shun-ichi. 2010. "Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities." Entropy 12, no. 6: 1532-1568.