# Information Geometric Duality of ϕ-Deformed Exponential Families

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Deformed Exponential Family

## 3. Deformed Divergences, Entropies, and Metrics

#### 3.1. Linear Constraints: Divergence a là Naudts

#### 3.2. Escort Constraints: Divergence a là Amari

#### 3.3. Cramér–Rao Bound of Naudts Type

## 4. The Information Geometric “Amari–Naudts” Duality

#### 4.1. Cramér–Rao Bound of the Amari Type

#### 4.2. Example: Duality of $(c,d)$-Entropy

## 5. Connection to the Deformed-Log Duality

## 6. Discussion

- (SK1) Entropy is a continuous function of the probabilities ${p}_{i}$ only and should not explicitly depend on any other parameters.
- (SK2) Entropy is maximal for the equi-distribution ${p}_{i}=1/W$.
- (SK3) Adding a state $W+1$ to a system with ${p}_{W+1}=0$ does not change the entropy of the system.

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Thurner, S.; Corominas-Murtra, B.; Hanel, R. Three faces of entropy for complex systems: Information, thermodynamics, and the maximum entropy principle. Phys. Rev. E
**2017**, 96, 032124. [Google Scholar] [CrossRef] [PubMed] - Jaynes, E.T. Information theory and statistical mechanics. Phys. Rev.
**1957**, 106, 620. [Google Scholar] [CrossRef] - Harremoës, P.; Topsøe, F. Maximum entropy fundamentals. Entropy
**2001**, 3, 191–226. [Google Scholar] [CrossRef] - Tsallis, C. Possible generalization of Boltzmann-Gibbs statistics. J. Stat. Phys.
**1988**, 52, 479–487. [Google Scholar] [CrossRef] - Kaniadakis, G. Statistical mechanics in the context of special relativity. Phys. Rev. E
**2002**, 66, 056125. [Google Scholar] [CrossRef] [PubMed] - Jizba, P.; Arimitsu, T. The world according to Rényi: Thermodynamics of multifractal systems. Ann. Phys.
**2004**, 312, 17–59. [Google Scholar] [CrossRef] - Tsallis, C.; Cirto, L.J. Black hole thermodynamical entropy. Eur. Phys. J. C
**2013**, 73, 2487. [Google Scholar] [CrossRef] [Green Version] - Thurner, S.; Hanel, R.; Klimek, P. Introduction to the Theory of Complex Systems; Oxford University Press: Oxford, UK, 2018. [Google Scholar]
- Hanel, R.; Thurner, S. A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and distribution functions. Europhys. Lett.
**2011**, 93, 20006. [Google Scholar] [CrossRef] - Hanel, R.; Thurner, S. When do generalized entropies apply? How phase space volume determines entropy. Europhys. Lett.
**2011**, 96, 50003. [Google Scholar] [CrossRef] [Green Version] - Tsallis, C.; Gell-Mann, M.; Sato, Y. Asymptotically scale-invariant occupancy of phase space makes the entropy S
_{q}extensive. Proc. Natl. Acad. Sci. USA**2005**, 102, 15377–15382. [Google Scholar] [CrossRef] - Jensen, H.J.; Pazuki, R.H.; Pruessner, G.; Tempesta, P. Statistical mechanics of exploding phase spaces: Ontic open systems. J. Phys. A
**2018**, 51, 375002. [Google Scholar] [CrossRef] - Korbel, J.; Hanel, R.; Thurner, S. Classification of complex systems by their sample-space scaling exponents. New J. Phys.
**2018**, 20, 093007. [Google Scholar] [CrossRef] - Tsallis, C.; Mendes, R.S.; Plastino, A.R. The role of constraints within generalized nonextensive statistics. Physica A
**1998**, 261, 534–554. [Google Scholar] [CrossRef] - Abe, S. Geometry of escort distributions. Phys. Rev. E
**2003**, 68, 031101. [Google Scholar] [CrossRef] [PubMed] - Ohara, A.; Matsuzoe, H.; Amari, S.I. A dually flat structure on the space of escort distributions. J. Phys. Conf. Ser.
**2010**, 201, 012012. [Google Scholar] [CrossRef] [Green Version] - Bercher, J.-F. A simple probabilistic construction yielding generalized entropies and divergences, escort distributions and q-Gaussians. Physica A
**2012**, 391, 4460–4469. [Google Scholar] [CrossRef] [Green Version] - Hanel, R.; Thurner, S.; Tsallis, C. On the robustness of q-expectation values and Renyi entropy. Europhys. Lett.
**2009**, 85, 20005. [Google Scholar] [CrossRef] - Hanel, R.; Thurner, S.; Tsallis, C. Limit distributions of scale-invariant probabilistic models of correlated random variables with the q-Gaussian as an explicit example. Eur. Phys. J. B
**2009**, 72, 263–268. [Google Scholar] [CrossRef] [Green Version] - Tsallis, C.; Souza, A.M.C. Constructing a statistical mechanics for Beck-Cohen superstatistics. Phys. Rev. E
**2003**, 67, 026106. [Google Scholar] [CrossRef] [Green Version] - Beck, C.; Cohen, E.D.G. Superstatistics. Physica A
**2003**, 322, 267–275. [Google Scholar] [CrossRef] [Green Version] - Hanel, R.; Thurner, S.; Gell-Mann, M. Generalized entropies and logarithms and their duality relations. Proc. Natl. Acad. Sci. USA
**2012**, 109, 19151–19154. [Google Scholar] [CrossRef] [Green Version] - Amari, S.I.; Cichocki, A. Information geometry of divergence functions. Bull. Pol. Acad. Sci. Tech. Sci.
**2010**, 58, 183–195. [Google Scholar] [CrossRef] [Green Version] - Amari, S.I.; Ohara, A.; Matsuzoe, H. Geometry of deformed exponential families: Invariant, dually-flat and conformal geometries. Physica A
**2012**, 391, 4308–4319. [Google Scholar] [CrossRef] - Ay, N.; Jost, J.; Le, H.V.; Schwachhöfer, L. Information Geometry; Springer: Berlin, Germany, 2017. [Google Scholar]
- Naudts, J. Deformed exponentials and logarithms in generalized thermostatistics. Physica A
**2002**, 316, 323–334. [Google Scholar] [CrossRef] [Green Version] - Naudts, J. Continuity of a class of entropies and relative entropies. Rev. Math. Phys.
**2004**, 16, 809–822. [Google Scholar] [CrossRef] - Naudts, J. Generalised Thermostatistics; Springer Science & Business Media: Berlin, Germany, 2011. [Google Scholar]
- Pistone, G.; Sempi, C. An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one. Ann. Stat.
**1995**, 23, 1543–1561. [Google Scholar] [CrossRef] - Csiszar, I. Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems. Ann. Stat.
**1991**, 19, 2032–2066. [Google Scholar] [CrossRef] - Vigelis, R.F.; Cavalcante, C.C. On ϕ-Families of probability distributions. J. Theor. Probab.
**2013**, 26, 870–884. [Google Scholar] [CrossRef] - Ohara, A. Conformal flattening for deformed information geometries on the probability simplex. Entropy
**2018**, 20, 186. [Google Scholar] [CrossRef] - Anteneodo, C.; Plastino, A.R. Maximum entropy approach to stretched exponential probability distributions. J. Phys. A
**1999**, 32, 1089. [Google Scholar] [CrossRef] - Ghikas, D.P.K.; Oikonomou, F.D. Towards an information geometric characterization/classification of complex systems. I. Use of generalized entropies. Physica A
**2018**, 496, 384–398. [Google Scholar] [CrossRef] [Green Version] - Tsallis, C. Generalization of the possible algebraic basis of q-triplets. Eur. Phys. J. Spec. Top.
**2017**, 226, 455–466. [Google Scholar] [CrossRef] - Ilić, V.M.; Stanković, M.S. Generalized Shannon–Khinchin axioms and uniqueness theorem for pseudo-additive entropies. Physica A
**2014**, 411, 138–145. [Google Scholar] [CrossRef] [Green Version] - Jizba, P.; Korbel, J. On the uniqueness theorem for pseudo-additive entropies. Entropy
**2017**, 19, 605. [Google Scholar] [CrossRef] - Uffink, J. Can the maximum entropy principle be explained as a consistency requirement? Stud. Hist. Philos. Sci. B
**1995**, 26, 223–261. [Google Scholar] [CrossRef] [Green Version] - Hanel, R.; Thurner, S.; Gell-Mann, M. Generalized entropies and the transformation group of superstatistics. Proc. Natl. Acad. Sci. USA
**2011**, 108, 6390–6394. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**Fisher metric for $p=(p,1-p)$ corresponding to various $(c,d)$-deformations ($(c,d)=(1,1)$, $(1,1/2)$, $(1/2,0)$) for (

**a**) the Naudts type, (

**b**) the Amari type, and (

**c**) the Cramér–Rao bound corresponding to the metric.

**Figure 2.**Fisher metric for $(c,d)$-deformations as a function of c and d of Naudts type (

**a**) and Amari type (

**b**). The metric is evaluated at a point $p=(1/3,2/3)$.

**Table 1.**$\varphi $-deformation of divergence, entropy, and Fisher information corresponding to the $\varphi $-exponential family under linear and escort constraints. For the ordinary logarithm, $\varphi \left(x\right)=x$, the two entropies become the Shannon entropy, and the divergence is Kullback–Leibler.

$\mathit{\varphi}$-Deformation | Linear Constraints | Escort Constraints | |
---|---|---|---|

divergence | ${D}_{\varphi}(\mathit{p}\parallel \mathit{q})$ | ${\sum}_{j}{\int}_{{q}_{j}}^{{p}_{j}}\mathrm{d}x\left({log}_{\varphi}\left(x\right)-{log}_{\varphi}\left({q}_{j}\right)\right)$ | $\frac{{\sum}_{j}\varphi \left({p}_{j}\right)({log}_{\varphi}\left({p}_{j}\right)-{log}_{\varphi}\left({q}_{j}\right))}{{\sum}_{k}\varphi \left({p}_{k}\right)}$ |

entropy | ${S}_{\varphi}\left(\mathit{p}\right)$ | $-{\sum}_{i}{\int}_{0}^{{p}_{i}}{log}_{\varphi}\left(x\right)\mathrm{d}x$ | $-{\sum}_{i}\varphi \left({p}_{i}\right){log}_{\varphi}\left({p}_{i}\right)/{\sum}_{k}\varphi \left({p}_{k}\right)$ |

metric | ${g}_{ij}^{\varphi}\left(\mathit{p}\right)$ | $\frac{1}{\varphi \left({p}_{i}\right)}{\delta}_{ij}+\frac{1}{\varphi \left({p}_{0}\right)}$ | $\frac{1}{{\sum}_{k}\varphi \left({p}_{k}\right)}\left(\frac{{\varphi}^{\prime}\left({p}_{i}\right)}{\varphi \left({p}_{i}\right)}{\delta}_{ij}+\frac{{\varphi}^{\prime}\left({p}_{0}\right)}{\varphi \left({p}_{0}\right)}\right)$ |

**Table 2.**Two important special cases of $(c,d)$-deformations and related quantities: Power laws (Tsallis) and stretched exponentials.

Tsallis q-Exponential [4] | Stretched $\mathit{\eta}$-Exponential [33] | |
---|---|---|

$\varphi \left(x\right)$ | ${x}^{q}$ | $x\eta log{\left(x\right)}^{1-1/\eta}$ |

${log}_{\varphi}\left(x\right)$ | $\frac{{x}^{1-q}-1}{1-q}$ | $log{\left(x\right)}^{1/\eta}$ |

${exp}_{\varphi}\left(x\right)$ | ${\left(1+(1-q)x\right)}^{1/(1-q)}$ | $exp\left({x}^{\eta}\right)$ |

${\chi}_{\varphi}\left(x\right)$ | $\frac{x}{q}$ | $\frac{x\eta log\left(x\right)}{(\eta -1)+\eta log\left(x\right)}$ |

${S}_{\varphi}^{N}\left(\mathit{p}\right)$ | $\frac{1}{q-1}\left({\sum}_{i}\frac{{p}_{i}^{2-q}}{2-q}-1\right)$ | ${\sum}_{i}\Gamma \left(\frac{\eta +1}{\eta},-log{p}_{i}\right)$ |

${S}_{\varphi}^{A}\left(\mathit{p}\right)$ | $\frac{1}{q-1}\left(\frac{1}{{\sum}_{i}{p}_{i}^{q}}-1\right)$ | $\frac{{\sum}_{i}{p}_{i}log{p}_{i}}{{\sum}_{i}{p}_{i}{(log{p}_{i})}^{1-1/\eta}}$ |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Korbel, J.; Hanel, R.; Thurner, S.
Information Geometric Duality of *ϕ*-Deformed Exponential Families. *Entropy* **2019**, *21*, 112.
https://doi.org/10.3390/e21020112

**AMA Style**

Korbel J, Hanel R, Thurner S.
Information Geometric Duality of *ϕ*-Deformed Exponential Families. *Entropy*. 2019; 21(2):112.
https://doi.org/10.3390/e21020112

**Chicago/Turabian Style**

Korbel, Jan, Rudolf Hanel, and Stefan Thurner.
2019. "Information Geometric Duality of *ϕ*-Deformed Exponential Families" *Entropy* 21, no. 2: 112.
https://doi.org/10.3390/e21020112