A Foliation by Deformed Probability Simplexes for Transition of α-Parameters ^†

Abstract

This study considers dualistic structures of the probability simplex from the information geometry perspective. We investigate a foliation by deformed probability simplexes for the transition of $\alpha$-parameters, not for a fixed $\alpha$-parameter. We also describe the properties of extended divergences on the foliation when different $\alpha$-parameters are defined on each of the various leaves.

1. Introduction

For instance, since the Cauchy distribution and the Student’s t-distribution are q-Gaussians, a set of q-normal distributions is considered a typical q-exponential family and has been related to nonextensive statistical mechanics [1,2]. Sets of q-normal distributions and q-exponential families have been investigated from the information geometry perspective, sometimes for nonextensive statistical mechanics and sometimes independently of it [3,4,5,6,7,8,9,10]. Deformed q-exponential families are defined using the deformed logarithm and reciprocal-deformed exponential functions. For instance, deformed q-exponential families have been used for studying escort distributions [11]. Their Hessian and conformal structures have also been investigated [12,13,14].

The current study considers a foliation by deformed probability simplexes representing sets of escort distributions, which are typical q-exponential families for the continuous transition of $\alpha$-parameters on the information geometry. Previous studies on escort distributions are for a fixed $\alpha$-parameter or among several $\alpha$-parameters. However, foliations and divergence decomposition in dually flat spaces using mixed parameterizations are crucial. Therefore, we investigate extended divergences on the foliation, setting different $\alpha$-parameters on each leaf.

First, we explain the dualistic structures, $\alpha$-divergences, and the Tsallis relative entropy on the probability simplex. Next, we describe the divergences generated by affine immersions as level surfaces on the deformed probability simplexes corresponding to sets of escort distributions. We then define an extended divergence on a foliation by deformed probability simplexes. Finally, we propose a new decomposition of an extended divergence on the foliation.

2. Dualistic Structures and Divergences on the Probability Simplex

Let ${\mathcal{S}}^n$ be the n-dimensional probability simplex, i.e.,

$${\mathcal{S}}^n=\left\{p:=\left(p_i\right)|p_i>0,\sum _{i=1}^{n+1}{p}_i=1\right\},$$

(1)

where $p_1,\dots ,p_{n+1}$ are the probabilities of $n+1$ states. Let $\{{\overline{p}}_1,\dots ,{\overline{p}}_n\}$ be an affine coordinate system on ${\mathcal{S}}^n$, where ${\overline{p}}_i\equiv p_i-p_{n+1}$ for $i=1,\dots ,n$, and

$${\{{\partial }_i\equiv \frac{\partial }{\partial p_i}-\frac{\partial }{\partial p_{n+1}}\}}_{i=1}^n$$

(2)

be a frame of a tangent vector field on ${\mathcal{S}}^n$. The Fisher metric $g=\left(g_{ij}\right)$ on ${\mathcal{S}}^n$ is defined by

$$g_{ij}\equiv g({\partial }_i,{\partial }_j)=\sum _{k=1}^{n+1}{p}_k\frac{\partial log{p}_k}{\partial p_i}\frac{\partial log{p}_k}{\partial p_j}=\frac{1}{p_i}{\delta }_{ij}+\frac{1}{p_{n+1}},i,j=1,\dots ,n,$$

(3)

where ${\delta }_{ij}$ is Kronecker’s delta. We define an $\alpha$-connection ${\nabla }^{\left(\alpha \right)}$ on ${\mathcal{S}}^n$ by

$${\nabla }_{{\partial }_i}^{\left(\alpha \right)}{\partial }_j=\sum _{k=1}^n{\mathsf{\Gamma }}_{ij}^{\left(\alpha \right)k}{\partial }_k,$$

(4)

$${\mathsf{\Gamma }}_{ij}^{\left(\alpha \right)k}=\frac{1+\alpha }{2}(-\frac{1}{p_k}{\delta }_{ij}^k+p_k{g}_{ij}),i,j,k=1,\dots ,n,$$

(5)

where ${\delta }_{ij}^k=1$ if $i=j=k$, and ${\delta }_{ij}^k=0$ if others. Then, the Levi-Civita connection ∇ of g coincides with ${\nabla }^{\left(0\right)}$. For $\alpha \in \mathbf{R}$, we have

$$Xg(Y,Z)=g({\nabla }_X^{\left(\alpha \right)}Y,Z)+g(Y,{\nabla }_X^{(-\alpha )}Z)\mathrm{for}X,Y,Z\in \mathcal{X}\left({\mathcal{S}}^n\right),$$

(6)

where $\mathcal{X}\left({\mathcal{S}}^n\right)$ is the set of all smooth tangent vector fields on ${\mathcal{S}}^n$. Then, ${\nabla }^{(-\alpha )}$ is called the dual connection of ${\nabla }^{\left(\alpha \right)}$. For each $\alpha$, ${\nabla }^{\left(\alpha \right)}$ is torsion-free and ${\nabla }^{\left(\alpha \right)}g$ is symmetric. Therefore, the triple $({\mathcal{S}}^n,{\nabla }^{\left(\alpha \right)},g)$ is a statistical manifold, and $({\mathcal{S}}^n,{\nabla }^{(-\alpha )},g)$ the dual statistical manifold of it [9,12,13].

For $\alpha \ne \pm 1$, an $\alpha$-divergence ${\mathbf{D}}^{\left(\alpha \right)}$ is defined by

$${\mathbf{D}}^{\left(\alpha \right)}(p,r)=\frac{4}{1-{\alpha }^2}\{\frac{1-\alpha }{2}\sum _{i=1}^{n+1}{p}_i+\frac{1+\alpha }{2}\sum _{i=1}^{n+1}{r}_i-\sum _{i=1}^{n+1}{\left(p_i\right)}^{\frac{1-\alpha }{2}}{\left(r_i\right)}^{\frac{1+\alpha }{2}}\},p,r\in {\mathbf{R}}_{+}^{n+1}.$$

(7)

For $q=(1-\alpha )/2$, it is known that

$${\mathbf{D}}^{\left(\alpha \right)}(p,r)=\frac{1}{q}{K}_q(p,r),p,r\in {\mathcal{S}}^n,$$

(8)

for the Tsallis relative entropy, $K_q$ defined by

$$K_q(p,r)\equiv -\sum _{i=1}^{n+1}{p}_i{ln}_q\left(\frac{r_i}{p_i}\right)=\frac{1}{1-q}\{1-\sum _{i=1}^{n+1}{\left(p_i\right)}^q{\left(r_i\right)}^{1-q}\},p,r\in {\mathcal{S}}^n,$$

(9)

where ${ln}_q$ is the q-logarithmic function defined by

$${ln}_{q}x\equiv \frac{x^{1-q}-1}{1-q},q\ne 1,x>0$$

(10)

[1,2]. The Tsallis relative entropy $K_q$ converges to the Kullback–Leibler divergence as $q\to 1$, because ${lim}_{q\to 1}{ln}_{q}x=logx$. In information geometric view, the $\alpha$-divergence ${\mathbf{D}}^{\left(\alpha \right)}$ converges to the Kullback–Leibler divergence as $\alpha \to -1$.

3. Deformed Probability Simplexes and Escort Distributions

For $n+1$ states $p_1,\dots ,p_{n+1}$ on ${\mathcal{S}}^n$ and $0<q<1$, if each probability $\mathbf{P}\left(p_i\right)$ satisfies

$$\mathbf{P}\left(p_i\right)=\frac{{\left(p_i\right)}^q}{{\sum }_{i=1}^{n+1}{\left(p_i\right)}^q},i=1,\dots ,n+1,$$

(11)

the probability distribution $\mathbf{P}$ is called the escort distribution [1,2], where ${\left(p_i\right)}^q$ is $p_i$ powered by q.

It realizes the dualistic structure of a set of escort distributions via the affine immersion into ${\mathbf{R}}_{+}^{n+1}$ [12,13]. Let $f_q$ be the affine immersion of ${\mathcal{S}}^n$ into ${\mathbf{R}}_{+}^{n+1}$ defined by

$$f_q:p=\left(p_i\right)\mapsto \theta =\left({\theta }^i\right),{\theta }^i=\frac{1}{q}{\left(p_i\right)}^q,i=1,\dots ,n+1,$$

(12)

where $\{{\theta }^1,\dots ,{\theta }^{n+1}\}$ is the canonical coordinate system on ${\mathbf{R}}^{n+1}$. Then, the escort distribution $\mathbf{P}$ is represented as follows:

$$\mathbf{P}\left(p_i\right)=\frac{{\theta }^i}{{\sum }_{i=1}^{n+1}{\theta }^i},i=1,\dots ,n+1.$$

(13)

For a function ${\psi }_q$ on ${\mathbf{R}}_{+}^{n+1}$ defined by

$${\psi }_q\left(\theta \right)=\frac{1}{1-q}\sum _{i=1}^{n+1}{\left(q{\theta }^i\right)}^{\frac{1}{q}},$$

(14)

the image $f_q\left({\mathcal{S}}^n\right)$ is a level surface of ${\psi }_q$ satisfying ${\psi }_q\left(\theta \right)=1/(1-q)$. For $0<q<1$, the Hessian matrix of the function ${\psi }_q$ is positive definite on ${\mathbf{R}}_{+}^{n+1}$. Then, ${\psi }_q$ induces the Hessian structure $({\mathbf{R}}_{+}^{n+1},D,h\equiv ({\partial }^2{\psi }_q/\partial {\theta }^i\partial {\theta }^j))$, where D is the canonical flat affine connection [9,15]. By definition

$${\tilde{\mathsf{\Gamma }}}_{ijk}=\sum _{l=1}^{n+1}{h}_{kl}{\tilde{\mathsf{\Gamma }}}_{ij}^l=\frac{{\partial }^3{\psi }_q}{\partial {\theta }^i\partial {\theta }^j\partial {\theta }^k},i,j,k=1,\dots ,n,$$

(15)

$$D_{\frac{\partial }{\partial {\theta }_i}}^{\left(\alpha \right)}\frac{\partial }{\partial {\theta }_j}=\frac{1-\alpha }{2}\sum _{k=1}^{n+1}{\tilde{\mathsf{\Gamma }}}_{ij}^k\frac{\partial }{\partial {\theta }_k},\alpha =1-2q,$$

(16)

the tetrad $({\mathbf{R}}_{+}^{n+1},D,D^{(-1)},h)$ is the dual flat structure.

The submanifold structure of $f_q\left({\mathcal{S}}^n\right)$ induced by $({\mathbf{R}}_{+}^{n+1},D,D^{(-1)},h)$ coincides with the dualistic structure induced by the equiaffine immersion $(f_q,E)$, where

$$E\equiv -d{\psi }_q{\left(\tilde{E}\right)}^{-1}\tilde{E}$$

(17)

for the gradient vector field $\tilde{E}$ of ${\psi }_q$ on ${\mathbf{R}}_{+}^{n+1}$ defined by

$$h(\tilde{X},\tilde{E})=d{\psi }_q\left(\tilde{X}\right)for\tilde{X}\in \mathcal{X}\left({\mathbf{R}}_{+}^{n+1}\right)$$

(18)

(cf. Theorem 2) [13,16,17]. In Equation (19), the induced affine connection $D^E$ is the restricted D. The affine fundamental form $h^E$ is the restricted h. The operator $S^E$ is called the shape operator. If the transversal connection form satisfies ${\tau }^E\equiv 0$, $(f_q,E)$, then it is called the equiaffine immersion [18].

$$\begin{array}{c}\hfill D_{X}Y=D_X^{E}Y+h^E(X,Y)E,D_{X}E=S^E\left(X\right)+{\tau }^E\left(X\right)EforX,Y\in \mathcal{X}\left(f_q\left({\mathcal{S}}^n\right)\right).\end{array}$$

(19)

For the restricted D and h on $f_q\left({\mathcal{S}}^n\right)$, we use the same notations. The pullback of $(f_q\left({\mathcal{S}}^n\right),D,h)$ to ${\mathcal{S}}^n$ is $(-1)$-conformally equivalent to $({\mathcal{S}}^n,{\nabla }^{\left(\alpha \right)},g)$ defined by Equations (3)–(5). In addition, $(f_q\left({\mathcal{S}}^n\right),D,h)$ has a constant curvature $\kappa =q(1-q)=(1-{\alpha }^2)/4$ [13].

4. Divergences Generated by Affine Immersions as Level Surfaces

Let $\tilde{D}$ be the canonical flat affine connection on an $(n+1)$-dimensional real affine space ${\mathbf{A}}^{n+1}$. The following theorem is known on the level surfaces of a Hessian function.

Theorem 1

([16]). Let M be a simply connected n-dimensional level surface of φ on an $(n+1)$-dimensional Hessian domain $(\mathsf{\Omega },\tilde{D},\tilde{g}=\tilde{D}d\phi )$ with a Riemannian metric $\tilde{g}$ and suppose that $n\ge 2$. If we consider $(\mathsf{\Omega },\tilde{D},\tilde{g})$ a flat statistical manifold, $(M,D,g)$ is a 1-conformally flat statistical submanifold of $(\mathsf{\Omega },\tilde{D},\tilde{g})$, where D and g denote the connection and the Riemannian metric on M induced by $\tilde{D}$ and $\tilde{g}$, respectively.

For $\alpha \in \mathbf{R}$, statistical manifolds $(N,\nabla ,h)$ and $(N,\overline{\nabla },\overline{h})$ are $\alpha$-conformally equivalent if there exists a function $\varphi$ on N such that

$$\overline{h}(X,Y)=e^{\varphi }h(X,Y),X,Y,Z\in \mathcal{X}\left(N\right),$$

$$h({\overline{\nabla }}_{X}Y,Z)=h({\nabla }_{X}Y,Z)-\frac{1+\alpha }{2}d\varphi \left(Z\right)h(X,Y)+\frac{1-\alpha }{2}\{d\varphi \left(X\right)h(Y,Z)+d\varphi \left(Y\right)h(X,Z)\}.$$

If $(N,\overline{\nabla },\overline{h})$ is 1-conformally equivalent to a flat statistical manifold $(N,\nabla ,h)$, $(N,\overline{\nabla },\overline{h})$ is called a 1-conformally flat statistical manifold. A statistical manifold $(N,\nabla ,h)$ is 1-conformally flat iff the dual statistical manifold $(N,{\nabla }^{\prime },h)$ is $(-1)$-conformally flat [19].

In terms of affine geometry, $(N,{\nabla }^{\prime },h)$ and $(N,{\overline{\nabla }}^{\prime },h)$ are $(-1)$-conformally equivalent if and only if ${\nabla }^{\prime }$ and ${\overline{\nabla }}^{\prime }$ are projectively equivalent [17,20].

The conformal immersion w for an affine immersion $(v,\xi )$ satisfies that $\langle w\left(b\right),{\xi }_b\rangle =1$, where b is a point on the surface, and $\langle a,b\rangle$ a pairing of $a\in {\mathbf{A}}_{n+1}^{*}$ and $b\in {\mathbf{A}}^{n+1}$. The next definition is given in relation to affine immersions and divergences.

Definition 1

([19]). Let $(N,\nabla ,h)$ be a 1-conformally flat statistical manifold realized by a non-degenerate affine immersion $(v,\xi )$ into ${\mathbf{A}}^{n+1}$, and w the conormal immersion for v. Then, the divergence ${\rho }_{\mathrm{c}onf}$ of $(N,\nabla ,h)$ is defined by

$${\rho }_{\mathrm{c}onf}(a,b)=\langle w\left(b\right),v\left(a\right)-v\left(b\right)\rangle fora,b\in N.$$

(20)

The ${\rho }_{\mathrm{c}onf}$ definition is independent of the choice of a realization of $(N,\nabla ,h)$.

This divergence ${\rho }_{\mathrm{c}onf}$ is referred to as Kurose geometric divergence in affine geometry and as Fenchel–Young divergence in the machine learning community [21]. The canonical divergence $\rho$ of a flat statistical manifold $(\mathsf{\Omega },\tilde{D},\tilde{g}=\tilde{D}d\phi )$ is defined by

$$\rho (a,b)=\phi \left(a\right)+{\phi }^{*}\left(\tilde{w}\left(b\right)\right)+\sum _{i=1}^{n+1}{\tilde{v}}^i\left(a\right){\tilde{v}}_i^{\prime }\left(b\right)fora,b\in \mathsf{\Omega },$$

(21)

where $\left({\tilde{v}}^i\right)$, $({\tilde{v}}_i^{\prime }=-\partial \phi /\partial {\tilde{v}}_i)$, and ${\phi }^{*}$ are the primal coordinate, the dual coordinate, and the Legendre transform of $\phi$, respectively [9]. The gradient mapping $\tilde{w}$ is defined by $\tilde{w}=-\partial \phi /\partial \tilde{v}\in {\mathbf{A}}_{n+1}^{*}$. For a 1-conformally flat statistical submanifold $(M,D,g)$ of a Hessian domain $(\mathsf{\Omega },\tilde{D},\tilde{g})$, we denote by ${\rho }_{\mathrm{s}ub}$ the restriction of the divergence $\rho$. Then, the next theorem holds.

Theorem 2

([17]). For a 1-conformally flat statistical submanifold $(M,D,g)$ of $(\mathsf{\Omega },\tilde{D},\tilde{g})$, two divergences ${\rho }_{\mathrm{c}onf}$ and ${\rho }_{\mathrm{s}ub}$ coincide.

On the level surface $(f_q\left({\mathcal{S}}^n\right),D,h)$ in Section 3, the restricted divergence from the canonical divergence of $({\mathbf{R}}_{+}^{n+1},D,h)$ coincides with the geometric divergence by Equation (20) for the affine immersion $(f_q,E)$. In addition, the pullback divergence to ${\mathcal{S}}^n$ coincides with ${\mathbf{D}}^{\left(\alpha \right)}$ and the Tsallis relative entropy $K_q$ [12].

5. Extended Divergence on a Foliation by Deformed Probability Simplexes

Previous sections described the dualistic structure, the affine immersion, and the divergence for each fixed q. This section defines a divergence on a foliation by deformed probability simplexes $(f_q\left({\mathcal{S}}^n\right),D,h)$ for all $0<q<1$, and shows the divergence decomposition property. We apply the discussion on ${\mathbf{A}}_{n+1}^{*}$ and ${\mathbf{A}}^{n+1}$ in Section 4 to the one on ${\mathbf{R}}_{n+1+}^{*}$ and ${\mathbf{R}}_{+}^{n+1}$.

Let ${\rho }_q$ be the divergence on $f_q\left({\mathcal{S}}^n\right)$ defined by the affine immersion $(f_q,E_q)$ by Equations (17) and (18).

Let $S_{fol}={\cup }_{0<q<1}{f}_q\left({\mathcal{S}}^n\right)\subset {\mathbf{R}}_{+}^{n+1}$, which corresponds to a foliation $\mathcal{F}=\left\{f_q\left({\mathcal{S}}^n\right)\right|0<q<1\}$. We define a function ${\rho }_{fol}$ on $S_{fol}\times S_{fol}$ as follows:

$${\rho }_{fol}(a,b)\equiv {\psi }_{q\left(a\right)}\left(a\right)-{\psi }_{q\left(b\right)}\left(b\right)-\sum _{i=1}^{n+1}{\eta }_i\left(b\right)({\theta }^i\left(a\right)-{\theta }^i\left(b\right))fora\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right),b\in f_{q\left(b\right)}\left({\mathcal{S}}^n\right),$$

(22)

$$0<q\left(a\right)<1,0<q\left(b\right)<1,$$

where

$${\eta }_i\left(b\right)\equiv \frac{\partial {\psi }_{q\left(b\right)}\left(b\right)}{\partial {\theta }^i}=\frac{1}{1-q\left(b\right)}{\left(q\left(b\right){\theta }^i\left(b\right)\right)}^{\frac{1-q\left(b\right)}{q\left(b\right)}},i=1,\dots ,n+1.$$

(23)

We identify the dual space ${\mathbf{R}}_{n+1+}^{*}$ with ${\mathbf{R}}_{+}^{n+1}$. The i-th component of the conformal immersion of $(f_q,E_q)$ is $-\partial {\psi }_q/\partial {\theta }^i$. Then, the dual coordinate of b, denoted by $\eta \left(b\right)$, satisfies that $\eta \left(b\right)\equiv ({\eta }_1\left(b\right),\dots ,{\eta }_{n+1}\left(b\right))\in f_{1-q\left(b\right)}\left({\mathcal{S}}^n\right)$ [12,17]. The next proposition holds.

Proposition 1.

The function ${\rho }_{fol}$ satisfies that:

(i) If $a,b\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$, ${\rho }_{fol}(a,b)={\rho }_{q\left(a\right)}(a,b)={\mathbf{D}}^{\left(\alpha \right(a\left)\right)}(f_{q\left(a\right)}^{-1}\left(a\right),f_{q\left(a\right)}^{-1}\left(b\right))$, where $\alpha \left(a\right)=1-2q\left(a\right)$.

(ii) In the case of $q\left(a\right)\ge q\left(b\right)$, ${\rho }_{fol}(a,b)\ge 0$ for $(a,b)\in S_{fol}\times S_{fol}$, and ${\rho }_{fol}(a,b)=0$ if and only if $a=b$.

Proof. 

The Legendre transform of ${\psi }_{q\left(b\right)}$ is defined by ${\psi }_{q\left(b\right)}^{*}\left(\eta \left(b\right)\right)=-{\psi }_{q\left(b\right)}\left(b\right)$ + ${\sum }_{i=1}^{n+1}{\theta }^i\left(b\right){\eta }_i\left(b\right)$. By Equation (21), (i) holds. The definition of ${\mathcal{S}}^n$ induces that

$${\psi }_{q\left(a\right)}\left(a\right)=\frac{1}{1-q\left(a\right)},{\psi }_{q\left(b\right)}\left(b\right)=\frac{1}{1-q\left(b\right)}.$$

(24)

If $1>q\left(a\right)>q\left(b\right)>0$, it holds that ${\psi }_{q\left(a\right)}\left(a\right)>{\psi }_{q\left(b\right)}\left(b\right)$. In addition, $f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$ and $f_{q\left(b\right)}\left({\mathcal{S}}^n\right)$ are convex centro-affine hypersurfaces, and $f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$ is more on the origin side than $f_{q\left(b\right)}\left({\mathcal{S}}^n\right)$. Then, $-{\sum }_{i=1}^{n+1}{\eta }_i\left(b\right)({\theta }^i\left(a\right)-{\theta }^i\left(b\right))\ge 0$. Thus, (ii) holds. □

Definition 2.

We refer to ${\rho }_{fol}$ defined by Equations (22) and (23) as an extended divergence on the foliation $S_{fol}$.

We define the extended dual divergence ${\rho }_{fol}^{*}$ of ${\rho }_{fol}$ as follows:

$${\rho }_{fol}^{*}(\eta \left(a\right),\eta \left(b\right))\equiv {\psi }_{q\left(a\right)}^{*}\left(\eta \left(a\right)\right)-{\psi }_{q\left(b\right)}^{*}\left(\eta \left(b\right)\right)-\sum _{i=1}^{n+1}{\theta }^i\left(b\right)({\eta }_i\left(a\right)-{\eta }_i\left(b\right))$$

(25)

$$\mathrm{for}a\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right),b\in f_{q\left(b\right)}\left({\mathcal{S}}^n\right),0<q\left(a\right)<1,0<q\left(b\right)<1,$$

where ${\psi }_q^{*}$ is the Legendre transform of ${\psi }_q$ for $0<q<1$. Then, the following holds.

Proposition 2.

The functions ${\rho }_{fol}$ and ${\rho }_{fol}^{*}$ satisfy the following:

$${\rho }_{fol}^{*}(\eta \left(b\right),\eta \left(a\right))={\rho }_{fol}(a,b)fora\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right),b\in f_{q\left(b\right)}\left({\mathcal{S}}^n\right).$$

(26)

Proof. 

By the definition of the Legendre transform, we have

$$\begin{array}{ccc}\hfill {\rho }_{fol}^{*}(\eta \left(b\right),\eta \left(a\right))& =& {\psi }_{q\left(b\right)}^{*}\left(\eta \left(b\right)\right)-{\psi }_{q\left(a\right)}^{*}\left(\eta \left(a\right)\right)-\sum _{i=1}^{n+1}{\theta }^i\left(a\right)({\eta }_i\left(b\right)-{\eta }_i\left(a\right))\hfill \\ & =& -{\psi }_{q\left(b\right)}\left(b\right)+\sum _{i=1}^{n+1}{\theta }^i\left(b\right){\eta }_i\left(b\right)+{\psi }_{q\left(a\right)}\left(a\right)-\sum _{i=1}^{n+1}{\theta }^i\left(a\right){\eta }_i\left(a\right)-\sum _{i=1}^{n+1}{\theta }^i\left(a\right)({\eta }_i\left(b\right)-{\eta }_i\left(a\right))\hfill \\ & =& {\psi }_{q\left(a\right)}\left(a\right)-{\psi }_{q\left(b\right)}\left(b\right)-\sum _{i=1}^{n+1}{\eta }_i\left(b\right)({\theta }^i\left(a\right)-{\theta }^i\left(b\right))={\rho }_{fol}(a,b).\hfill \end{array}$$

□

The extended divergence using Equations (22) and (23) is related to the duo Bregman (pseudo-)divergence where the parameters also define the convex functions [22]. Their relationship will be studied in future works.

6. Decomposition of an Extended Divergence

At the beginning of this section, to make a decomposition theorem of an extended divergence, we give flows which are orthogonal to each leaf of $\mathcal{F}$.

For the foliation $\mathcal{F}=\left\{f_q\left({\mathcal{S}}^n\right)\right|0<q<1\}$, we consider the flow on $S_{fol}$ defined using the following equation:

$$\frac{d{\eta }_i}{dt}={\eta }_i,i=1,\dots ,n+1,$$

(27)

where a function ${\eta }_i$ on $S_{fol}$ takes the i-th component of the dual coordinate on $f_q\left({\mathcal{S}}^n\right)$ as Equation (23) for each $0<q<1$. An integral curve of Equation (27) is orthogonal to $f_q\left({\mathcal{S}}^n\right)$ for each q with respect to the pairing $\langle ,\rangle$. The set of the integral curves becomes the orthogonal foliation of $\mathcal{F}$. We denote it by ${\mathcal{F}}^{\perp }$.

Translating into the primal coordinate system, we have the next equation on $S_{fol}$:

$$\frac{d{\theta }^i}{dt}={\tilde{E}}^i,{\tilde{E}}^i\equiv {\tilde{E}}_q^i=\sum _{j=1}^{n+1}{h}_q^{ij}\frac{\partial {\psi }_q}{\partial {\theta }^j}\mathrm{if}\left({\theta }^i\right)\in f_q\left({\mathcal{S}}^n\right),i=1,\dots ,n+1.$$

(28)

The right-hand side of Equation (28) is calculated using Equations (17) and (18) for ${\psi }_q$. A leaf of ${\mathcal{F}}^{\perp }$ is an integral curve of the vector field $\tilde{E}$ that takes the value ${\tilde{E}}_q$ on $f_q\left({\mathcal{S}}^n\right)$ for each q.

The following theorem is about the decomposition of the extended divergence.

Theorem 3.

Let ${\mathcal{S}}^n$ be the probability simplex, and $(f_q\left(S^n\right),D,h_q=Dd{\psi }_q)$ the 1-conformally flat statistical manifold generated by the affine immersion $(f_q,E_q)$, where $f_q$ is defined as

$$f_q:p=\left(p_i\right)\mapsto \theta =\left({\theta }^i\right),{\theta }^i=\frac{1}{q}{\left(p_i\right)}^q,i=1,\dots ,n+1,$$

(29)

${\psi }_q\left(\theta \right)\equiv 1/(1-q){\sum }_{i=1}^{n+1}{\left(q{\theta }^i\right)}^{1/q}$, $E_q\equiv -d\psi {\left({\tilde{E}}_q\right)}^{-1}{\tilde{E}}_q$, and ${\tilde{E}}_q^i\equiv {\sum }_{j=1}^{n+1}{h}_q^{ij}\partial {\psi }_q/\partial {\theta }^j$. Let $a,b\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$, $0<q\left(a\right)<1$, and $c\in S_{fol}\equiv {\cup }_{0<q<1}{f}_q\left({\mathcal{S}}^n\right)$. If there exists an orthogonal leaf $L^{\perp }\in {\mathcal{F}}^{\perp }$, which includes b and c, we have

$${\rho }_{fol}(a,c)=\mu {\rho }_{fol}(a,b)+{\rho }_{fol}(b,c),\eta \left(c\right)=\mu \eta \left(b\right),\mu >0,$$

(30)

where $\eta (·)$ is the dual coordinate of $f_q\left({\mathcal{S}}^n\right)$ for each q.

Proof. 

From $a,b\in f_{q\left(a\right)}\left({\mathcal{S}}^n\right)$, it holds that ${\psi }_{q\left(a\right)}\left(a\right)={\psi }_{q\left(b\right)}\left(b\right)$, where $q\left(b\right)=q\left(a\right)$. By the definition in Equations (22) and (23), we have

$$\begin{array}{ccc}\hfill {\rho }_{fol}(a,c)& =& {\psi }_{q\left(a\right)}\left(a\right)-{\psi }_{q\left(c\right)}\left(c\right)-\sum _{i=1}^{n+1}{\eta }_i\left(c\right)({\theta }^i\left(a\right)-{\theta }^i\left(c\right))\hfill \\ & =& {\psi }_{q\left(b\right)}\left(b\right)-{\psi }_{q\left(c\right)}\left(c\right)-\sum _{i=1}^{n+1}\{{\eta }_i\left(c\right)({\theta }^i\left(a\right)-{\theta }^i\left(b\right))+{\eta }_i\left(c\right)({\theta }^i\left(b\right)-{\theta }^i\left(c\right))\}\hfill \\ & =& -\mu \sum _{i=1}^{n+1}{\eta }_i\left(b\right)({\theta }^i\left(a\right)-{\theta }^i\left(b\right))+\{{\psi }_{q\left(b\right)}\left(b\right)-{\psi }_{q\left(c\right)}\left(c\right)-\sum _{i=1}^{n+1}{\eta }_i\left(c\right)({\theta }^i\left(b\right)-{\theta }^i\left(c\right))\}\hfill \\ & =& \mu {\rho }_{fol}(a,b)+{\rho }_{fol}(b,c).\hfill \end{array}$$

□

A decomposition similar to Equation (30) on a foliation of Hessian level surfaces is also available [17]. Theorem 3 generalizes the previous decomposition.

Finally, we describe the gradient flow on a leaf $f_q\left(S^n\right)$ using the extended divergence.

Theorem 4.

For a submanifold $(f_q\left(S^n\right),D,h_q)$ of $S_{fol}$, we denote by $({\theta }^1,\dots ,{\theta }^n)$ an affine coordinate system on $f_q\left(S^n\right)$ such that $Dd{\theta }^i=0$, $i=1,\dots ,n$, and set $h_{qij}=h_q(\partial /\partial {\theta }^i,\partial /\partial {\theta }^j)$, $\left(h_q^{ij}\right)={\left(h_{qij}\right)}^{-1}$. The gradient flow on $f_q\left(S^n\right)$ defined by

$$\frac{d{\theta }^i}{dt}=-\sum _{j=1}^n{h}_q^{ij}\frac{\partial }{\partial {\theta }^j}{\rho }_{fol}(a_{\theta },c),a_{\theta }\in f_q\left(S^n\right),c\in L^{\perp },i=1,\dots ,n$$

(31)

converges to the unique point $b\in L^{\perp }\cap f_q\left(S^n\right)$, where $a_{\theta }$ is a variable point parametrized by $\left({\theta }^i\right)$.

Proof. 

By Theorem 3, there exists $\mu >0$ such that $\eta \left(c\right)=\mu \eta \left(b\right)$ and ${\rho }_{fol}(a_{\theta },c)=\mu {\rho }_{fol}(a_{\theta },b)+{\rho }_{fol}(b,c)$ for any $a_{\theta }\in f_q\left(S^n\right)$. Equation (31) is described by the dual coordinate system $({\eta }_1,\dots ,{\eta }_n)$ on $f_q\left(S^n\right)$ as follows:

$$\frac{d{\eta }_i}{dt}=-\mu \frac{\partial }{\partial {\theta }^j}{\rho }_{fol}(a_{\theta },b),i=1,\dots ,n.$$

(32)

On $f_q\left(S^n\right)$, from Proposition 1 (i), ${\rho }_{fol}$ coincides with the geometric divergence ${\rho }_q$, generated by the affine immersion $(f_q,E_q)$. The geometric divergence generates the dual coordinate ${\eta }_i$ such that $D^{*}d{\eta }_i=0$, $i=1,\dots ,n$, to be derived by ${\theta }^i$ [19]. Then, it holds that

$$\frac{d{\eta }_i}{dt}=-\mu ({\eta }_i\left(a_{\theta }\right)-{\eta }_i\left(b\right)),i=1,\dots ,n.$$

(33)

and that

$${\eta }_i={\eta }_i\left(b\right)+({\eta }_i\left(a{|}_{t=0}\right)-{\eta }_i\left(b\right))e^{-\mu },i=1,\dots ,n,$$

(34)

where ${a|}_{t=0}$ is an initial point of Equation (31). Then, the gradient flow of Equation (31) converges to $b\in L^{\perp }\cap f_q\left(S^n\right)$ following a geodesic for the dual coordinate system. □

The gradient flow similar to Equation (31) has been provided on a flat statistical submanifold [23]. The similar one on a Hessian level surface, i.e., a 1-conformally statistical submanifold, has been given in [17].

7. Conclusions

This study considers a foliation of probability simplexes, which are typical q-exponential families, for the continuous transition of $\alpha$-parameters on information geometry. We still need to provide details on the extended divergence and natural definition of the foliation of q-exponentials.