Conjugate Representations and Characterizing Escort Expectations in Information Geometry

Abstract

Based on the maximum entropy (MaxEnt) principle for a generalized entropy functional and the conjugate representations introduced by Zhang, we have reformulated the method of information geometry. For a set of conjugate representations, the associated escort expectation is naturally introduced and characterized by the generalized score function which has zero-escort expectation. Furthermore, we show that the escort expectation induces a conformal divergence.

1. Introduction

Information geometry (IG) [1,2] is a differential geometrical method based on a Riemannian metric on a statistical manifold, which is constructed from a given parameterized probability distribution function (pdf) $p_{\theta }(x)$. It provides a useful tool to study, for example, the dually flat structures of a statistical manifold. Recently, much effort has been made to study some deformed exponential families of pdfs, in which the standard exponential function $exp(x)$ and its inverse function $ln(x)$ are replaced with a deformed exponential and its inverse function, which is called a deformed logarithmic function. Among different deformed exponential functions, relatively well known ones include Tsallis’ q-deformed exponential [3] and Kaniadakis’ $\kappa$-deformed exponential functions [4]. Naudts [5] introduced the so-called $\phi$-logarithmic function in terms of a positive increasing function $\phi (x)$, and studied the generalized thermostatistics. It is shown that a q-deformed relative entropy is proportional to Amari’s $\alpha$-divergence and is related with the $\alpha$-geometry on the statistical manifold with a constant curvature [6]. In order to construct a suitable statistical manifold in IG, usually the $\alpha$-representation (rep.), or $\alpha$-immersion, of a pdf is used. It is well known that the $\alpha$-rep. works fine with an exponential family but does not necessarily work fine for a non-exponential pdf, e.g., a $\kappa$-deformed exponential family [4]. A generalization of the $\alpha$-rep. is conjugate representations, or $(\rho ,\tau )$-reps., by Zhang [7]. He also introduced the $(\rho ,\tau )$-divergences from the point of view of “representation duality”. By finding out a suitable conjugate rep. for $\kappa$-deformed exponential pdf, the IG of the $\kappa$-generalized thermostatistics [8] was studied. We further studied the IG structures among the thermodynamic potentials in the $\kappa$-thermostatistics [9], in which the escort pdfs and escort expectations play an important role. Zhang [10] further showed that his conjugate reps. also include Naudts’ $\phi$-logarithm [5] as a special case. In this way, Zhang’s conjugate reps. are very useful as a generalization of $\alpha$-reps in IG. Amari [11] showed that $(\rho ,\tau )$-divergences generate a dually flat structure in the manifold of positive measures and in that of positive-definite matrices. In this contribution, we reformulate the IG structures based on Zhang’s conjugate reps. and the maximum entropy (MaxEnt) principle for a generalized entropy functional. Our approach is different from the previous works [10,11] in that we relate the $\rho$-rep. of a pdf $p(x)$ with the Lagrange multipliers in the MaxEnt problem. This enables us to introduce a generalized score function and characterize the escort expectations.

The rest of the paper is organized as follows. The next section provides us with the preliminaries for the basics of IG for the exponential families of pdf. In Section 3, after a brief review of conjugate reps. introduced by Zhang [7], we reformulate the IG structures based on the conjugate reps. and discuss the maximum entropy (MaxEnt) principle for a generalized entropy. For a set of conjugate reps., the associated generalized score function is introduced. The escort rep. and escort expectation are then naturally induced. Section 4 relates the conformal divergence to the difference of the entropies in terms of the escort expectations. The final section is devoted to our concluding remarks. Throughout the paper, we use the abbreviations ${\partial }_i$ for $\partial /\partial {\theta }^i$, and ${\partial }^i$ for $\partial /\partial {\eta }^i$.

2. Preliminaries

Information geometry [1,2] provides us a useful tool for studying a family

$$\begin{array}{c}\hfill \mathcal{S}=\left\{p_{\theta }(x)|p_{\theta }(x)>0,\int dx{p}_{\theta }(x)=1\right\},\end{array}$$

(1)

of a probability distribution function (pdf) $p_{\theta }(x)$ characterized by a set of real parameters $\mathit{\theta }=({\theta }^1,{\theta }^2\dots ,{\theta }^M)$. $\mathcal{S}$ is called a (M-dimensional) statistical model and the pdf $p_{\theta }(x)$ of $\mathcal{S}$ can be regarded as a point in a differential manifold $\mathcal{M}$ with local coordinates $\{{\theta }^i\}$. $\mathcal{M}$ is called a statistical manifold and a Riemannian metric on $\mathcal{M}$ is provided by the Fisher information matrix $g_{ij}^{\mathrm{F}}$ [2,

$$\begin{array}{c}\hfill g_{ij}^{\mathrm{F}}(\mathit{\theta })={\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[{\partial }_i{\ell }_{\theta }(x){\partial }_j{\ell }_{\theta }(x)\right],i,j=1,2,\dots ,M,\end{array}$$

(2)

where ${\ell }_{\theta }(x)\equiv ln{p}_{\theta }(x)$. In this contribution, we assume that $g^{\mathrm{F}}$ is positive definite, and ${\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[·\right]$ stands for the linear expectation with respect to the pdf $p_{\theta }(x)$.

A manifold $\mathcal{M}$ is said to be e-flat (exponential-flat) if a set of coordinate systems $\{{\theta }^i\}$ satisfies

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[{\partial }_i{\partial }_j{\ell }_{\theta }(x){\partial }_k{\ell }_{\theta }(x)\right]=0,\forall i,j,k,\end{array}$$

(3)

identically. Any set of coordinates $\{{\theta }^i\}$ satisfying (3) is called e-affine coordinates. A well-known example of e-flat manifolds is the exponential family

$$\begin{array}{c}\hfill {\mathcal{S}}_{\exp }=\left\{p_{\theta }(x)|p_{\theta }(x)=exp\left[\sum _{m=1}^M{\theta }^m{F}_m(x)-\Psi (\mathit{\theta })\right],\int dx{p}_{\theta }(x)=1\right\},\end{array}$$

(4)

where each $F_m(x)$ is a given function of a random value x and $\Psi (\mathit{\theta })$ is the normalization factor of a pdf $p_{\theta }(x)$. From the normalization of the pdf $p_{\theta }(x)$, we see that

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[{\partial }_i{\ell }_{\theta }(x)\right]={\partial }_i{\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[1\right]=0,\end{array}$$

(5)

and, for the exponential family, we have

$$\begin{array}{c}\hfill {\partial }_i{\partial }_j{\ell }_{\theta }(x)=-{\partial }_i{\partial }_j\Psi (\mathit{\theta }),\end{array}$$

(6)

which does not depend on x. Hence, the condition (3) is satisfied and one confirms that the exponential family is e-flat. In addition, for the exponential family, we have

$$\begin{array}{c}\hfill {\partial }_i{\ell }_{\theta }(x)=f_i(x)-{\partial }_i\Psi (\mathit{\theta }).\end{array}$$

(7)

Taking the expectation of both sides and using Equation (5), we see that the m-affine coordinates ($\mathit{\eta }$-coordinates) of the exponential family are given by

$$\begin{array}{c}\hfill {\eta }_i={\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[f_i(x)\right]={\partial }_i\Psi (\mathit{\theta }).\end{array}$$

(8)

Accounting for Equations (7) and (8), from definition (2), we obtain

$$\begin{array}{c}\hfill g_{ij}^{\mathrm{F}}(\mathit{\theta })={\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[\left(f_i-{\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[f_i\right]\right)\left(f_j-{\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[f_j\right]\right)\right],i,j=1,2,\dots ,M,\end{array}$$

(9)

which is the covariance matrix for the statistical model ${\mathcal{S}}_{\exp }$.

A manifold $\mathcal{M}$ is said to be m-flat (mixture-flat) if a set of coordinate systems $\{{\eta }_i\}$ satisfies

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[\frac{1}{p_{\eta }(x)}{\partial }^i{\partial }^j{p}_{\eta }(x){\partial }^{k}ln{p}_{\eta }(x)\right]=0,\forall i,j,k,\end{array}$$

(10)

identically. In this case, the set of coordinates $\{{\eta }_i\}$ is called m-affine coordinates.

In a dually flat structure, the $\mathit{\theta }$- and $\mathit{\eta }$-coordinates are related by the Legendre transformation

$$\begin{array}{cc}\hfill \Psi (\mathit{\theta })& +{\Psi }^{*}(\mathit{\eta })-\mathit{\theta }·\mathit{\eta }=0,\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\theta }^i& ={\partial }^i{\Psi }^{*}(\mathit{\eta }),\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\eta }_i& ={\partial }_i\Psi (\mathit{\theta }),\hfill \end{array}$$

(13)

where $\Psi (\mathit{\theta })$ and ${\Psi }^{*}(\mathit{\eta })$ are Legendre–Fenchel dual to each other and are called $\theta$- and $\eta$-potential functions, respectively. The canonical divergence function [2] for a set of two pdf $p_{{\theta }_p}(x)$ and $r_{{\theta }_r}(x)$ can be defined by

$$\begin{array}{c}\hfill D(p|r)\equiv \Psi ({\mathit{\theta }}_p)-\Psi ({\mathit{\theta }}_r)-\nabla \Psi ({\mathit{\theta }}_r)·({\mathit{\theta }}_p-{\mathit{\theta }}_r),\end{array}$$

(14)

which is a Bregman divergence with the convex function $\Psi (\mathit{\theta })$.

For a dually flat manifold $\mathcal{S}$, Pythagorean relation is generalized in terms of divergence. Let $p,r,s$ be three probability distributions in $\mathcal{S}$. When the e-geodesic connecting p and r is orthogonal at r to the m-geodesic connecting r and s, the following generalized Pythagorean relation [2] holds.

$$\begin{array}{c}\hfill D(p|r)+D(r|s)=D(p|s).\end{array}$$

(15)

As is well known, maximizing the Boltzmann–Gibbs–Shannon (BGS) entropy

$$\begin{array}{c}\hfill S^{\mathrm{BGS}}\equiv -\int dxp(x)lnp(x)={\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[-lnp\right],\end{array}$$

(16)

under the M-constraints

$$\begin{array}{cc}\hfill {\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[F_m(x)\right]& =U_m,m=1,2,\cdots ,M,\hfill \end{array}$$

(17)

for a given set of $U_m$ and the normalization $\int dxp(x)=1$, leads to the optimized pdf belonging to the exponential family ${\mathcal{S}}_{\exp }$. The Lagrange multipliers are the control parameters $\{{\theta }^m\}$ for the above M-constraints. From the normalization of an exponential pdf, we readily obtain the $\theta$-potential function $\Psi (\theta )$ as

$$\begin{array}{c}\hfill \Psi (\theta )=ln\left(\int dxexp\left[\sum _{m=1}^M{\theta }^m{F}_m(x)\right]\right).\end{array}$$

(18)

We note that, in addition to Equation (2), the Fisher metric $g^{\mathrm{F}}$ can be written equivalently in other different expressions

$$\begin{array}{cc}\hfill g_{ij}^{\mathrm{F}}& =\int dx{\partial }_i{p}_{\theta }(x){\partial }_j{\ell }_{\theta }(x)\hfill \end{array}$$

(19)

$$\begin{array}{cc}& =-\int dx{p}_{\theta }(x){\partial }_i{\partial }_j{\ell }_{\theta }(x)\hfill \end{array}$$

(20)

$$\begin{array}{cc}& =\int dx\frac{1}{p_{\theta }(x)}{\partial }_i{p}_{\theta }(x){\partial }_j{p}_{\theta }(x).\hfill \end{array}$$

(21)

In particular, combining Equation (6) with (20), we readily confirm the important relation

$$\begin{array}{c}\hfill g_{ij}^{\mathrm{F}}={\partial }_i{\partial }_j\Psi (\mathit{\theta }),\end{array}$$

(22)

that is, the Fisher metric coincides with the Hessian matrix of the $\theta$-potential function $\Psi (\mathit{\theta })$. It is known that an exponential family naturally has the dualistic Hessian structures and their canonical divergences coincide with the Kullback–Leibler divergences. Furthermore, using Equation (8), the Fisher matrix can also be rewritten as

$$\begin{array}{c}\hfill g_{ij}^{\mathrm{F}}={\partial }_i{\eta }_j={\partial }_i{\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[f_j\right],\end{array}$$

(23)

which holds for the exponential family ${\mathcal{S}}_{\exp }$.

In general, the dual affine connections are induced from the metric. By applying ${\partial }_i$ to Equation (19) for $g^{\mathrm{F}}$, we see that the following relation holds

$$\begin{array}{c}\hfill {\partial }_i{g}_{jk}^{\mathrm{F}}={\Gamma }_{ij,k}^{(e)}+{\Gamma }_{ij,k}^{(m)},\end{array}$$

(24)

where the Christoffel symbol of the first kind for the e-affine connection and that for the m-affine connection are defined by

$$\begin{array}{cc}\hfill {\Gamma }_{ij,k}^{(e)}& \equiv \int dx{\partial }_k{p}_{\theta }(x){\partial }_i{\partial }_j{\ell }_{\theta }(x)={\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[{\partial }_k{\ell }_{\theta }{\partial }_i{\partial }_j{\ell }_{\theta }\right],\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill {\Gamma }_{ij,k}^{(m)}& \equiv \int dx{\partial }_i{\partial }_j{p}_{\theta }(x){\partial }_k{\ell }_{\theta }(x)={\mathbb{E}}_{{\mathrm{p}}_{\theta }}\left[\frac{1}{p_{\theta }(x)}{\partial }_i{\partial }_j{p}_{\theta }(x){\partial }_k{\ell }_{\theta }\right],\hfill \end{array}$$

(26)

respectively. In addition, we can introduce a cubic form

$$\begin{array}{c}\hfill C_{ijk}\equiv {\Gamma }_{ij,k}^{(m)}-{\Gamma }_{ij,k}^{(e)},\end{array}$$

(27)

which characterizes the difference between the affine connection ${\nabla }^{(e)}$ (or ${\nabla }^{(m)}$) and Levi–Civita connection ${\nabla }^{(0)}$ through the relations

$$\begin{array}{cc}\hfill {\Gamma }_{ij,k}^{(e)}& ={\Gamma }_{ij,k}^{(0)}-\frac{1}{2}{C}_{ijk},\hfill \end{array}$$

(28)

$$\begin{array}{cc}\hfill {\Gamma }_{ij,k}^{(m)}& ={\Gamma }_{ij,k}^{(0)}+\frac{1}{2}{C}_{ijk}.\hfill \end{array}$$

(29)

3. Conjugate Representations

Here, we briefly review Zhang’s conjugate representations [7]. For a parameterized probability density function (pdf) $p_{\theta }(x)$ with a set of real parameters $\{{\theta }^m\},m=1,2,\dots ,M$, information geometry is founded by Prof. Amari [1], based on his $\alpha$-representations (reps.) defined by

$$\begin{array}{c}\hfill {\ell }^{\alpha }\left(p_{\theta }(x)\right)=\frac{2}{1-\alpha }{p}_{\theta }^{\frac{1-\alpha }{2}}(x),{\ell }^{-\alpha }\left(p_{\theta }(x)\right)=\frac{2}{1+\alpha }{p}_{\theta }^{\frac{1+\alpha }{2}}(x),\end{array}$$

(30)

for a real parameter $\alpha \ne 1$, and on the $\alpha$-divergence

$$\begin{array}{c}\hfill D^{\alpha }(p|r)=\frac{4}{1-{\alpha }^2}\int dx\left\{\left(\frac{1-\alpha }{2}\right)p(x)+\left(\frac{1+\alpha }{2}\right)r(x)-p^{\frac{1-\alpha }{2}}(x)r^{\frac{1+\alpha }{2}}(x)\right\}.\end{array}$$

(31)

As a generalization of the $\alpha$-reps., Zhang [7] introduced the conjugate representations as follows.

Definition 1.

A ρ-representation of a real positive number ξ is a mapping $\xi \mapsto \rho (\xi )$, where $\rho (\xi )$ is a strictly monotone function. For a smooth and strictly convex function $f(\rho )$, a τ-representation: $\xi \mapsto \tau (\xi )$ is said to be conjugate to the ρ-representation with respect to $f(\rho )$ if the following relations are satisfied,

$$\begin{array}{cc}\hfill \tau (\xi )& =\frac{df(\rho )}{d\rho }{|}_{\rho ={\rho (\xi )}^{\prime }}\rho (\xi )=\frac{d{f}^{★}(\tau )}{d\tau }{|}_{\tau ={\tau (\xi )}^{\prime }}\hfill \end{array}$$

(32)

where the convex functions $f(\rho )$ and $f^{★}(\tau )$ are Legendre dual to each other:

$$\begin{array}{c}\hfill f(\rho )=\rho \tau (\rho )-f^{★}\left(\tau (\rho )\right),f^{★}(\tau )=\rho (\tau )\tau -f\left(\rho (\tau )\right).\end{array}$$

(33)

By utilizing the conjugate reps., the associated Bregman divergence can be defined as

$$\begin{array}{cc}\hfill D_{f,\rho }(p|r)& =\int dx\left[f\left(\rho (p(x))\right)-f\left(\rho (r(x))\right)-\tau (r(x))\left(\rho (p(x))-\rho (r(x))\right)\right],\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill D_{f^{★},\tau }(p|r)& =\int dx\left[f^{★}\left(\tau (p(x))\right)-f^{★}\left(\tau (r(x))\right)-\rho (r(x))\left(\tau (p(x))-\tau (r(x))\right)\right].\hfill \end{array}$$

(35)

The $\alpha$-rep. is, of course, an example of the conjugate reps., and they are related as follows.

$$\begin{array}{cccc}\hfill {\rho }_{\alpha }(p)& ={\ell }^{\alpha }(p),\hfill & \hfill {\tau }_{\alpha }(p)& ={\ell }^{-\alpha }(p),\hfill \end{array}$$

(36)

$$\begin{array}{cccc}\hfill f_{\alpha }({\rho }_{\alpha })& =\frac{2}{1+\alpha }{\left(\frac{1-\alpha }{2}{\rho }_{\alpha }\right)}^{\frac{2}{1-\alpha }},\hfill & \hfill f_{\alpha }^{★}({\tau }_{\alpha })& =\frac{2}{1-\alpha }{\left(\frac{1+\alpha }{2}{\tau }_{\alpha }\right)}^{\frac{2}{1+\alpha }}.\hfill \end{array}$$

(37)

The $\alpha$-divergence (31) is expressed as $D_{f_{\alpha },{\rho }_{\alpha }}(p|r)$.

Remark 1.

We assume that ρ- and τ-functions satisfy the suitable regularity conditions throughout this paper. It is important to describe the domains and the tangents of the relevant ρ- and τ-functions. However, this is a very difficult matter in general. For example, consider a statistical manifold which is a set of q-Gaussian distributions, and using the α-rep. (36). We see that it is an α-affine manifold with $\alpha =2q-1$ [1,7]. In this case, if the domain Ω (the total sample space) is $\mathbf{R}$, then α must satisfy $1<\alpha <5$. If $\Omega ={\mathbf{R}}^2$, then α must satisfy $1<\alpha <3$. The lower bound comes from the regularity conditions of the statistical manifold (Amari and Nagaoka [1], Chapter 2), and the upper bounds come from the integrability conditions of probability densities. In this way, the regularity conditions for a set of ρ- and τ-functions are not determined from these functions themselves only, but depend on the total sample space and the given statistical model. Some arguments have been given in our previous paper [12].

3.1. MaxEnt

For a set $(\rho ,\tau ,f(\rho ),f^{★}(\tau ))$ of conjugate reps., let us introduce a generalized entropy functional S defined by

$$\begin{array}{c}\hfill S:=-\int dx{f}^{★}\left(\tau (p(x))\right),\end{array}$$

(38)

and consider the following MaxEnt problem.

$$\begin{array}{c}\hfill \frac{\delta }{\delta p(x)}\left[S+\sum _{m=1}^M{\theta }^m\int dx\tau (p(x))F_m(x)-\gamma \int dx\tau (p(x))\right]=0,\end{array}$$

(39)

where $F_m(x)$ is a given function of x, and ${\theta }^m,m=1,2,\dots ,M$ and $\gamma$ are the Lagrange multipliers. Using the relation $d{f}^{★}(\tau )/d\tau =\rho$ defined in Equation (32), this MaxEnt problem leads to

$$\begin{array}{c}\hfill \rho (p(x)){\tau }^{\prime }(p(x))=\sum _{m=1}^M{\theta }^m{F}_m(x){\tau }^{\prime }(p(x))-\gamma (\mathbf{\theta }){\tau }^{\prime }(p(x)),\end{array}$$

(40)

where ${\tau }^{\prime }(p)$ stands for $d\tau (p)/dp$. We assume ${\tau }^{\prime }(p(x))\ne 0$ because if ${\tau }^{\prime }(p)=0$ then the $\tau$-rep. is a constant mapping, which fails to work as a rep., or immersion, of a pdf $p(x)$. We thus obtain

$$\begin{array}{c}\hfill \rho (p(x))=\sum _{m=1}^M{\theta }^m{F}_m(x)-\gamma (\mathbf{\theta }).\end{array}$$

(41)

Remark 2.

Note that unless $\tau (p)=p$, the constraints of this generalized MaxEnt problem are neither the standard expectations $\int dxp(x)F_m(x)$ nor the normalization $\int dxp(x)=1$ of the pdf $p(x)$. However, the solution of this MaxEnt problem is expressed in terms of the inverse function of $\rho (p)$ as

$$\begin{array}{c}\hfill p(x)={\rho }^{-1}\left(\sum _{m=1}^M{\theta }^m{F}_m(x)-\gamma (\mathbf{\theta })\right).\end{array}$$

(42)

Definition 2.

For any given ρ-rep., the generalized score function $s_{\rho }(x)$ is defined by

$$\begin{array}{c}\hfill s_{\rho }(x):={\partial }_i\rho (p(x)).\end{array}$$

(43)

Remark 3.

In the above MaxEnt setting, substituting Equation (42) into the generalized score function, we obtain that

$$\begin{array}{c}\hfill s_{\rho }(x)=F_i(x)-{\partial }_i\gamma .\end{array}$$

(44)

Theorem 1.

For any set of conjugate reps., and the associated generalized score function $s(x)$,

$$\begin{array}{c}\hfill \int dx\tau (p(x))s_{\rho }(x)={\partial }_i\int dxf\left(\rho (p(x))\right)\end{array}$$

(45)

holds.

Proof. 

From the definition (32) of the conjugate reps., we see $\tau =df(\rho )/d\rho$. It follows that

$$\begin{array}{c}\hfill \tau (p(x)){\partial }_i\rho (p(x))=\frac{df(\rho )}{d\rho }{\partial }_i\rho (p(x))={\partial }_{i}f\left(\rho (p(x))\right),\end{array}$$

(46)

and integrating both sides by x, we obtain the result. ☐

Definition 3 (Escort rep.).

For a given ρ-rep. which satisfies $d\rho (p)/dp\ne 0$, we can introduce a new $\tilde{\tau }$-rep., which is called the escort rep. of a pdf $p(x)$ and is defined by

$$\begin{array}{c}\hfill \tilde{\tau }\left(p\right):=\frac{c}{\frac{d\rho (p)}{dp}}=c\frac{d{\rho }^{-1}(\xi )}{d\xi }{|}_{\xi =\rho (p)},\end{array}$$

(47)

where c is an appropriate constant and ${\rho }^{-1}(\xi )$ is the inverse function of $\rho (\xi )$.

Remark 4.

For the α-reps., we have

$$\begin{array}{c}\hfill {\rho }_{\alpha }^{-1}(\xi )={\left(\frac{1-\alpha }{2}\xi \right)}^{\frac{2}{1-\alpha }}.\end{array}$$

(48)

We thus see that

$$\begin{array}{c}\hfill \tilde{\tau }(p)=\frac{2}{1+\alpha }\frac{d{\rho }_{\alpha }^{-1}(\xi )}{d\xi }{|}_{\xi ={\rho }_{\alpha }(p)}=\frac{2}{1+\alpha }{p}^{\frac{1+\alpha }{2}}={\tau }_{\alpha }(p),\end{array}$$

(49)

which states that ${\tau }_{\alpha }(p)$ is a self-escort rep. with the constant $c=2/(1+\alpha )$.

One of the merits of introducing the escort rep. $\tilde{\tau }(p)$ is the next theorem.

Theorem 2.

A $\tilde{\tau }$-rep. satisfies

$$\begin{array}{c}\hfill \int dx\tilde{\tau }\left(p(x)\right)s_{\rho }(x)=\int dx\tilde{\tau }\left(p(x)\right){\partial }_i\rho \left(p(x)\right)=0,\end{array}$$

(50)

Proof. 

Substituting the relation

$$\begin{array}{c}\hfill {\partial }_i\rho (p)=\frac{d\rho (p)}{dp}{\partial }_{i}p,\end{array}$$

(51)

into Equation (50) leads to

$$\begin{array}{c}\hfill {\partial }_i\int dxp(x)={\partial }_{i}1=0,\end{array}$$

(52)

because the pdf $p(x)$ is normalized. ☐

For this $\tilde{\tau }$-rep., we can introduce the associated convex functions $\tilde{f}(\rho )$ and ${\tilde{f}}^{★}(\tilde{\tau })$ that satisfy

$$\begin{array}{c}\hfill \frac{d\tilde{f}(\rho )}{d\rho }=\tilde{\tau },\frac{d{\tilde{f}}^{★}(\tilde{\tau })}{d\tilde{\tau }}=\rho ,\end{array}$$

(53)

respectively.

Note that combining Theorem 1 with Theorem 2 leads to

$$\begin{array}{c}\hfill \int dx\tilde{\tau }\left(p(x)\right)s_{\rho }(x)={\partial }_i\int dx\tilde{f}\left(\rho (p(x))\right)=0.\end{array}$$

(54)

We then obtain the following corollary

Corollary 1.

For the conjugate reps. ρ and $\tilde{\tau }$, the associated $\tilde{f}(\rho )$ function satisfies that

$$\begin{array}{c}\hfill \int dx\tilde{f}\left(\rho (p(x))\right)=c\int dxp(x)=c,\end{array}$$

(55)

which is the constant c defined in Equation (47) for any normalized pdf $p(x)$.

Remark 5.

For the α-rep., we see that

$$\begin{array}{c}\hfill f_{\alpha }\left({\rho }_{\alpha }(p)\right)=\frac{2}{1+\alpha }p.\end{array}$$

(56)

Definition 4 (Escort pdf and escort exprectation).

Define the escort pdf $P^{\mathrm{esc}}(x)$ with regards to a pdf $p(x)$ by utilizing the escort rep. $\tilde{\tau }(p)$ as follows.

$$\begin{array}{c}\hfill P^{\mathrm{esc}}(x):=\frac{\tilde{\tau }(p(x))}{\int dx\tilde{\tau }(p(x))},\end{array}$$

(57)

and define the escort expectation with regards to $p(x)$ of a given function $F_i(x)$ as

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[F_i\right]:=\int dx{P}^{\mathrm{esc}}(x)F_i(x).\end{array}$$

(58)

Theorem 3.

In the MaxEnt setting of Equation (39), the score function $s_{\rho }(x)$ has zero-escort expectation, i.e., ${\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[s_{\rho }(x)\right]=0$, and it follows that

$$\begin{array}{c}\hfill {\partial }_i\gamma (\mathit{\theta })={\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[F_i\right].\end{array}$$

(59)

Proof. 

From Equation (41) we have

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[s_{\rho }\right]={\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[{\partial }_i\rho \left(p(x)\right)\right]={\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[F_i\right]-{\partial }_i\gamma ,\end{array}$$

(60)

where we used ${\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[1\right]=1$. Since ${\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[s_{\rho }\right]=0$, we obtain Equation (59). ☐

Remark 6.

In our formalism, the escort expectation is characterized by the generalized score function $s_{\rho }(x)$ which is unbiased, i.e., $s_{\rho }(x)$ has zero-escort expectation.

We see that the Lagrange multiplier $\gamma (\mathit{\theta })$ is the θ-potential $\tilde{\Psi }(\mathit{\theta })$ associated with the escort expectation. The dual affine coordinate ${\mathit{\eta }}^{\mathrm{esc}}$ is

$$\begin{array}{c}\hfill {\eta }_i^{\mathrm{esc}}={\partial }_i\gamma (\mathit{\theta })={\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[F_i\right],\end{array}$$

(61)

and the associated Riemannian metric and cubic form are

$$\begin{array}{c}\hfill g_{ij}^{\mathrm{esc}}(\mathit{\theta })={\partial }_i{\partial }_j\gamma (\mathit{\theta }),\end{array}$$

(62)

$$\begin{array}{c}\hfill C_{ijk}^{\mathrm{esc}}(\mathit{\theta })={\partial }_i{\partial }_j{\partial }_k\gamma (\mathit{\theta }),\end{array}$$

(63)

respectively. Since $g^{\mathrm{esc}}$ is a Hessian metric, the statistical manifold described by the θ- and ${\mathit{\eta }}^{\mathrm{esc}}$-coordinates is dually flat.

4. Conformal Divergence

Let us consider the Bregman divergence () of the escort reps., i.e.,

$$\begin{array}{c}\hfill D_{{\tilde{f}}^{★},\tilde{\tau }}(p|r)=\int dx\left[{\tilde{f}}^{★}\left(\tilde{\tau }(p(x))\right)-{\tilde{f}}^{★}\left(\tilde{\tau }(r(x))\right)-\rho (r(x))\left(\tilde{\tau }(p(x))-\tilde{\tau }(r(x))\right)\right].\end{array}$$

(64)

The next theorem is a main result of this contribution.

Theorem 4.

The relative escort expectation of ρ-reps. is the conformal (or scaled) divergence of $D_{{\tilde{f}}^{★},\tilde{\tau }}(p|r)$ with the scaling factor $1/\int dx\tilde{\tau }(p(x))$, i.e.,

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[\rho (p(x))\right]-{\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[\rho (r(x))\right]=\frac{1}{\int dx\tilde{\tau }(p(x))}{D}_{{\tilde{f}}^{★},\tilde{\tau }}(p|r).\end{array}$$

(65)

Proof. 

From Corollary 1, we see that

$$\begin{array}{c}\hfill \int dx{\tilde{f}}^{★}\left(\tilde{\tau }(p(x))\right)=\int dx\tilde{\tau }(p(x))\rho \left(p(x)\right)-c,\end{array}$$

(66)

where c is an appropriate constant for any normalized pdf $p(x)$, and it follows that

$$\begin{array}{c}\hfill \int dx\left[{\tilde{f}}^{★}\left(\tilde{\tau }(p(x))\right)-{\tilde{f}}^{★}\left(\tilde{\tau }(r(x))\right)\right]=\int dx\left[\tilde{\tau }(p(x)\right)\rho \left(p(x)\right)-\tilde{\tau }(r(x))\rho \left(r(x)\right)].\end{array}$$

(67)

Substituting this relation into Equation (64) leads to

$$\begin{array}{c}\hfill D_{{\tilde{f}}^{★},\tilde{\tau }}(p|r)=\int dx\tilde{\tau }\left(p(x)\right)\left\{\rho \left(p(x)\right)-\rho \left(r(x)\right)\right\}.\end{array}$$

(68)

Dividing both sides by $\int dx\tilde{\tau }(p(x))$ and using the escort expectation, we obtain the result. ☐

Remark 7.

As an example of Theorem 4, let us consider the α-rep. case. Since ${\tilde{\tau }}_{\alpha }={\tau }_{\alpha }$ as shown in Remark 4, it follows that ${\tilde{f}}_{\alpha }^{★}(\tilde{\tau })=f_{\alpha }^{★}(\tau )$. The corresponding escort pdf becomes

$$\begin{array}{c}\hfill P_{\alpha }^{\mathrm{esc}}(x)=\frac{p^{\frac{1+\alpha }{2}}}{\int dx{p}^{\frac{1+\alpha }{2}}},\end{array}$$

(69)

and Equation (65) becomes

$$\begin{array}{c}\hfill \frac{2}{1-\alpha }\left({\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[p^{\frac{1-\alpha }{2}}\right]-{\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[r^{\frac{1-\alpha }{2}}\right]\right)=\frac{(1+\alpha )/2}{\int dx{p}^{\frac{1+\alpha }{2}}(x)}{D}_{{\tilde{f}}_{\alpha }^{★},{\tau }_{\alpha }}(p|r).\end{array}$$

(70)

When we set $q=(1+\alpha )/2$ this relation becomes

$$\begin{array}{c}\hfill {\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[{ln}_{q}p\right]-{\mathbb{E}}_{{\mathrm{P}}_{\mathrm{esc}}}\left[{ln}_{q}r\right]=\frac{q}{\int dx{p}^q(x)}{D}_{{\tilde{f}}_{\alpha }^{★},{\tau }_{\alpha }}(p|r),\end{array}$$

(71)

which was first shown by Matsuzoe and Ohara [13].

5. Concluding Remarks

We have discussed and reformulated the method of information geometry in terms of the conjugate reps. introduced by Zhang [7]. For an appropriate set of conjugate reps., the MaxEnt principle for a generalized entropy relates the associated Lagrange multipliers to the corresponding $\rho$ rep. (41) of the optimal pdf (42). For a generalized score function (2), the escort rep. and escort expectation are then naturally induced. The conformal divergence is related to the difference of the entropies in terms of the escort expectations, as shown in Theorem 4.

In previous work [9], we studied, for the $\kappa$-deformed exponential family, the dualistic Hessian geometries among the thermodynamic potentials in the $\kappa$-deformed thermostatistics, and found that there exist two different kinds of dual affine-coordinates: one $\eta$ is associated with the standard expectation; and the other ${\eta }^{\mathrm{esc}}$ is associated with the escort expectation. There, the double escort distributions, i.e., the escort of the escort distributions, play an important role. For the q-deformed exponential family, one of the authors (H.M.) further studied a sequence (or hierarchy) of escort distributions [14]. We think that these results are not specific to the q- or $\kappa$-deformed exponential pdf. We believe that these results [9,14] can be systematically studied by applying the reformulated method developed in this work. Further studies are needed and will be carried out in future work.