Geometry Induced by a Generalization of Rényi Divergence

Abstract

In this paper, we propose a generalization of Rényi divergence, and then we investigate its induced geometry. This generalization is given in terms of a φ-function, the same function that is used in the definition of non-parametric φ-families. The properties of φ-functions proved to be crucial in the generalization of Rényi divergence. Assuming appropriate conditions, we verify that the generalized Rényi divergence reduces, in a limiting case, to the φ-divergence. In generalized statistical manifold, the φ-divergence induces a pair of dual connections $D^{(-1)}$ and $D^{\left(1\right)}$. We show that the family of connections $D^{\left(\alpha \right)}$ induced by the generalization of Rényi divergence satisfies the relation $D^{\left(\alpha \right)}=\frac{1-\alpha }{2}{D}^{(-1)}+\frac{1+\alpha }{2}{D}^{\left(1\right)}$, with $\alpha \in [-1,1]$.

1. Introduction

Information geometry, the study of statistical models equipped with a differentiable structure, was pioneered by the work of Rao [1], and gained maturity with the work of Amari and many others [2,3,4]. It has been successfully applied in many different areas, such as statistical inference, machine learning, signal processing or optimization [4,5]. In appropriate statistical models, the differentiable structure is induced by a (statistical) divergence. The Kullback–Leibler divergence induces a Riemannian metric, called the Fisher–Rao metric, and a pair of dual connections, the exponential and mixture connections. A statistical model endowed with the Fisher–Rao metric is called a (classical) statistical manifold. Amari also considered a family of α-divergences that induce a family of α-connections.

Much research in recent years has focused on the geometry of non-standard statistical models [6,7,8]. These models are defined in terms of a deformed exponential (also called ϕ-exponential). In particular, κ-exponential models and q-exponential families are investigated in [9,10]. Non-parametric (or infinite-dimensional) φ-families were introduced by the authors in [11,12], which generalize exponential families in the non-parametric setting [13,14,15,16]. Based on the similarity between exponential and φ-families, we defined the so-called φ-divergence, with respect to which the Kullback–Leibler divergence is a particular case. Statistical models equipped with a geometric structure induced by φ-divergences, which are called generalized statistical manifolds, are investigated in [17,18]. With respect to these connections, parametric φ-families are dually flat.

The φ-divergence is intrinsically related to the $(\rho ,\tau )$-model of Zhang, which was proposed in [19,20], extended to the infinite-dimension setting in [21], and explained in more details in [22,23]. For instance, the metric induced by φ-divergence and the $(\rho ,\tau )$-generalization of the Fisher–Rao metric, for the choices $\rho ={\phi }^{-1}$ and $f={\rho }^{-1}$, differ by a conformal factor.

Among many attempts to generalize Kullback–Leibler divergence, Rényi divergence [24] is one of the most successful, having found many applications [25]. In the present paper, we propose a generalization of Rényi divergence, which we use to define a family of α-connections. This generalization is based on an interpretation of Rényi divergence as a kind of normalizing function. To generalize Rényi divergence, we considered functions satisfying some suitable conditions. To a function for which these conditions hold, we give the name of φ-function. In a limiting case, the generalized Rényi divergence reduces to the φ-divergence. In [17,18], the φ-divergence gives rise to a pair of dual connections $D^{(-1)}$ and $D^{\left(1\right)}$. We show that the connection $D^{\left(\alpha \right)}$ induced by the generalization of Rényi divergence satisfies the convex combination $D^{\left(\alpha \right)}=\frac{1-\alpha }{2}{D}^{(-1)}+\frac{1+\alpha }{2}{D}^{\left(1\right)}$.

Eguchi in [26] investigated a geometry based on a normalizing function similar to the one used in the generalization of Rényi divergence. In [26], results were derived supposing that this normalizing function exists; conditions for its existence were not given. In the present paper, the existence of the normalizing function is ensured by conditions involved in the definition of φ-functions.

The rest of the paper is organized as follows. In Section 2, φ-functions are introduced and some properties are discussed. The Rényi divergence is generalized in Section 3. We investigate in Section 4 the geometry induced by the generalization of Rényi divergence. Section 4.2 provides evidence of the role of the generalized Rényi divergence in φ-families.

2. φ-Functions

Rényi divergence is defined in terms of the exponential function (to be more precise, the logarithm). A way of generalizing Rényi divergence is to replace the exponential function by another function, which satisfies some suitable conditions. To a function for which these conditions hold, we give the name φ-function. In this section, we define and investigate some properties of φ-functions.

Let $(T,\Sigma ,\mu )$ be a measure space. Although we do not restrict our analysis to a particular measure space, the reader can think of T as the set of real numbers $\mathbb{R}$, Σ as the Borel σ-algebra on $\mathbb{R}$, and μ as the Lebesgue measure. We can also consider T to be a discrete set, a case in which μ is the counting measure.

We say that $\phi :\mathbb{R}\to (0,\infty )$ is a φ-function if the following conditions are satisfied:

(a1)

$\phi (·)$ is convex;

(a2)

${lim}_{u\to -\infty }\phi \left(u\right)=0$ and ${lim}_{u\to \infty }\phi \left(u\right)=\infty$;

(a3)

there exists a measurable function $u_0:T\to (0,\infty )$ such that

$${\displaystyle {\int }_T\phi (c+\lambda u_0)d\mu <\infty },\mathrm{for}\mathrm{all}\lambda >0,$$

(1)

for each measurable function $c:T\to \mathbb{R}$ satisfying ${\int }_T\phi \left(c\right)d\mu =1$.

Thanks to condition (a3), we can generalize Rényi divergence using φ-functions. These conditions appeared first at [12] where the authors constructed non-parametric φ-families of probability distributions. We remark that if T is finite, condition (a3) is always satisfied.

Examples of functions $\phi :\mathbb{R}\to (0,\infty )$ satisfying (a1)–(a3) abound. An example of great relevance is the exponential function $\phi \left(u\right)=exp\left(u\right)$, which satisfies conditions (a1)–(a3) with $u_0={\mathbf{1}}_T$. Another example of φ-function is the Kaniadakis’ κ-exponential [12,27,28].

Example 1.

The Kaniadakis’ κ-exponential ${exp}_{\kappa }:\mathbb{R}\to (0,\infty )$ for $\kappa \in [-1,1]$ is defined as

$${exp}_{\kappa }\left(u\right)=\left\{\begin{array}{cc}{(\kappa u+\sqrt{1+{\kappa }^2{u}^2})}^{1/\kappa },\hfill & \mathit{if}\kappa \ne 0,\hfill \\ exp\left(u\right),\hfill & \mathit{if}\kappa =0,\hfill \end{array}\right.$$

whose inverse is the so called the Kaniadakis’ κ-logarithm ${log}_k:(0,\infty )\to \mathbb{R}$, which is given by

$${log}_{\kappa }\left(u\right)=\left\{\begin{array}{cc}\frac{u^{\kappa }-u^{-\kappa }}{2\kappa },\hfill & \mathit{if}\kappa \ne 0,\hfill \\ ln\left(u\right),\hfill & \mathit{if}\kappa =0.\hfill \end{array}\right.$$

It is clear that ${exp}_{\kappa }(·)$ satisfies (a1) and (a2). Let $u_0:T\to (0,\infty )$ be any measurable function for which ${\int }_T{exp}_{\kappa }\left(u_0\right)d\mu <\infty$. We will show that $u_0$ satisfies expression (1). For any $u\in \mathbb{R}$ and $\alpha \ge 1$, we can write

$$\begin{array}{cc}\hfill {exp}_{\kappa }\left(\alpha u\right)& ={\alpha }^{1/\left|\kappa \right|}\left(\right|\kappa |u+\sqrt{1/{\alpha }^2+{\left|\kappa \right|}^2{u}^2}{)}^{1/\left|\kappa \right|}\hfill \\ & \le {\alpha }^{1/\left|\kappa \right|}\left(\right|\kappa |u+\sqrt{1+{\left|\kappa \right|}^2{u}^2}{)}^{1/\left|\kappa \right|}\hfill \\ & ={\alpha }^{1/\left|\kappa \right|}{exp}_{\kappa }\left(u\right),\hfill \end{array}$$

where we used that ${exp}_{\kappa }(·)={exp}_{-\kappa }(·)$. Then, we conclude that ${\int }_T{exp}_{\kappa }\left(\alpha u_0\right)d\mu <\infty$ for all $\alpha \ge 0$. Fix any measurable function $c:T\to \mathbb{R}$ such that ${\int }_T\phi \left(c\right)d\mu =1$. For each $\lambda >0$, we have

$$\begin{array}{cc}\hfill {\int }_T{exp}_{\kappa }(c+\lambda u_0)d\mu & \le \frac{1}{2}{\int }_T{exp}_{\kappa }\left(2c\right)d\mu +\frac{1}{2}{\int }_T{exp}_{\kappa }\left(2\lambda u_0\right)d\mu \hfill \\ & \le 2^{1/\left|\kappa \right|-1}{\int }_T{exp}_{\kappa }\left(c\right)d\mu +2^{1/\left|\kappa \right|-1}{\int }_T{exp}_{\kappa }\left(\lambda u_0\right)d\mu \hfill \\ & <\infty ,\hfill \end{array}$$

which shows that ${exp}_{\kappa }(·)$ satisfies (a3). Therefore, the Kaniadakis’ κ-exponential ${exp}_{\kappa }(·)$ is an example of φ-function.

The restriction that ${\int }_T\phi \left(c\right)d\mu =1$ can be weakened, as asserted in the next result.

Lemma 1.

Let $\tilde{c}:T\to \mathbb{R}$ be any measurable function such that ${\int }_T\phi \left(\tilde{c}\right)d\mu <\infty$. Then, ${\int }_T\phi (\tilde{c}+\lambda u_0)d\mu <\infty$ for all $\lambda >0$.

Proof. 

Notice that if ${\int }_T\phi \left(\tilde{c}\right)d\mu \ge 1$, then ${\int }_T\phi (\tilde{c}-\alpha u_0)d\mu =1$ for some $\alpha >1$. From the definition of $u_0$, it follows that ${\int }_T\phi (\tilde{c}+\lambda u_0)d\mu ={\int }_T\phi (c+(\alpha +\lambda )u_0)d\mu <\infty$, where $c=\tilde{c}-\alpha u_0$. Now assume that ${\int }_T\phi \left(\tilde{c}\right)d\mu <1$. Consider any measurable set $A\subseteq T$ with measure $0<\mu \left(A\right)<\mu \left(T\right)$. Let $u:T\to [0,\infty )$ be a measurable function supported on A satisfying $\phi (\tilde{c}+u){\mathbf{1}}_A=[\phi \left(\tilde{c}\right)+\alpha ]{\mathbf{1}}_A$, where $\alpha =(1-{\int }_T\phi \left(\tilde{c}\right)d\mu )/\mu \left(A\right)$. Defining $c=(\tilde{c}+u){\mathbf{1}}_A+\tilde{c}{\mathbf{1}}_{T\backslash A}$, we see that ${\int }_T\phi \left(c\right)d\mu =1$. By the definition of $u_0$, we can write

$${\int }_T\phi (\tilde{c}+\lambda u_0)d\mu \le {\int }_T\phi (c+\lambda u_0)d\mu <\infty ,\mathrm{for}\mathrm{any}\lambda >0,$$

which is the desired result. ☐

As a consequence of Lemma 1, condition (a3) can be replaced by the following one:

(a3’)  There exists a measurable function $u_0:T\to (0,\infty )$ such that

$${\displaystyle {\int }_T\phi (c+\lambda u_0)d\mu <\infty },\mathrm{for}\mathrm{all}\lambda >0,$$

(2)

for each measurable function $c:T\to \mathbb{R}$ for which ${\int }_T\phi \left(c\right)d\mu <\infty$.

Without the equivalence between conditions (a3) and (a3’), we could not generalize Rényi divergence in the manner we propose. In fact, φ-functions could be defined directly in terms of (a3’), without mentioning (a3). We chose to begin with (a3) because this condition appeared initially in [12].

Not all functions $\phi :\mathbb{R}\to (0,\infty )$, for which conditions (a1) and (a2) hold, satisfy condition (a3). Such a function is given below.

Example 2.

Assume that the underlying measure μ is σ-finite and non-atomic. This is the case of the Lebesgue measure. Let us consider the function

$$\phi \left(u\right)=\left\{\begin{array}{cc}{e}^{{(u+1)}^2/2},\hfill & u\ge 0,\hfill \\ e^{(u+1/2)},\hfill & u\le 0,\hfill \end{array}\right.$$

(3)

which clearly is convex, and satisfies the limits ${lim}_{u\to -\infty }\phi \left(u\right)=0$ and ${lim}_{u\to \infty }\phi \left(u\right)=\infty$. Given any measurable function $u_0:T\to (0,\infty )$, we will find a measurable function $c:T\to \mathbb{R}$ with ${\int }_T\phi \left(c\right)d\mu <\infty$, for which expression (2) is not satisfied.

For each $m\ge 1$, we define

$$v_m\left(t\right):=\left(m\frac{log\left(2\right)}{u_0\left(t\right)}-\frac{u_0\left(t\right)}{2}-1\right){\mathbf{1}}_{E_m}\left(t\right),$$

where $E_m=\{t\in T:m\frac{log\left(2\right)}{u_0\left(t\right)}-\frac{u_0\left(t\right)}{2}-1>0\}$. Because $v_m\uparrow \infty$, we can find a sub-sequence $\left\{v_{m_n}\right\}$ such that

$${\int }_{E_{m_n}}{e}^{{(v_{m_n}+u_0+1)}^2/2}d\mu \ge 2^n.$$

According to (Lemma 8.3 in [29]) , there exists a sub-sequence $w_k=v_{m_{n_k}}$ and pairwise disjoint sets $A_k\subseteq E_{m_{n_k}}$ for which

$${\int }_{A_k}{e}^{{(w_k+u_0+1)}^2/2}d\mu =1.$$

Let us define $c=\overline{c}{\mathbf{1}}_{T\backslash A}+{\sum }_{k=1}^{\infty }{w}_k{\mathbf{1}}_{A_k}$, where $A={\bigcup }_{k=1}^{\infty }{A}_k$ and $\overline{c}$ is any measurable function such that $\phi \left(\overline{c}\left(t\right)\right)>0$ for $t\in T\backslash A$ and ${\int }_{T\backslash A}\phi \left(\overline{c}\right)d\mu <\infty$. Observing that

$$e^{{(w_k\left(t\right)+u_0\left(t\right)+1)}^2/2}=2^{m_{n_k}}{e}^{{(w_k\left(t\right)+1)}^2/2},\mathit{for}t\in A_k,$$

we get

$${\int }_{A_k}{e}^{{(w_k+1)}^2/2}d\mu =\frac{1}{2^{m_{n_k}}},\mathit{for}\mathit{every}m\ge 1.$$

Then, we can write

$$\begin{array}{cc}\hfill {\int }_T\phi \left(c\right)d\mu & ={\int }_{T\backslash A}\phi \left(\overline{c}\right)d\mu +\sum _{k=1}^{\infty }{\int }_{A_k}{e}^{{(w_k+1)}^2/2}d\mu \hfill \\ & ={\int }_{T\backslash A}\phi \left(\overline{c}\right)d\mu +\sum _{k=1}^{\infty }\frac{1}{2^{m_{n_k}}}<\infty .\hfill \end{array}$$

On the other hand,

$$\begin{array}{cc}\hfill {\int }_T\phi (c+u_0)d\mu & ={\int }_{T\backslash A}\phi \left(\overline{c}\right)d\mu +\sum _{k=1}^{\infty }{\int }_{A_k}{e}^{{(u_0+w_k+1)}^2/2}d\mu \hfill \\ & ={\int }_{T\backslash A}\phi \left(\overline{c}\right)d\mu +\sum _{k=1}^{\infty }1=\infty ,\hfill \end{array}$$

which shows that (2) is not satisfied.

3. Generalization of Rényi Divergence

In this section, we provide a generalization of Rényi divergence, which is given in terms of a φ-function. This generalization also depends on a parameter $\alpha \in [-1,1]$; for $\alpha =\pm 1$, it is defined as a limit. Supposing that the underlying φ-function is continuously differentiable, we show that this limit exists and results in the φ-divergence [12]. In what follows, all probability distributions are assumed to have positive density. In other words, they belong to the collection

$${\mathcal{P}}_{\mu }=\left\{p\in L^0:{\int }_{T}pd\mu =1\mathrm{and}p>0\right\},$$

where $L^0$ is the space of all real-valued, measurable functions on T, with equality μ-a.e. (μ-almost everywhere).

The Rényi divergence of order $\alpha \in (-1,1)$ between two probability distributions p and q in ${\mathcal{P}}_{\mu }$ is defined as

$${\mathcal{D}}_{\mathrm{R}}^{\left(\alpha \right)}(p\parallel q)=\frac{4}{{\alpha }^2-1}log\left({\int }_T{p}^{\frac{1-\alpha }{2}}{q}^{\frac{1+\alpha }{2}}d\mu \right).$$

(4)

For $\alpha =\pm 1$, the Rényi divergence is defined by taking a limit:

$$\begin{array}{cc}\hfill {\mathcal{D}}_{\mathrm{R}}^{(-1)}(p\parallel q)& =\underset{\alpha \downarrow -1}{lim}{\mathcal{D}}_{\mathrm{R}}^{\left(\alpha \right)}(p\parallel q),\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\mathcal{D}}_{\mathrm{R}}^{\left(1\right)}(p\parallel q)& =\underset{\alpha \uparrow 1}{lim}{\mathcal{D}}_{\mathrm{R}}^{\left(\alpha \right)}(p\parallel q).\hfill \end{array}$$

(6)

Under some conditions, the limits in (5) and (6) are finite-valued, and converge to the Kullback–Leibler divergence. In other words,

$${\mathcal{D}}_{\mathrm{R}}^{(-1)}(p\parallel q)={\mathcal{D}}_{\mathrm{R}}^{\left(1\right)}(q\parallel p)={\mathcal{D}}_{\mathrm{KL}}(p\parallel q)<\infty ,$$

where ${\mathcal{D}}_{\mathrm{KL}}(p\parallel q)$ denotes the Kullback–Leibler divergence between p and q, which is given by

$${\mathcal{D}}_{\mathrm{KL}}(p\parallel q)={\int }_{T}plog\left(\frac{p}{q}\right)d\mu .$$

These conditions are stated in Proposition 1, given in the end of this section, for the case involving the generalized Rényi divergence.

The Rényi divergence in its standard form is given by

$${\mathcal{D}}^{\left(\alpha \right)}(p\parallel q)=\frac{1}{1-\alpha }log\left({\int }_T{p}^{\alpha }{q}^{1-\alpha }d\mu \right),\mathrm{for}\alpha \in (0,1).$$

(7)

Expression (4) is related to this form by

$${\mathcal{D}}_{\mathrm{R}}^{\left(\alpha \right)}(p\parallel q)=\frac{2}{1-\alpha }{\mathcal{D}}^{\left(\right(1-\alpha )/2)}(p\parallel q).$$

Beyond the change of variables, which results in α ranging in $[-1,1]$, expressions (4) and (7) differ by the factor $2/(1-\alpha )$. We opted to insert the term $2/(1-\alpha )$ so that some kind of symmetry could be maintained when the limits $\alpha \downarrow -1$ and $\alpha \uparrow 1$ are considered. In addition, the geometry induced by the version (4) conforms with Amari’s notation [5].

The Rényi divergence ${\mathcal{D}}_{\mathrm{R}}^{\left(\alpha \right)}(·\parallel ·)$ can be defined for every $\alpha \in \mathbb{R}$. However, for $\alpha \notin (-1,1)$, the expression (4) may not be finite-valued for every p and q in ${\mathcal{P}}_{\mu }$. To avoid some technicalities, we just consider $\alpha \in [-1,1]$.

Given p and q in ${\mathcal{P}}_{\mu }$, let us define

$$\kappa \left(\alpha \right)=-log\left({\int }_T{p}^{\frac{1-\alpha }{2}}{q}^{\frac{1+\alpha }{2}}d\mu \right),\mathrm{for}\alpha \in [-1,1],$$

which can be used to express the Rényi divergence as

$${\mathcal{D}}_{\mathrm{R}}^{\left(\alpha \right)}(p\parallel q)=\frac{4}{1-{\alpha }^2}\kappa \left(\alpha \right),\mathrm{for}\alpha \in (-1,1).$$

The function $\kappa \left(\alpha \right)$, which depends on p and q, can be defined as the unique non-negative real number for which

$${\int }_{T}exp\left(\frac{1-\alpha }{2}ln\left(p\right)+\frac{1+\alpha }{2}ln\left(q\right)+\kappa \left(\alpha \right)\right)d\mu =1.$$

(8)

The function $\kappa \left(\alpha \right)$ makes the role of a normalizing term. The generalization of Rényi divergence, which we propose, is based on the interpretation of $\kappa \left(\alpha \right)$ given in (8). Instead of the exponential function, we consider a φ-function in (8).

Fix any φ-function $\phi :\mathbb{R}\to (0,\infty )$. Given any p and q in ${\mathcal{P}}_{\mu }$, we take $\kappa \left(\alpha \right)=\kappa (\alpha ;p,q)\ge 0$ so that

$${\int }_T\phi \left(\frac{1-\alpha }{2}{\phi }^{-1}\left(p\right)+\frac{1+\alpha }{2}{\phi }^{-1}\left(q\right)+\kappa \left(\alpha \right)u_0\right)d\mu =1,$$

(9)

or, in other words, the term inside the integral is a probability distribution in ${\mathcal{P}}_{\mu }$. The existence and uniqueness of $\kappa \left(\alpha \right)$ as defined in (9) is guaranteed by condition (a3’).

We define a generalization of the Rényi divergence of order $\alpha \in (-1,1)$ as

$${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p\parallel q)=\frac{4}{1-{\alpha }^2}\kappa \left(\alpha \right).$$

(10)

For $\alpha =\pm 1$, this generalization is defined as a limit:

$$\begin{array}{cc}\hfill {\mathcal{D}}_{\phi }^{(-1)}(p\parallel q)& =\underset{\alpha \downarrow -1}{lim}{\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p\parallel q),\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\mathcal{D}}_{\phi }^{\left(1\right)}(p\parallel q)& =\underset{\alpha \uparrow 1}{lim}{\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p\parallel q).\hfill \end{array}$$

(12)

The cases $\alpha =\pm 1$ are related to a generalization of the Kullback–Leibler divergence, the so-called φ-divergence, which was introduced by the authors in [12]. The φ-divergence is given by (It was pointed out to us by an anonymous referee that this form of divergence is a special case of the $(\rho ,\tau )$-divergence for $\rho ={\phi }^{-1}$ and $f={\rho }^{-1}$ (see Section 3.5 in [19]) apart from a conformal factor, which is the denominator of (13)):

$${\mathcal{D}}_{\phi }(p\parallel q)=\frac{{\int }_T\frac{{\phi }^{-1}\left(p\right)-{\phi }^{-1}\left(q\right)}{{\left({\phi }^{-1}\right)}^{\prime }\left(p\right)}d\mu }{{\int }_T\frac{u_0}{{\left({\phi }^{-1}\right)}^{\prime }\left(p\right)}d\mu }.$$

(13)

Under some conditions, the limit in (11) or (12) is finite-valued and converges to the φ-divergence:

$${\mathcal{D}}_{\phi }^{(-1)}(p\parallel q)={\mathcal{D}}_{\phi }^{\left(1\right)}(q\parallel p)={\mathcal{D}}_{\phi }(p\parallel q)<\infty .$$

(14)

To show (14), we make use of the following result.

Lemma 2.

Assume that $\phi (·)$ is continuously differentiable. If for ${\alpha }_0,{\alpha }_1\in \mathbb{R}$, the expression

$${\int }_T\phi \left(\frac{1-\alpha }{2}{\phi }^{-1}\left(p\right)+\frac{1+\alpha }{2}{\phi }^{-1}\left(q\right)\right)d\mu <\infty$$

(15)

is satisfied for all $\alpha \in [{\alpha }_0,{\alpha }_1]$, then the derivative of $\kappa \left(\alpha \right)$ exists at any $\alpha \in ({\alpha }_0,{\alpha }_1)$, and is given by

$$\frac{\partial \kappa }{\partial \alpha }\left(\alpha \right)=-\frac{1}{2}\frac{{\int }_T[{\phi }^{-1}\left(q\right)-{\phi }^{-1}\left(p\right)]{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }{{\int }_T{\phi }^{\prime }\left(c_{\alpha }\right)u_{0}d\mu },$$

(16)

where $c_{\alpha }=\frac{1-\alpha }{2}{\phi }^{-1}\left(p\right)+\frac{1+\alpha }{2}{\phi }^{-1}\left(q\right)+\kappa \left(\alpha \right)u_0$.

Proof. 

For $\alpha \in ({\alpha }_0,{\alpha }_1)$ and $\kappa >0$, define

$$g(\alpha ,\kappa )={\int }_T\phi \left(\frac{1-\alpha }{2}{\phi }^{-1}\left(p\right)+\frac{1+\alpha }{2}{\phi }^{-1}\left(q\right)+\kappa u_0\right)d\mu .$$

The function $\kappa \left(\alpha \right)$ is defined implicitly by $g(\alpha ,\kappa (\alpha \left)\right)=1$. If we show that

(i)

the function $g(\alpha ,\kappa )$ is continuous in a neighborhood of $(\alpha ,\kappa (\alpha \left)\right)$,

(ii)

the partial derivatives $\frac{\partial g}{\partial \alpha }$ and $\frac{\partial g}{\partial \kappa }$ exist and are continuous at $(\alpha ,\kappa (\alpha \left)\right)$,

(iii)

and $\frac{\partial g}{\partial \kappa }(\alpha ,\kappa \left(\alpha \right))>0$,

then by the Implicit Function Theorem $\kappa \left(\alpha \right)$ is differentiable at $\alpha \in ({\alpha }_0,{\alpha }_1)$, and

$$\frac{\partial \kappa }{\partial \alpha }\left(\alpha \right)=-\frac{(\partial g/\partial \alpha )(\alpha ,\kappa (\alpha \left)\right)}{(\partial g/\partial \kappa )(\alpha ,\kappa (\alpha \left)\right)}.$$

(17)

We begin by verifying that $g(\alpha ,\kappa )$ is continuous. For fixed $\alpha \in ({\alpha }_0,{\alpha }_1)$ and $\kappa >0$, set ${\kappa }_0=2\kappa$. Denoting $A=\{t\in T:{\phi }^{-1}\left(q\left(t\right)\right)>{\phi }^{-1}\left(p\left(t\right)\right)\}$, we can write

$$\begin{array}{cc}\hfill \phi \left(\frac{1-\beta }{2}{\phi }^{-1}\left(p\right)+\frac{1+\beta }{2}{\phi }^{-1}\left(q\right)+\lambda u_0\right)& \le \phi \left({\phi }^{-1}\left(p\right)+\frac{1+\beta }{2}[{\phi }^{-1}\left(q\right)-{\phi }^{-1}\left(p\right)]+{\kappa }_0{u}_0\right)\hfill \\ & \le \phi \left({\phi }^{-1}\left(p\right)+\frac{1+{\alpha }_1}{2}[{\phi }^{-1}\left(q\right)-{\phi }^{-1}\left(p\right)]+{\kappa }_0{u}_0\right){\mathbf{1}}_A\hfill \\ & +\phi \left({\phi }^{-1}\left(p\right)+\frac{1+{\alpha }_0}{2}[{\phi }^{-1}\left(q\right)-{\phi }^{-1}\left(p\right)]+{\kappa }_0{u}_0\right){\mathbf{1}}_{T\backslash A},\hfill \end{array}$$

(18)

for every $\beta \in ({\alpha }_0,{\alpha }_1)$ and $\lambda \in (0,{\kappa }_0)$. Because the function on the right-hand side of (18) is integrable, we can apply the Dominated Convergence Theorem to conclude that

$$\underset{(\beta ,\lambda )\to (\alpha ,\kappa )}{lim}g(\beta ,\lambda )=g(\alpha ,\kappa ).$$

Now, we will show that the derivative of $g(\alpha ,\kappa )$ with respect to α exists and is continuous. Consider the difference

$$\frac{g(\gamma ,\lambda )-g(\beta ,\lambda )}{\gamma -\beta }={\int }_T\frac{1}{\gamma -\beta }\left[\phi \left(c_{\beta }+\frac{\gamma -\beta }{2}[{\phi }^{-1}\left(q\right)-{\phi }^{-1}\left(p\right)]+\lambda u_0\right)-\phi (c_{\beta }+\lambda u_0)\right]d\mu ,$$

(19)

where $c_{\beta }=\frac{1-\beta }{2}{\phi }^{-1}\left(p\right)+\frac{1+\beta }{2}{\phi }^{-1}\left(q\right)$. Represent by $f_{\beta ,\gamma ,\lambda }$ the function inside the integral sign in (19). For fixed $\alpha \in ({\alpha }_0,{\alpha }_1)$ and $\kappa >0$, denote ${\overline{\alpha }}_0=({\alpha }_0+\alpha )/2$, ${\overline{\alpha }}_1=(\alpha +{\alpha }_1)/2$, and ${\kappa }_0=2\kappa$. Because $\phi (·)$ is convex and increasing, it follows that

$$|f_{\beta ,\gamma ,\lambda }|\le f_{{\overline{\alpha }}_1,{\alpha }_1,{\kappa }_0}{\mathbf{1}}_A-f_{{\overline{\alpha }}_0,{\alpha }_0,{\kappa }_0}{\mathbf{1}}_{T\backslash A}=:f,\mathrm{for}\mathrm{all}\beta ,\gamma \in ({\overline{\alpha }}_0,{\overline{\alpha }}_1)\mathrm{and}\lambda \in (0,{\kappa }_0),$$

where $A=\{t\in T:{\phi }^{-1}\left(q\left(t\right)\right)>{\phi }^{-1}\left(p\left(t\right)\right)\}$. Observing that f is integrable, we can use the Dominated Convergence Theorem to get

$$\underset{\gamma \to \beta }{lim}{\int }_T{f}_{\beta ,\gamma ,\lambda }d\mu ={\int }_T\left(\underset{\gamma \to \beta }{lim}{f}_{\beta ,\gamma ,\lambda }\right)d\mu ,$$

and then

$$\frac{\partial g}{\partial \alpha }(\beta ,\lambda )=\frac{1}{2}{\int }_T[{\phi }^{-1}\left(q\right)-{\phi }^{-1}\left(p\right)]{\phi }^{\prime }(c_{\beta }+\lambda u_0)d\mu .$$

(20)

For $\beta \in ({\overline{\alpha }}_0,{\overline{\alpha }}_1)$ and $\lambda \in (0,{\kappa }_0)$, the function inside the integral sign in (20) is dominated by f. As a result, a second use of the Dominated Convergence Theorem shows that $\frac{\partial g}{\partial \alpha }$ is continuous at $(\alpha ,\kappa )$:

$$\underset{(\beta ,\lambda )\to (\alpha ,\kappa )}{lim}\frac{\partial g}{\partial \alpha }(\beta ,\lambda )=\frac{\partial g}{\partial \alpha }(\alpha ,\kappa ).$$

Using similar arguments, one can show that $\frac{\partial g}{\partial \kappa }(\alpha ,\kappa )$ exists and is continuous at any $\alpha \in ({\alpha }_0,{\alpha }_1)$ and $\kappa >0$, and is given by

$$\frac{\partial g}{\partial \kappa }(\alpha ,\kappa )={\int }_T{u}_0{\phi }^{\prime }(c_{\alpha }+\kappa u_0)d\mu .$$

(21)

Clearly, expression (21) implies that $\frac{\partial g}{\partial \kappa }(\alpha ,\kappa )>0$ for all $\alpha \in (0,{\alpha }_0)$ and $\kappa >0$.

We proved that items (i)–(iii) are satisfied. As consequence, the derivative of $\kappa \left(\alpha \right)$ exists at any $\alpha \in ({\alpha }_0,{\alpha }_1)$. Expression (16) for the derivative of $\kappa \left(\alpha \right)$ follows from (17), (20) and (21). ☐

As an immediate consequence of Lemma 2, we get the proposition below.

Proposition 1.

Assume that $\phi (·)$ is continuously differentiable.

(a)

If, for some ${\alpha }_0<-1$, expression (15) is satisfied for all $\alpha \in [{\alpha }_0,-1)$, then

$${\mathcal{D}}_{\phi }^{(-1)}(p\parallel q)=\underset{\alpha \downarrow -1}{lim}{\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p\parallel q)=2\frac{\partial \kappa }{\partial \alpha }(-1)={\mathcal{D}}_{\phi }(p\parallel q)<\infty .$$

(b)

If, for some ${\alpha }_1>1$, expression (15) is satisfied for all $\alpha \in (1,{\alpha }_1]$, then

$${\mathcal{D}}_{\phi }^{\left(1\right)}(p\parallel q)=\underset{\alpha \uparrow 1}{lim}{\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p\parallel q)=-2\frac{\partial \kappa }{\partial \alpha }\left(1\right)={\mathcal{D}}_{\phi }(q\parallel p)<\infty .$$

4. Generalized Statistical Manifolds

Statistical manifolds consist of a collection of probability distributions endowed with a metric and α-connections, which are defined in terms of the derivative of $l(t;\theta )=logp(t;\theta )$. In a generalized statistical manifold, the metric and connection are defined in terms of $f(t;\theta )={\phi }^{-1}\left(p(t;\theta )\right)$. Instead of the logarithm, we consider the inverse ${\phi }^{-1}(·)$ of a φ-function. Generalized statistical manifolds were introduced by the authors in [17,18]. Among examples of the generalized statistical manifold, (parametric) φ-families of probability distributions are of greatest importance. The non-parametric counterpart was investigated in [11,12]. The metric in φ-families can be defined as the Hessian of a function; i.e., φ-families are Hessian manifolds [30]. In [17,18], the φ-divergence gives rise to a pair of dual connections $D^{(-1)}$ and $D^{\left(1\right)}$; and then for $\alpha \in (-1,1)$ the α-connection $D^{\left(\alpha \right)}$ is defined as the convex combination $D^{\left(\alpha \right)}=\frac{1-\alpha }{2}{D}^{(-1)}+\frac{1+\alpha }{2}{D}^{\left(1\right)}$. In the present paper, we show that the connection induced by ${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(·\parallel ·)$, the generalization of Rényi divergence, corresponds to $D^{\left(\alpha \right)}$.

4.1. Definitions

Let $\phi :\mathbb{R}\to (0,\infty )$ be a φ-function. A generalized statistical manifold $\mathcal{P}=\left\{p\right(t;\theta ):\theta \in \mathsf{\Theta }\}$ is a collection of probability distributions $p_{\theta }\left(t\right):=p(t;\theta )$, indexed by parameters $\theta =({\theta }^1,\cdots ,{\theta }^n)\in \mathsf{\Theta }$ in a one-to-one relation, such that

(m1)

Θ is a domain (open and connected set) in ${\mathbb{R}}^n$;

(m2)

$p(t;\theta )$ is differentiable with respect to θ;

(m3)

the matrix $g=\left(g_{ij}\right)$ defined by

$$g_{ij}=-E_{\theta }^{\prime }\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}\right],$$

(22)

is positive definite at each $\theta \in \mathsf{\Theta }$, where

$$E_{\theta }^{\prime }[·]=\frac{{\int }_T(·){\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu };$$

(23)

(m4)

the operations of integration with respect to μ and differentiation with respect to ${\theta }^i$ commute in all calculations found below, which are related to the metric and connections.

The matrix $g=\left(g_{ij}\right)$ equips $\mathcal{P}$ with a metric. By the chain rule, the tensor related to $g=\left(g_{ij}\right)$ is invariant under change of coordinates. The (classical) statistical manifold is a particular case in which $\phi \left(u\right)=exp\left(u\right)$ and $u_0={\mathbf{1}}_T$.

We introduce a notation similar to Equation (23) that involves higher order derivatives of $\phi (·)$. For each $n\ge 1$, we define

$$E_{\theta }^{\left(n\right)}[·]=\frac{{\int }_T(·){\phi }^{\left(n\right)}\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu }.$$

(24)

We also use $E_{\theta }^{\prime }[·]$, $E_{\theta }^{″}[·]$ and $E_{\theta }^{‴}[·]$ to denote $E_{\theta }^{\left(n\right)}[·]$ for $n=1,2,3$, respectively. The notation (24) appears in expressions related to the metric and connections.

Using property (m4), we can find an alternate expression for $g_{ij}$ as well as an identification involving tangent spaces. The matrix $g=\left(g_{ij}\right)$ can be equivalently defined by

$$g_{ij}=E_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\right].$$

(25)

As a consequence of this equivalence, the tangent space $T_{p_{\theta }}\mathcal{P}$ can be identified with ${\tilde{T}}_{p_{\theta }}\mathcal{P}$, the vector space spanned by $\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}$, and endowed with the inner product ${\langle \tilde{X},\tilde{Y}\rangle }_{\theta }:=E_{\theta }^{″}[\tilde{X}\tilde{Y}]$. The mapping

$$\sum _i{a}_i\frac{\partial }{\partial {\theta }^i}\mapsto \sum _i{a}_i\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}$$

defines an isometry between $T_{p_{\theta }}\mathcal{P}$ and ${\tilde{T}}_{p_{\theta }}\mathcal{P}$.

To verify (25), we differentiate ${\int }_T{p}_{\theta }d\mu =1$, with respect to ${\theta }^i$, to get

$$0=\frac{\partial }{\partial {\theta }^i}{\int }_T{p}_{\theta }d\mu ={\int }_T\frac{\partial }{\partial {\theta }^i}\phi \left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu ={\int }_T\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}{\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu .$$

(26)

Now, differentiating with respect to ${\theta }^j$, we obtain

$$0={\int }_T\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}{\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu +{\int }_T\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}{\phi }^{″}\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu ,$$

and then (25) follows. In view of (26), we notice that every vector $\tilde{X}$ belonging to ${\tilde{T}}_{p_{\theta }}\mathcal{P}$ satisfies $E_{\theta }^{\prime }\left[\tilde{X}\right]=0$.

The metric $g=\left(g_{ij}\right)$ gives rise to a Levi–Civita connection ∇ (i.e., a torsion-free, metric connection), whose corresponding Christoffel symbols ${\mathsf{\Gamma }}_{ijk}$ are given by

$${\mathsf{\Gamma }}_{ijk}:=\frac{1}{2}\left(\frac{\partial g_{ki}}{\partial {\theta }^j}+\frac{\partial g_{kj}}{\partial {\theta }^i}-\frac{\partial g_{ij}}{\partial {\theta }^k}\right).$$

(27)

Using expression (25) to calculate the derivatives in (27), we can express

$$\begin{array}{cc}\hfill {\mathsf{\Gamma }}_{ijk}& =E_{\theta }^{″}\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]+\frac{1}{2}{E}_{\theta }^{‴}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]\hfill \\ & \phantom{=}-\frac{1}{2}{E}_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\right]-\frac{1}{2}{E}_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\right]\hfill \\ & \phantom{=}+\frac{1}{2}{E}_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right].\hfill \end{array}$$

As we will show later, the Levi–Civita connection ∇ corresponds to the connection derived from the divergence ${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(·\parallel ·)$ with $\alpha =0$.

4.2. φ-Families

Let $c:T\to \mathbb{R}$ be a measurable function for which $p=\phi \left(c\right)$ is a probability density in ${\mathcal{P}}_{\mu }$. Fix measurable functions $u_1,\cdots ,u_n:T\to \mathbb{R}$. A (parametric) φ-family ${\mathcal{F}}_p=\{p_{\theta }:\theta \in \mathsf{\Theta }\}$, centered at $p=\phi \left(c\right)$, is a set of probability distributions in ${\mathcal{P}}_{\mu }$, whose members can be written in the form

$$p_{\theta }:=\phi \left(c+\sum _{i=1}^n{\theta }^i{u}_i-\psi \left(\theta \right)u_0\right),\mathrm{for}\mathrm{each}\theta =\left({\theta }^i\right)\in \mathsf{\Theta },$$

(28)

where $\psi :\mathsf{\Theta }\to [0,\infty )$ is a normalizing function, which is introduced so that expression (28) defines a probability distribution belonging to ${\mathcal{P}}_{\mu }$.

The functions $u_1,\cdots ,u_n$ are not arbitrary. They are chosen to satisfy the following assumptions:

(i)

$u_0,u_1,\cdots ,u_n$ are linearly independent,

(ii)

${\int }_T{u}_i{\phi }^{\prime }\left(c\right)d\mu =0$, and

(iii)

there exists $\epsilon >0$ such that ${\int }_T\phi (c+\lambda u_i)d\mu <\infty$, for all $\lambda \in (-\epsilon ,\epsilon )$.

Moreover, the domain $\mathsf{\Theta }\subseteq {\mathbb{R}}^n$ is defined as the set of all vectors $\theta =\left({\theta }^i\right)$ for which

$${\int }_T\phi \left(c+\lambda \sum _{i=1}^n{\theta }^i{u}_i\right)d\mu <\infty ,\mathrm{for}\mathrm{some}\lambda >1.$$

Condition (i) implies that the mapping defined by (28) is one-to-one. Assumption (ii) makes of ψ a non-negative function. Indeed, by the convexity of $\phi (·)$, along with (ii), we can write

$${\int }_T\phi \left(c\right)d\mu ={\int }_T\left[\phi \left(c\right)+\left(\sum _{i=1}^n{\theta }^i{u}_i\right){\phi }^{\prime }\left(c\right)\right]d\mu \le {\int }_T\phi \left(c+\sum _{i=1}^n{\theta }^i{u}_i\right)d\mu ,$$

which implies $\psi \left(\theta \right)\ge 0$. By condition (iii), the domain Θ is an open neighborhood of the origin. If the set T is finite, condition (iii) is always satisfied. One can show that the domain Θ is open and convex. Moreover, the normalizing function ψ is also convex (or strictly convex if $\phi (·)$ is strictly convex). Conditions (ii) and (iii) also appears in the definition of non-parametric φ-families. For further details, we refer to [11,12].

In a φ-family ${\mathcal{F}}_p$, the matrix $\left(g_{ij}\right)$ given by (22) or (25) can be expressed as the Hessian of ψ. If $\phi (·)$ is strictly convex, then$\left(g_{ij}\right)$ is positive definite. From

$$\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}=u_i-\frac{\partial \psi }{\partial {\theta }^i},-\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}=-\frac{{\partial }^2\psi }{\partial {\theta }^i\partial {\theta }^j},$$

it follows that $g_{ij}=\frac{{\partial }^2\psi }{\partial {\theta }^i\partial {\theta }^j}$.

The next two results show how the generalization of Rényi divergence and the φ-divergence are related to the normalizing function in φ-families.

Proposition 2.

In a φ-family ${\mathcal{F}}_p$, the generalization of Rényi divergence for $\alpha \in (-1,1)$ can be expressed in terms of the normalizing function ψ as follows:

$${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p_{\theta }\parallel p_{\vartheta })=\frac{2}{1+\alpha }\psi \left(\theta \right)+\frac{2}{1-\alpha }\psi \left(\vartheta \right)-\frac{4}{1-{\alpha }^2}\psi \left(\frac{1-\alpha }{2}\theta +\frac{1+\alpha }{2}\vartheta \right),$$

(29)

for all $\theta ,\vartheta \in \mathsf{\Theta }$.

Proof. 

Recall the definition of $\kappa \left(\alpha \right)$ as the real number for which

$${\int }_T\phi \left(\frac{1-\alpha }{2}{\phi }^{-1}\left(p_{\theta }\right)+\frac{1+\alpha }{2}{\phi }^{-1}\left(p_{\vartheta }\right)+\kappa \left(\alpha \right)u_0\right)d\mu =1.$$

Using expression (28) for probability distributions in ${\mathcal{F}}_p$, we can write

$$\begin{array}{ll}\frac{1-\alpha }{2}{\phi }^{-1}\left(p_{\theta }\right)+\frac{1+\alpha }{2}{\phi }^{-1}& \left(p_{\vartheta }\right)+\kappa \left(\alpha \right)u_0\\ & =c+\sum _{i=1}^n\left(\frac{1-\alpha }{2}{\theta }^i+\frac{1+\alpha }{2}{\vartheta }^i\right)u_i-\left(\frac{1-\alpha }{2}\psi \left(\theta \right)+\frac{1+\alpha }{2}\psi \left(\vartheta \right)-\kappa \left(\alpha \right)\right)u_0\\ & =c+\sum _{i=1}^n\left(\frac{1-\alpha }{2}{\theta }^i+\frac{1+\alpha }{2}{\vartheta }^i\right)u_i-\psi \left(\frac{1-\alpha }{2}\theta +\frac{1+\alpha }{2}\vartheta \right)u_0.\end{array}$$

The last equality is a consequence of the domain Θ being convex. Thus, it follows that

$$\kappa \left(\alpha \right)=\frac{1-\alpha }{2}\psi \left(\theta \right)+\frac{1+\alpha }{2}\psi \left(\vartheta \right)-\psi \left(\frac{1-\alpha }{2}\theta +\frac{1+\alpha }{2}\vartheta \right).$$

By the definition of ${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(·\parallel ·)$, we get (29). ☐

Proposition 3.

In a φ-family ${\mathcal{F}}_p$, the φ-divergence is related to the normalizing function ψ by the equality

$${\mathcal{D}}_{\phi }(p_{\theta }\parallel p_{\vartheta })=\psi \left(\vartheta \right)-\psi \left(\theta \right)-\nabla \psi \left(\theta \right)·(\vartheta -\theta ),$$

(30)

for all $\theta ,\vartheta \in \mathsf{\Theta }$.

Proof. 

To show (30), we use

$$\frac{\partial \psi }{\partial {\theta }^i}\left(\theta \right)=\frac{{\int }_T{u}_i{\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu },$$

which is a consequence of (Lemma 10 in [12]). In view of ${\left({\phi }^{-1}\right)}^{\prime }\left(u\right)=1/{\phi }^{\prime }\left({\phi }^{-1}\left(u\right)\right)$, expression (13) with $p=p_{\theta }$ and $q=p_{\vartheta }$ results in

$${\mathcal{D}}_{\phi }(p_{\theta }\parallel p_{\vartheta })=\frac{{\int }_T[{\phi }^{-1}\left(p_{\theta }\right)-{\phi }^{-1}\left(p_{\vartheta }\right)]{\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu }.$$

(31)

Inserting into (31) the difference

$$\begin{array}{cc}\hfill {\phi }^{-1}\left(p_{\theta }\right)-{\phi }^{-1}\left(p_{\vartheta }\right)& =\left(c+\sum _{i=1}^n{\theta }^i{u}_i-\psi \left(\theta \right)u_0\right)-\left(c+\sum _{i=1}^n{\vartheta }^i{u}_i-\psi \left(\vartheta \right)u_0\right)\hfill \\ & =\psi \left(\vartheta \right)u_0-\psi \left(\theta \right)u_0-\sum _{i=1}^n({\vartheta }^i-{\theta }^i)u_i,\hfill \end{array}$$

we get expression (30). ☐

In Proposition 2, the expression on the right-hand side of Equation (29) defines a divergence on its own, which was investigated by Jun Zhang in [19]. Proposition 3 asserts that the φ-divergence ${\mathcal{D}}_{\phi }(p_{\theta }\parallel p_{\vartheta })$ coincides with the Bregman divergence [31,32] associated with the normalizing function ψ for points ϑ and θ in Θ. Because ψ is convex and attains a minimum at $\theta =0$, it follows that $\frac{\partial \psi }{\partial {\theta }^i}\left(\theta \right)=0$ at $\theta =0$. As a result, equality (30) reduces to ${\mathcal{D}}_{\phi }(p\parallel p_{\theta })=\psi \left(\theta \right)$.

4.3. Geometry Induced by ${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(·\parallel ·)$

In this section, we assume that $\phi (·)$ is continuously differentiable and strictly convex. The latter assumption guarantees that

$${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p\parallel q)=0\mathrm{if}\mathrm{and}\mathrm{only}\mathrm{if}p=q.$$

(32)

The generalized Rényi divergence induces a metric $g=\left(g_{ij}\right)$ in generalized statistical manifolds $\mathcal{P}$. This metric is given by

$$g_{ij}=-{\left[{\left(\frac{\partial }{\partial {\theta }^i}\right)}_p{\left(\frac{\partial }{\partial {\theta }^j}\right)}_q{\mathcal{D}}_{\phi }^{\alpha }(p\parallel q)\right]}_{q=p}.$$

(33)

To show that this expression defines a metric, we have to verify that $g_{ij}$ is invariant under change of coordinates, and $\left(g_{ij}\right)$ is positive definite. The first claim follows from the chain rule. The positive definiteness of $\left(g_{ij}\right)$ is a consequence of Proposition 4, which is given below.

Proposition 4.

The metric induced by ${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(·\parallel ·)$ coincides with the metric given by (22) or (25).

Proof. 

Fix any $\alpha \in (-1,1)$. Applying the operator ${\left(\frac{\partial }{\partial {\theta }^j}\right)}_{p_{\vartheta }}$ to

$${\int }_T\phi \left(c_{\alpha }\right)d\mu =1,$$

where $c_{\alpha }=\frac{1-\alpha }{2}{\phi }^{-1}\left(p_{\theta }\right)+\frac{1+\alpha }{2}{\phi }^{-1}\left(p_{\vartheta }\right)+\kappa \left(\alpha \right)u_0$, we obtain

$${\int }_T\left(\frac{1+\alpha }{2}\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^j}+{\left(\frac{\partial }{\partial {\theta }^j}\right)}_{p_{\vartheta }}\kappa \left(\alpha \right)u_0\right){\phi }^{\prime }\left(c_{\alpha }\right)d\mu =0,$$

which results in

$${\left(\frac{\partial }{\partial {\theta }^j}\right)}_{p_{\vartheta }}\kappa \left(\alpha \right)=-\frac{1+\alpha }{2}\frac{{\int }_T\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^j}{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }.$$

By the standard differentiation rules, we can write

$$\begin{array}{cc}& {\left(\frac{\partial }{\partial {\theta }^i}\right)}_{p_{\theta }}{\left(\frac{\partial }{\partial {\theta }^j}\right)}_{p_{\vartheta }}\kappa \left(\alpha \right)=-\frac{1+\alpha }{2}\frac{{\int }_T[\frac{1-\alpha }{2}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}+{\left(\frac{\partial }{\partial {\theta }^i}\right)}_{p_{\theta }}\kappa \left(\alpha \right)u_0]\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^j}{\phi }^{″}\left(c_{\alpha }\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }\hfill \\ & +\frac{1+\alpha }{2}\frac{{\int }_T\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^j}{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }\frac{{\int }_T{u}_0[\frac{1-\alpha }{2}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}+{\left(\frac{\partial }{\partial {\theta }^i}\right)}_{p_{\theta }}\kappa \left(\alpha \right)u_0]{\phi }^{″}\left(c_{\alpha }\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }.\hfill \end{array}$$

(34)

Noticing that ${\int }_T\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^j}{\phi }^{\prime }\left(c_{\alpha }\right)d\mu =0$ for $p_{\vartheta }=p_{\theta }$, the second term on the right-hand side of Equation (34) vanishes, and then

$${\left[{\left(\frac{\partial }{\partial {\theta }^i}\right)}_{p_{\theta }}{\left(\frac{\partial }{\partial {\theta }^j}\right)}_{p_{\vartheta }}\kappa \left(\alpha \right)\right]}_{p_{\vartheta }=p_{\theta }}=-\frac{1-{\alpha }^2}{4}\frac{{\int }_T\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}{\phi }^{″}\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left({\phi }^{-1}\left(p_{\theta }\right)\right)d\mu }.$$

If we use the notation introduced in (24), we can write

$$g_{ij}=-{\left[{\left(\frac{\partial }{\partial {\theta }^i}\right)}_{p_{\theta }}{\left(\frac{\partial }{\partial {\theta }^j}\right)}_{p_{\vartheta }}{\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p_{\theta }\parallel p_{\vartheta })\right]}_{p_{\vartheta }=p_{\theta }}=E_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\right].$$

It remains to show the case $\alpha =\pm 1$. Comparing (13) and (23), we can write

$${\mathcal{D}}_{\phi }(p_{\theta }\parallel p_{\vartheta })=E_{\theta }^{\prime }[{\phi }^{-1}\left(p_{\theta }\right)-{\phi }^{-1}\left(p_{\vartheta }\right)].$$

(35)

We use the equivalent expressions

$$g_{ij}={\left[{\left(\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\right)}_p{\mathcal{D}}_{\phi }^{\alpha }(p\parallel q)\right]}_{q=p}={\left[{\left(\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\right)}_q{\mathcal{D}}_{\phi }^{\alpha }(p\parallel q)\right]}_{q=p},$$

which follows from condition (32), to infer that

$$g_{ij}={\left[{\left(\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\right)}_{p_{\vartheta }}{\mathcal{D}}_{\phi }(p_{\theta }\parallel p_{\vartheta })\right]}_{p_{\theta }=p_{\vartheta }}=-E_{\theta }^{\prime }\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}\right].$$

(36)

Because ${\mathcal{D}}_{\phi }^{(-1)}(p\parallel q)={\mathcal{D}}_{\phi }^{\left(1\right)}(q\parallel p)={\mathcal{D}}_{\phi }(p\parallel q)$, we conclude that the metric defined by (22) coincides with the metric induced by ${\mathcal{D}}_{\phi }^{(-1)}(·\parallel ·)$ and ${\mathcal{D}}_{\phi }^{\left(1\right)}(·\parallel ·)$. ☐

In generalized statistical manifolds, the generalized Rényi divergence ${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(·\parallel ·)$ induces a connection $D^{\left(\alpha \right)}$, whose Christoffel symbols ${\mathsf{\Gamma }}_{ijk}^{\left(\alpha \right)}$ are given by

$${\mathsf{\Gamma }}_{ijk}^{\left(\alpha \right)}=-{\left[{\left(\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\right)}_p{\left(\frac{\partial }{\partial {\theta }^k}\right)}_q{\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p\parallel q)\right]}_{q=p}.$$

Because ${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p\parallel q)={\mathcal{D}}_{\phi }^{(-\alpha )}(q\parallel p)$, it follows that $D^{\left(\alpha \right)}$ and $D^{(-\alpha )}$ are mutually dual for any $\alpha \in [-1,1]$. In other words, ${\mathsf{\Gamma }}_{ijk}^{\left(\alpha \right)}$ and ${\mathsf{\Gamma }}_{ijk}^{(-\alpha )}$ satisfy the relation $\frac{\partial g_{jk}}{\partial {\theta }^i}={\mathsf{\Gamma }}_{ijk}^{\left(\alpha \right)}+{\mathsf{\Gamma }}_{ikj}^{(-\alpha )}$. A development involving expression (35) results in

$$\begin{array}{cc}\hfill {\mathsf{\Gamma }}_{ijk}^{\left(1\right)}& =E_{\theta }^{″}\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]-E_{\theta }^{\prime }\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right],\hfill \end{array}$$

(37)

and

$$\begin{array}{cc}\hfill {\mathsf{\Gamma }}_{ijk}^{(-1)}& =E_{\theta }^{″}\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]+E_{\theta }^{‴}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]\hfill \\ & \phantom{=}-E_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\right]\hfill \\ & \phantom{=}-E_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\right].\hfill \end{array}$$

(38)

For $\alpha \in (-1,1)$, the Christoffel symbols ${\mathsf{\Gamma }}_{ijk}^{\left(\alpha \right)}$ can be written as a convex combination of ${\mathsf{\Gamma }}_{ijk}^{(-1)}$ and ${\mathsf{\Gamma }}_{ijk}^{(-1)}$, as asserted in the next result.

Proposition 5.

The Christoffel symbols ${\mathsf{\Gamma }}_{ijk}^{\left(\alpha \right)}$ induced by the divergence ${\mathcal{D}}_{\phi }^{\left(\alpha \right)}(·\parallel ·)$ satisfy the relation

$${\mathsf{\Gamma }}_{ijk}^{\left(\alpha \right)}=\frac{1-\alpha }{2}{\mathsf{\Gamma }}_{ijk}^{(-1)}+\frac{1+\alpha }{2}{\mathsf{\Gamma }}_{ijk}^{\left(1\right)},\mathit{for}\alpha \in [-1,1].$$

(39)

Proof. 

For $\alpha =\pm 1$, equality (39) follows trivially. Thus, we assume $\alpha \in (-1,1)$. By (34), we can write

$$\begin{array}{cc}\hfill {\left(\frac{\partial }{\partial {\theta }^i}\right)}_{p_{\theta }}{\left(\frac{\partial }{\partial {\theta }^k}\right)}_{p_{\vartheta }}\kappa \left(\alpha \right)& =-\frac{1+\alpha }{2}\frac{{\int }_T[\frac{1-\alpha }{2}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}+{\left(\frac{\partial }{\partial {\theta }^i}\right)}_{p_{\theta }}\kappa \left(\alpha \right)u_0]\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^k}{\phi }^{″}\left(c_{\alpha }\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }\hfill \\ & \phantom{=}+\frac{1+\alpha }{2}\frac{{\int }_T\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^k}{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }\frac{{\int }_T{u}_0[\frac{1-\alpha }{2}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}+{\left(\frac{\partial }{\partial {\theta }^i}\right)}_{p_{\theta }}\kappa \left(\alpha \right)u_0]{\phi }^{″}\left(c_{\alpha }\right)d\mu }{{\int }_T{u}_0{\phi }^{\prime }\left(c_{\alpha }\right)d\mu }.\hfill \end{array}$$

(40)

Applying ${\left(\frac{\partial }{\partial {\theta }^j}\right)}_{p_{\theta }}$ to the first term on the right-hand side of (40), and then equating $p_{\vartheta }=p_{\theta }$, we obtain

$$\begin{array}{cc}& -\frac{1-{\alpha }^2}{4}{E}_{\theta }^{″}\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]-\frac{1+\alpha }{2}{\left(\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\right)}_{p_{\theta }}\kappa \left(\alpha \right)E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]\hfill \\ & -\frac{1-{\alpha }^2}{4}\frac{1-\alpha }{2}{E}_{\theta }^{‴}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]\hfill \\ & +\frac{1-{\alpha }^2}{4}\frac{1-\alpha }{2}{E}_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^k}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^j}\right].\hfill \end{array}$$

(41)

Similarly, if we apply ${\left(\frac{\partial }{\partial {\theta }^j}\right)}_{p_{\theta }}$ to the second term on the right-hand side of (40), and make $p_{\vartheta }=p_{\theta }$, we get

$$\frac{1-{\alpha }^2}{4}\frac{1-\alpha }{2}{E}_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\right].$$

(42)

Collecting (41) and (42), we can write

$$\begin{array}{cc}\hfill {\mathsf{\Gamma }}_{ijk}^{\left(\alpha \right)}& =-\frac{4}{1-{\alpha }^2}{\left[{\left(\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\right)}_{p_{\theta }}{\left(\frac{\partial }{\partial {\theta }^k}\right)}_{p_{\vartheta }}\kappa \left(\alpha \right)\right]}_{p_{\theta }=p_{\vartheta }}\hfill \\ & =E_{\theta }^{″}\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]+\frac{1-\alpha }{2}{E}_{\theta }^{‴}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]\hfill \\ & \phantom{=}-\frac{1-\alpha }{2}{E}_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^j}\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\right]\hfill \\ & \phantom{=}-\frac{1-\alpha }{2}{E}_{\theta }^{″}\left[\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i}\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^k}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\vartheta }\right)}{\partial {\theta }^j}\right]\hfill \\ & \phantom{=}-\frac{1+\alpha }{2}{E}_{\theta }^{\prime }\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^j}\right]E_{\theta }^{″}\left[u_0\frac{\partial {\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^k}\right],\hfill \end{array}$$

(43)

where we used

$$\begin{array}{cc}\hfill {\left(\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\right)}_{p_{\theta }}\kappa \left(\alpha \right)& =\frac{1-{\alpha }^2}{4}{\left[{\left(\frac{{\partial }^2}{\partial {\theta }^i\partial {\theta }^j}\right)}_{p_{\theta }}{\mathcal{D}}_{\phi }^{\left(\alpha \right)}(p_{\theta }\parallel p_{\vartheta })\right]}_{p_{\vartheta }=p_{\theta }}\hfill \\ & =\frac{1-{\alpha }^2}{4}{g}_{ij}=-\frac{1-{\alpha }^2}{4}{E}_{\theta }^{\prime }\left[\frac{{\partial }^2{\phi }^{-1}\left(p_{\theta }\right)}{\partial {\theta }^i\partial {\theta }^i}\right].\hfill \end{array}$$

Expression (39) follows from (37), (38) and (43). ☐

5. Conclusions

In [17,18], the authors introduced a pair of dual connections $D^{(-1)}$ and $D^{\left(1\right)}$ induced by φ-divergence. The main motivation of the present work was to find a (non-trivial) family of α-divergences, whose induced α-connections are convex combinations of $D^{(-1)}$ and $D^{\left(1\right)}$. As a result of our efforts, we proposed a generalization of Rényi divergence. The connection $D^{\left(\alpha \right)}$ induced by the generalization of Rényi divergence satisfies the relation $D^{\left(\alpha \right)}=\frac{1-\alpha }{2}{D}^{(-1)}+\frac{1+\alpha }{2}{D}^{\left(1\right)}$. To generalize Rényi divergence, we made use of properties of φ-functions. This makes evident the importance of φ-functions in the geometry of non-standard models. In standard statistical manifolds, even though Amari’s α-divergence and Rényi divergence (with $\alpha \in [-1,1]$) do not coincide, they induce the same family of α-connections. This striking result requires further investigation. Future work should focus on how the generalization of Rényi divergence is related to Zhang’s $(\rho ,\tau )$-divergence, and also how the present proposal is related to the model presented in [33].