Sharp Estimates for Proximity of Geometric and Related Sums Distributions to Limit Laws

Abstract

The convergence rate in the famous Rényi theorem is studied by means of the Stein method refinement. Namely, it is demonstrated that the new estimate of the convergence rate of the normalized geometric sums to exponential law involving the ideal probability metric of the second order is sharp. Some recent results concerning the convergence rates in Kolmogorov and Kantorovich metrics are extended as well. In contrast to many previous works, there are no assumptions that the summands of geometric sums are positive and have the same distribution. For the first time, an analogue of the Rényi theorem is established for the model of exchangeable random variables. Also within this model, a sharp estimate of convergence rate to a specified mixture of distributions is provided. The convergence rate of the appropriately normalized random sums of random summands to the generalized gamma distribution is estimated. Here, the number of summands follows the generalized negative binomial law. The sharp estimates of the proximity of random sums of random summands distributions to the limit law are established for independent summands and for the model of exchangeable ones. The inverse to the equilibrium transformation of the probability measures is introduced, and in this way a new approximation of the Pareto distributions by exponential laws is proposed. The integral probability metrics and the techniques of integration with respect to sign measures are essentially employed.

1. Introduction

The theory of sums of random variables belongs to the core of modern probability theory. The fundamental contribution to the formation of the classical core was made by A. de Moivre, J. Bernoulli, P.-S. Laplace, D. Poisson, P.L. Chebyshev, A.A. Markov, A.M. Lyapunov, E. Borel, S.N. Bernstein, P. Lévy, J. Lindeberg, H. Cramér, A.N. Kolmogorov, A.Ya. Khinchin, B.V. Gnedenko, J.L. Doob, W. Feller, Yu.V. Prokhorov, A.A. Borovkov, Yu.V. Linnik, I.A. Ibragimov, A. Rényi, P. Erdös, M. Csörgö, P. Révész, C. Stein, P. Hall, V.V. Petrov, V.M. Zolotarev, J. Jacod and A.N. Shiryaev among others. The first steps led to limit theorems for appropriately normalized partial sums of sequences of independent random variables. Besides the laws of large numbers, special attention was paid to emergence of Gaussian and Poisson limit laws. Note that despite many efforts to find necessary and sufficient conditions for the validity of the central limit theorem (the term was proposed by G. Pólya for a class of limit theorems describing weak convergence of distributions of normalized sums of random variables to the Gaussian law), this problem was completely resolved for independent summands only in the second part of the 20th century in the works by V.M. Zolotarev and V.I. Rotar. Also in the last century, the beautiful theory of infinitely divisible and stable laws was constructed. New developments of infinite divisibility along with classical theory can be found in [1]. For exposition of the theory of stable distributions and their applications, we refer to [2], see also references therein.

Parallel to partial sums of a sequence of random variables (and vectors), other significant schemes have appeared, for instance, the arrays of random variables. Moreover, in physics, biology and other domains, researchers found that it was essential to study the sums of random variables when the number of summands was random. Thus, the random sums with random summands became an important object of investigation. One can mention the branching processes which stem from the 19th century population models by I.J. Bienaymé, F. Galton and H.W. Watson that are still intensively being developed, see, e.g., [3]. In the theory of risk, it is worth recalling the celebrated Cramér–Lundberg model for dynamics of the capital of an insurance company, see, e.g., Ch. 6 in [4]. Various examples of models described by random sums are considered in Ch. 1 of [5], including (see Example 1.2.1) the relationship between certain random sums analysis and the famous Pollaczek–Khinchin formula in queuing theory. A vast literature deals with the so-called geometric sums. There, one studies the sum of independent identically distributed random variables, and the summation index follows the geometric distribution, being independent with summands. Such random sums can model many real world phenomena, e.g., in queuing, insurance and reliability, see the Section “Origin of Geometric Sums” in the Introduction of [6]. Furthermore, a multitude of important stochastic models described by systems of dependent random variables occurred to meet diverse applications, see, e.g., [7]. In particular, the general theory of stochastic processes and random fields arose in the last century (for introduction to random fields, see, e.g., [8]).

An intriguing problem of estimating the convergence rate to a limit law was addressed by A.C. Berry and C.-G. Esseen. Their papers initiated the study of proximity for distribution functions of the normalized partial sums of independent random variables to the distribution function of a standard Gaussian law in the framework of the classical theory of random sums.

To assess the proximity of distributions, we will employ various integral probability metrics. Usually, for random variables Y, Z and a specified class $\mathcal{H}$ of functions $h:\mathbb{R}\to \mathbb{R}$, one sets

$$d_{\mathcal{H}}(Y,Z):=\underset{h\in \mathcal{H}}{sup}|\mathbb{E}\left[h\left(Y\right)\right]-\mathbb{E}\left[h\left(Z\right)\right]|\in [0,\infty ].$$

(1)

Clearly, $d_{\mathcal{H}}(Y,Z)$ is a functional depending on $law\left(Y\right)$ and $law\left(Z\right)$, i.e., distributions of Y and Z. A class $\mathcal{H}$ should be rich enough to guarantee that $d_{\mathcal{H}}$ possesses the properties of a metric (or semi-metric). The general theory of probability metrics is presented, e.g., in [9,10]. In terms of such metrics, one often compares the distribution of a random variable Y under consideration with that of a target random variable Z. In Section 2, we recall the definitions of the Kolmogorov and Kantorovich (alternatively called Wasserstein) distances and Zolotarev ideal metrics corresponding to the adequate choice of $\mathcal{H}$, denoted below as $\mathcal{K}$, ${\mathcal{H}}_1$ and ${\mathcal{H}}_2$, respectively.

It should be emphasized that for sums of random variables, deep results were established along with creation and development of different methods of analysis. One can mention the method of characteristic functions due to the works of J.Fourier, P.-S.Laplace and A.M.Lyapunov, the method of moments proposed by P.L.Chebyshev and developed by A.A.Markov, the Lindeberg method of employing auxiliary Gaussian random variables and the Bernstein techniques of large and small boxes. In 1972, C.Stein in [11] (see also [12]) introduced the new method to estimate the proximity of the distribution under consideration to a normal law. Furthermore, this powerful method was developed in the framework of classical limit theorems of the probability theory. We describe this method in Section 2. Applying the Stein method along with other tools, one can establish in certain cases the sharp estimates of closeness between a target distribution and other ones in specified metrics (see, e.g., [13,14]). We recommend the books [15,16] and the paper [17] for basic ideas of the ingenious Stein method. The development of this techniques under mild moment restrictions for summands is treated in [18,19]. We mention in passing that there are deep generalizations of Stein techniques involving generators of certain Markov processes; a compact exposition is provided, e.g., on p. 2 of [20].

In the theory of random sums of random summands, the limit theorems with exponential law as a target distribution play a role similar to the central limit theorem for (nonrandom) sums of random variables. Here, one has to underline the principal role of the Rényi classical theorem for geometric sums published in [21]. Recall this famous result. Let $X_1,X_2,\dots$ be a sequence of independent identically distributed (i.i.d.) random variables such that $\mu :=\mathbb{E}\left[X_1\right]\ne 0$. Take a geometric random variable $N_p$ with parameter $p\in (0,1)$, defined as follows:

$$\mathbb{P}(N_p=k)=p{(1-p)}^k,k\in \mathbb{N}\cup \left\{0\right\}.$$

(2)

Assume that $N_p$ and ${\left(X_n\right)}_{n\in \mathbb{N}}$ are independent. Set $S_0:=0$, $S_n:=X_1+\dots +X_n$, $n\in \mathbb{N}$. Then,

$$W_p:=\frac{S_{N_p}}{\mathbb{E}\left[S_{N_p}\right]}\stackrel{\mathcal{D}}{\to }Z\sim Exp\left(1\right)asp\to 0+,$$

(3)

where $\stackrel{\mathcal{D}}{\to }$ stands for convergence in distribution, and Z follows the exponential law $Exp\left(\lambda \right)$ with parameter $\lambda =1$, $\mathbb{E}\left[S_{N_p}\right]=\mu (1-p)/p$. In fact, instead of $N_p$, A.Rényi considered the shifted geometric random variable $N\left(p\right)$ such that $\mathbb{P}(N\left(p\right)=k)=p{(1-p)}^{k-1}$, $k\in \mathbb{N}$. Clearly, $N_p$ has the same law as $N\left(p\right)-1$. He supposed that i.i.d. random variables $X_1,X_2,\dots$ are non-negative, and $N\left(p\right)$ and ${\left(X_n\right)}_{n\in \mathbb{N}}$ are independent. Then, $S_{N\left(p\right)}/\mathbb{E}\left[S_{N\left(p\right)}\right]$ converges in distribution to $Z\sim Exp\left(1\right)$ as $p\to 0+$, where $\mathbb{E}\left[S_{N\left(p\right)}\right]=\mu /p$. It was explained in [22] that both statements are equivalent and the assumption of nonnegativity of summands can be omitted.

Building on the previous investigations discussed below in this section, we study different instances of quantifying the approximation of random sums by limit laws and also extend the Stein method employment. The main goals of our paper are the following: (1) to find sharp estimates (i.e., optimal ones which cannot be diminished) of proximity of geometric sums of independent (in general non-identically distributed) random variables to exponential law using the probability metric $d_{{\mathcal{H}}_2}$; (2) to prove the new version of the Rényi theorem when the summands are described by a model of exchangeable random variables, establishing the due non-exponential limit law together with an optimal bound of the convergence rate applying $d_{{\mathcal{H}}_2}$; (3) to obtain the exact convergence rate of appropriately normalized random sums of random summands to the generalized gamma distribution when the number of summands follows the generalized negative binomial distribution employing $d_{{\mathcal{H}}_2}$; (4) to introduce the inverse transformation to an “equilibrium distribution transformation”, give full description of its existence and demonstrate the advantage of applying the Stein method combined with that inverse transform; and (5) to use such approach in deriving the new approximation in the Kolmogorov metric $d_{\mathcal{K}}$ of the Pareto distribution by an exponential one, which is important in signal processing.

The main idea is to apply the Stein method and deduce (Lemma 2) new estimates of the solution of Stein’s equation (corresponding to an exponential law $Exp\left(\lambda \right)$ as a target distribution) when a function h appearing in its right-hand side belongs to a class ${\mathcal{H}}_2$. This entails the established sharp estimates. The integral probability metrics and the techniques of integration with respect to sign measures are essentially employed. It should be stressed that we consider random summands which take, in general, positive and negative values and in certain cases need not have the same law.

Now, we briefly comment on the relevance of the five groups of the paper results mentioned above. Some upper bounds for convergence rates in Equation (3) were obtained previously by different tools (the renewal techniques and the memoryless property of the geometric distribution), and the estimates were not sharp. We refer to the results by A.D. Soloviev, V.V. Kalashnikov and S.Y. Vsekhsvyatskii, M. Brown, V.M. Kruglov and V.Yu. Korolev, where the authors either used the Kolmogorov distance or proved specified nonuniform estimates for differences of the corresponding distribution functions. For instance, in [23] the following estimate was proved

$$\underset{x\in \mathbb{R}}{sup}|\mathbb{P}(W_p\le x)-\mathbb{P}(Z\le x)|\le p\frac{\mathbb{E}\left[X_1^2\right]}{{\mu }^2}max\left\{1,\frac{1}{2(1-p)}\right\},$$

where $Z\sim Exp\left(1\right)$. Moreover, this estimate is asymptotically exact when $p\to 0+$. Some improvements are in [24] under certain (hazard rate) assumptions. E.V. Sugakova obtained a version of the Rényi theorem for independent, in general, not identically distributed random variables. We also mention contributions by V.V. Kalashnikov, E.F. Peköz, A. Röllin, N. Ross and T.L. Hung which gave the estimates in terms of the Zolotarev ideal metrics. We do not reproduce all these results here since they can be viewed on pages 3 and 4 of [22] with references where they were published.

In Corollary 3.6 of [25] for nondegenerate i.i.d. positive random variables $X_1,X_2,\dots$ with mean $\mu$ and finite second moment, it was proved that

$${\zeta }_2\left(pS\left(p\right),Z(1/\mu )\right)\le p(\mathbb{E}\left[X_1^2\right]+2{\mu }^2),$$

where $S\left(p\right):={\sum }_{j=1}^{N\left(p\right)}{X}_j$, ${\zeta }_2$ is the Zolotarev ideal metric of order two, $Z\left(\lambda \right)\sim Exp\left(\lambda \right)$, $\lambda >0$. In [22], the estimates for proximity of geometric sums distributions to $Z\sim Exp\left(1\right)$ were provided in the Kantorovich and ${\zeta }_2$ metrics. A substantial contribution of the authors of [22] is the study of random summands $X_1,X_2,\dots$ that need not be positive (see also [26]). The general estimate for deviation of $W_p$ from $Z\sim Exp\left(1\right)$ in the ideal metric of order s was proved in [27]. We do not assume that $W_p$ is constructed by means of i.i.d. random variables and, moreover, demonstrate that our estimate (for summands taking real values) involving the metric $d_{{\mathcal{H}}_2}$ is sharp.

The exchangeable random variables form an important class having various applications in statistics and combinatorics, see, e.g., [28]. As far as we know, the model of exchangeable random variables is studied in the context of random sums for the first time here. It is interesting that instead of the exponential limit law we indicate explicit expression of the new limit law. In addition, we establish the sharp estimate of proximity of random sums distributions to this law using $d_{{\mathcal{H}}_2}$.

A natural generalization of the Rényi theorem is to study the summation index following non-geometrical distribution. In this way, the upper bound of the convergence rate of random sums of random summands to generalized gamma distribution was proved in [29]. Theorem 3.1 in [30] contains the estimates in the Kolmogorov and Kantorovich distances for approximations of non-negative random variable law by specified (nongeneralized) gamma distribution. The proof relies on Stein’s identity for gamma distribution established in H.M.Luk’s PhD thesis (see the reference in [30]). New estimates of the solutions of the gamma Stein equation are given in [31]. We derive the sharp estimate for approximation of random sums by generalized gamma law using the Zolotarev metric of order two. In a quite recent paper [32] the author established deep results concerning further generalizations of the Rényi theorem. Namely, Theorem 1 of [32] demonstrates how one can provide the upper bounds of the convergence rate of specified random sums to a more general law than an exponential one using the estimates in the Rényi theorem. This approach is appealing since the author employs the ideal metric of order $s>0$. However, the sharpness of these estimates was not examined.

Note that in [33] the important “equilibrium transformation of distributions” was proposed and employed along with the Stein techniques. We will consider this transformation $X^e$ for a random variable X in Section 7 and also tackle other useful transformations. In the present paper, the inverse to the “equilibrium distribution transformation” is introduced. We completely describe the possibility to construct such transformation and provide an explicit formula for the corresponding density. The idea to apply such inverse transformation whenever it exists is based on the result [33] demonstrating that one can obtain a more precise estimate for proximity in the Kantorovich metric between $X^e$ and Z than between X and Z, where $Z\sim Exp\left(1\right)$ and $\mathbb{E}\left[X\right]=1$, $\mathbb{E}\left[X^2\right]<\infty$. We extend this result. Moreover, we prove that in this way one can obtain a new estimate of approximation of the Pareto distribution by an exponential one. It is shown that our new estimate is advantageous for a wide range of parameters of the Pareto distribution. Let $X^e\sim Pareto(\alpha ,\beta )$, i.e., the distribution function of $X^e$ is

$$F^e\left(x\right)=1-{\left(\frac{\beta }{x+\beta }\right)}^{\alpha },x\ge 0,\alpha >0,\beta >0.$$

We show that the preimage $X\sim Pareto(\alpha +1,\beta )$. Thus, for any $\alpha >2$, $\beta >0$, one has $d_{\mathcal{K}}(X^e,Z)\le 1/(\alpha -1),$ where $Z\sim Exp(\alpha /\beta )$ and $d_{\mathcal{K}}$ stands for the Kolmogorov distance. This bound is more precise than the previous ones applied in signal processing, see, e.g., [34].

This paper is organized as follows. After the Introduction, the auxiliary results are provided in Section 2. Here we include the material important for understanding the main results. We recall the concept of probability metrics, consider the Kolmogorov and the Kantorovich distances and examine the Zolotarev ideal metrics. We describe the basic ideas of Stein’s method, especially for the exponential target distribution. In this section, we formulate a simple but useful Lemma 1 concerning the essential supremum of the Lipschitz function, an important Lemma 2 giving the solution of the Stein equation for different functional classes. We explain the essential role of the generalized equilibrium transformation proposed in [22] which permits study of the summands taking both positive and negative values. We formulate Lemma 3 to be able to solve an integral equation involving the generalized equilibrium transformation when $\mathbb{E}\left[X\right]\ne 0$ and $\mathbb{E}\left[X^2\right]<\infty$. The proofs of auxiliary lemmas are placed in Appendix A. Section 3 is devoted to an approximation of the normalized geometric sums $W_p$ by an exponential law. Here, the sharp convergence rate is found (see Theorem 1) by means of the probability metric $d_{{\mathcal{H}}_2}$. The proof is based on the Lebesgue–Stieltjes integration techniques, the formula of integration by parts for functions of bounded variations, Lemma 2, various limit theorems for integrals and the important result of [22] concerning the estimates involving the Kantorovich distance. In Section 4, for the first time an analog of the Rényi theorem is proved for a model of exchangeable random variables proposed in [35]. We demonstrate (Theorem 2) that, in contrast to Rényi’s theorem, the limit distribution for random sums under consideration is a specified mixture of two explicitly indicated laws. Moreover, the sharp convergence rate to this limit law is obtained (Theorem 3) by means of $d_{{\mathcal{H}}_2}$. In Section 5, the distance between the generalized gamma law and the suitably normalized sum of independent random variables is estimated when the number of summands has the generalized negative binomial distribution. Theorem 4 demonstrates that this estimate is sharp. For the proof, we employ various truncation techniques, the transformations of parameters of initial random variables, the monotone convergence theorem and explicit formula for the generalized gamma distribution moments of order $\delta >0$, obtained in [27]. Section 6 provides the pioneering study of the same problem in the framework of exchangeable random variables and also gives the sharp estimate for the $d_{{\mathcal{H}}_2}$ metric (Theorem 5). In Section 7, we introduce the inverse to the equilibrium transformation of the probability measures. Lemma 6 contains a full description of situations when a unique preimage X of a random variable $X^e$ exists and gives an explicit formula for distribution of X. This approach permits us to obtain the new estimates of closeness of probability measures in the Kolmogorov and Kantorovich metrics (Theorem 6). In particular, due to Theorem 6 and Lemmas 2, 6, it becomes possible to find a useful estimate of proximity of the Pareto law to the exponential one (Example 2). Section 8 containing the conclusions and indications for further research work is followed by Appendix A and the list of references.

2. Auxiliary Results

Let $\mathcal{K}:=\{h:h_z\left(x\right)=\mathbb{I}\{x\le z\},x,z\in \mathbb{R}\},$ where $\mathbb{I}\left\{A\right\}:=1$ if A holds and zero otherwise. The choice $\mathcal{H}=\mathcal{K}$ in Equation (1) corresponds to the Kolmogorov distance. Note that h above is a function in x, whereas z is the index parameterizing the class.

A function $h:\mathbb{R}\to \mathbb{R}$ is called the Lipschitz one if

$$Lip\left(h\right):=\underset{x,u\in \mathbb{R};x\ne u}{sup}\frac{|h(x)-h(u)|}{|x-u|}<\infty .$$

(4)

Then,

$$|h(x)-h(u)|\le C|x-u|,x,u\in \mathbb{R},$$

(5)

and in light of Equation (4), $Lip\left(h\right)$ is the smallest possible constant C appearing in Equation (5). We write $\mathsf{Lip}\left(C\right)$, where $C\in [0,\infty )$ for a collection of the Lipschitz functions having $Lip\left(h\right)\le C$. For $s>0$ set $m=m\left(s\right):=\lceil s-1\rceil \in \mathbb{N}\cup \left\{0\right\}$ (where, for $a\in \mathbb{R}$, $\lceil a\rceil$ stands for the minimal integer number which is equal or greater than a). Introduce a class of functions

$${\mathcal{H}}_s:=\{h:\mathbb{R}\to \mathbb{R},|h^{\left(m\right)}\left(x\right)-h^{\left(m\right)}{\left(u\right)|\le |x-u|}^{s-m},x,u\in \mathbb{R}\},s>0.$$

As usual, $h^{\left(0\right)}\left(x\right)=h\left(x\right)$, $x\in \mathbb{R}$. We write $d_{{\mathcal{H}}_s}$ for a metric defined according to Equation (1) with $\mathcal{H}={\mathcal{H}}_s$. V.M. Zolotarev and many other researchers defined an ideal metric ${\zeta }_s$ of order $s>0$ involving only bounded functions from ${\mathcal{H}}_s$. We will use collections ${\mathcal{H}}_1$ and ${\mathcal{H}}_2$ without assumption that functions h are bounded on $\mathbb{R}$. This is the reason why we write $d_{{\mathcal{H}}_s}$ instead of ${\zeta }_s$. Thus, we employ

$${\mathcal{H}}_1:=\left\{\mathsf{Lip}\left(1\right)\right\},{\mathcal{H}}_2:=\{h:h^{\prime }\in \mathsf{Lip}\left(1\right)\}.$$

Note that in definitions of ${\mathcal{H}}_2$ we deal with $h\in C^{\left(1\right)}$, where the space $C^{\left(1\right)}\left(\mathbb{R}\right)$ consists of functions $h:\mathbb{R}\to \mathbb{R}$ such that $h^{\prime }\left(x\right)$ exists for all $x\in \mathbb{R}$, and $h^{\prime }$ is continuous on $\mathbb{R}$ (evidently the Lipschitz function is continuous). One calls $d_{{\mathcal{H}}_1}$ the Kantorovich metric (the term Wasserstein metric appears in the literature as well). One also uses the bounded Kantorovich metric when the class ${\mathcal{H}}_1$ contains all the bounded functions from $\mathsf{Lip}\left(1\right)$. The metric ${\zeta }_s$ was introduced in [36] and called an ideal metric in light of its important properties. The properties of ${\zeta }_s$ metrics, where $s>0$, are collected in Sec. 2 of [32]. We mention in passing that various functionals are ubiquitous in assessing the proximity of distributions. In this regard, we refer, e.g., to [37,38].

To apply the Stein method, we begin with fixing the target random variable Z (or its distribution) and describe a class $\mathcal{H}$ to estimate $d_{\mathcal{H}}(Y,Z)$ for a random variable Y under consideration. Then, the problem is to indicate an operator T (with specified domain of definition) so that the Stein equation

$$Tf\left(x\right)=h\left(x\right)-\mathbb{E}\left[h\right(Z\left)\right]$$

(6)

has a solution $f_h\left(x\right)$, $x\in \mathbb{R}$, for each function $h\in \mathcal{H}$. After that, one can substitute Y instead of x in Equation (6) and take the expectation of both sides, assuming that all these expectations are finite. As a result, one comes to the relation

$$\mathbb{E}\left[T{f}_h\left(Y\right)\right]=\mathbb{E}\left[h\left(Y\right)\right]-\mathbb{E}\left[h\left(Z\right)\right].$$

(7)

It is not a priori clear why the estimation of the left-hand side of Equation (7) is more adequate than the estimation of $|\mathbb{E}[h\left(Y\right)]-\mathbb{E}[h\left(Z\right)]|$ for $h\in \mathcal{H}$. However, in many situations, justifying the method this occurs. The choice of T depends on the distribution of Z. Note that in certain cases (e.g., when Z follows the Poisson law) one considers functions f defined on a subset of $\mathbb{R}$. We emphasize that the construction of operator T is a nontrivial problem, see, e.g., [33,39,40,41].

The basic idea in this way is the following. For many probability distributions (Gaussian, Laplace, Exponential, etc.), one can find an operator T characterizing the law of a target variable Z. In other words, for a rather large class of functions f, $\mathbb{E}\left[Tf\right(Y\left)\right]=0$ if and only if $law\left(Y\right)=law\left(Z\right)$ (i.e., the laws of Y and Z coincide). Thus, if $|\mathbb{E}[T{f}_h\left(Y\right)]|$ is small enough for a suitable class of functions h, this leads to the assertion that the law of Y is close (in a sense) to the law of Z. One has to verify that this kind of “continuity” takes place. Clearly, if for any $h\in \mathcal{H}$, where $\mathcal{H}$ defines the integral probability metric in Equation (1), one can find a solution $f_h$ of Equation (6), then the relation $\mathbb{E}\left[T{f}_h\left(Y\right)\right]=0$ for all $f_h$, $h\in \mathcal{H}$, yields $d_{\mathcal{H}}(Y,Z)=0$ and, consequently, $law\left(Y\right)=law\left(Z\right)$.

Further, we assume that $Z\sim Exp\left(\lambda \right)$, i.e., Z has exponential distribution with parameter $\lambda >0$. In this case (see, e.g., Sec. 5 in [17]), one uses the operator

$$Tf\left(x\right):=f^{\prime }\left(x\right)-\lambda f\left(x\right)+\lambda f\left(0\right),x\in \mathbb{R},\lambda >0,$$

(8)

and writes the Stein Equation (6) as follows

$$f^{\prime }\left(x\right)-\lambda f\left(x\right)+\lambda f\left(0\right)=h\left(x\right)-\mathbb{E}\left[h\left(Z\right)\right],x\in \mathbb{R}.$$

(9)

It should be stipulated that $\mathbb{E}\left[h\right(Z\left)\right]\in \mathbb{R}$ for a test function $h\in \mathcal{H}$, and there exists a differentiable solution f of Equation (9). Therefore, if one can find such solution f, then

$$\mathbb{E}\left[f^{\prime }\left(Y\right)\right]-\lambda \mathbb{E}\left[f\left(Y\right)\right]+\lambda f\left(0\right)=\mathbb{E}\left[h\left(Y\right)\right]-\mathbb{E}\left[h\left(Z\right)\right]$$

(10)

under the hypothesis that all these expectations are finite. If $f:\mathbb{R}\to \mathbb{R}$ is absolutely continuous, then (see, e.g., Theorem 13.18 of [42]) for almost all $x\in \mathbb{R}$ with respect to the Lebesgue measure, there exists $f^{\prime }\left(x\right)$. Moreover, one can find an integrable (on each interval) function $g:\mathbb{R}\to \mathbb{R}$, $x\in \mathbb{R}$, to guarantee, for each $x,u\in \mathbb{R}$, that

$$f\left(x\right)=f\left(u\right)+{\int }_u^{x}g\left(v\right)dv,$$

(11)

where $g\left(v\right)=f^{\prime }\left(v\right)$ for almost all $v\in \mathbb{R}$. Thus, $\left(Tf\right)\left(x\right)$ is defined for such f according to Equation (8) for almost all $x\in \mathbb{R}$. In general, for an arbitrary random variable Y, one cannot write $\mathbb{E}\left[\right(Tf\left)\right(Y\left)\right]$ since the value of expectation depends on the choice of a version of $\left(Tf\right)\left(x\right)$, $x\in \mathbb{R}$. Really, let $B\in \mathcal{B}\left(\mathbb{R}\right)$ be such that $m\left(B\right)=0$, where m stands for the Lebesgue measure. Assume that Y takes values in B. Then, it is clear that $\mathbb{E}\left[\right(Tf\left)\right(Y\left)\right]$ depends on the choice of a function $\left(Tf\right)\left(x\right)$ version defined on $\mathbb{R}$. However, if the distribution ${\mathbb{P}}_Y$ of a random variable Y has a density with respect to m, then $\mathbb{E}\left[\right(Tf\left)\right(Y\left)\right]$ will be the same for any version of $Tf$ (with respect to the Lebesgue measure). In certain cases, the Stein operator is applied to smoothed functions (see, e.g., [33,43]). Otherwise, Equation (6) does not hold at each point of $\mathbb{R}$ (see, e.g., Lemma 2.2 in [16]), and complementary efforts are needed. For our study, it is convenient to employ in Equation (8) for T in the capacity of $f^{\prime }\left(x\right)$, $x\in \mathbb{R}$, the right derivative. In many cases, for a real-valued function f defined on a fixed set $D\subset \mathbb{R}$ one considers ${sup}_{x\in D}|f\left(x\right)|$ as "essential supremum". Recall that a function $\tilde{f}$ is a version of f (and vice versa) if the measure (here the Lebesgue measure) of points x such that $\tilde{f}\left(x\right)\ne f\left(x\right)$ is zero. The notation ${\parallel f\parallel }_{\infty }$ means that one takes ${inf}_{\tilde{f}}{sup}_{x\in D}|\tilde{f}\left(x\right)|$, where $\tilde{f}$ belongs to the class of all versions of f. Clearly, ${\parallel f\parallel }_{\infty }$ will be the same if we change f on a subset of D having a measure which is equal to zero. Thus, we write $\parallel f^{\prime }{\parallel }_{\infty }$ instead of ${\parallel g\parallel }_{\infty }$ appearing in Equation (11). The following simple observation is useful. Its proof is provided in Appendix A.

Lemma 1.

A function h is the Lipschitz function on $\mathbb{R}$ with $Lip\left(h\right)=C<\infty$ if and only if h is absolutely continuous and (its essential supremum)$\parallel h^{\prime }{\parallel }_{\infty }=C<\infty$.

Remark 1.

Note that $0\le h\left(x\right)\le 1$, $x\in \mathbb{R}$, for any $h\in \mathcal{K}$. If, for some positive constant C, $h\in \mathsf{Lip}\left(C\right)$, then Equation (5) yields that $|h(x)|\le C|x|+|h(0)|$. If $h^{\prime }$ is a Lipschitz function (with $Lip\left(h^{\prime }\right)=C$), then $h^{″}\left(x\right)$ exists for almost all $x\in \mathbb{R}$ and an application of Lemma 1 gives

$$|h^{\prime }\left(x\right)-h^{\prime }\left(0\right)|=\left|{\int }_0^x{h}^{″}\left(u\right)du\right|\le C|x|,x\in \mathbb{R}.$$

Consequently, $|h^{\prime }\left(x\right)|\le A|x|+B$ for some positive A, B (one can take $A=C$, $B=|h^{\prime }\left(0\right)|$) and any $x\in \mathbb{R}$. As $h^{\prime }\left(x\right)$ is continuous on each interval, it follows that $|h\left(x\right)|\le a{x}^2+b|x|+c$ for some positive $a,b,c$ and all $x\in \mathbb{R}$ ($a=C/2$, $b=|h^{\prime }\left(0\right)|$, $c=|h(0)|$). Therefore, $|h\left(x\right)|\le A_0{x}^2+B_0$ for some positive $A_0,B_0$ and each $x\in \mathbb{R}$.

Lemma 2.

For any $\lambda >0$ and each $h\in \mathcal{K}\cup {\mathcal{H}}_1\cup {\mathcal{H}}_2$, the equation

$$f^{\prime }\left(x\right)-\lambda f\left(x\right)=h\left(x\right),x\in \mathbb{R},$$

(12)

has a solution

$$f_h\left(x\right)=-e^{\lambda x}{\int }_x^{\infty }h\left(u\right)e^{-\lambda u}du,x\in \mathbb{R},$$

(13)

where $f_h\left(0\right)=-\mathbb{E}\left[h\left(Z\right)\right]/\lambda$. If $h\in \mathcal{K}$, then for all $x\in \mathbb{R}$ there exists $f_h^{\prime }\left(x\right)$ and $\parallel f_h^{\prime }{\parallel }_{\infty }\le 1$. If $h\in {\mathcal{H}}_1\cup {\mathcal{H}}_2$, then $f_h^{\prime }$ is defined on $\mathbb{R}$ and $\parallel f_h^{\prime }{\parallel }_{\infty }\le {\parallel h^{\prime }\parallel }_{\infty }/\lambda$. For $h\in {\mathcal{H}}_2$, a function $f_h^{″}$ is defined on $\mathbb{R}$ and $\parallel f_h^{″}{\parallel }_{\infty }\le min\{2\parallel h^{\prime }{\parallel }_{\infty },\parallel h^{″}{\parallel }_{\infty }/\lambda \}$.

The right-hand side of Equation (13) is well defined for each $x\in \mathbb{R}$ in light of Remark 1. Lemma 4.1 of [33] contains for $\lambda =1$ some statements of Lemma 1. We will use the above estimates for any $\lambda >0$. Estimates for $h\in {\mathcal{H}}_2$ were not considered in [33]. The proof of Lemma 2 is given in Appendix A.

The following concept was introduced in [33].

Definition 1

([33]). Let X be a non-negative random variable with finite $\mathbb{E}\left[X\right]>0$. One says that a random variable $X^e$ has distribution of equilibrium with respect to X if for any Lipschitz function $f:\mathbb{R}\to \mathbb{R}$,

$$\mathbb{E}\left[f\left(X\right)\right]-f\left(0\right)=\mathbb{E}\left[X\right]\mathbb{E}\left[f^{\prime }\left(X^e\right)\right].$$

(14)

Note that Definition 1 deals separately with distributions of X and $X^e$. One says that $X^e$ is the result of the equilibrium transformation applied to X. The same terminology is used for transition from $law\left(X\right)$ to $law\left(X^e\right)$. For the sake of completeness, we explain in Appendix A (Comments to Definition 1) why one can take the law of $X^e$ having a density with respect to the Lebesgue measure

$$p^e\left(x\right)=\left\{\begin{array}{cc}\frac{1}{\mathbb{E}\left[X\right]}\mathbb{P}(X>x),\hfill & x\ge 0,\hfill \\ 0,\hfill & x<0,\hfill \end{array}\right.$$

(15)

to guarantee the validity of Equation (14).

Remark 2.

For a non-negative random variable X with finite $\mathbb{E}\left[X\right]>0$, one can construct a random variable $X^e$ having a density (15). Accordingly, we then have a random vector $(X,X^e)$ with specified marginal distributions. However, the joint law of X and $X^e$ is not fixed and can be chosen in appropriate way. If $X_1,X_2,\dots$ is a sequence of independent random variables, we will assume that a sequence ${(X_n,X_n^e)}_{n\in \mathbb{N}}$ consists of independent vectors, and these vectors are independent with all considered random variables which are independent with ${\left(X_n\right)}_{n\in \mathbb{N}}$.

In the recent paper [22], a generalization of the equilibrium transformation of distributions was proposed without assuming that random variable X is non-negative.

Definition 2

([22]). Let X be a random variable having a distribution function $F\left(x\right):=\mathbb{P}(X\le x)$, $x\in \mathbb{R}$. Assume the existence of finite $\mathbb{E}\left[X\right]\ne 0$. An equilibrium distribution function corresponding to X (or $F\left(x\right)$) is introduced by way of

$$F^e\left(x\right):=\left\{\begin{array}{cc}-\frac{1}{\mathbb{E}\left[X\right]}{\int }_{-\infty }^{x}F\left(u\right)du,\hfill & x\le 0,\hfill \\ -\frac{\mathbb{E}\left[X^{-}\right]}{\mathbb{E}\left[X\right]}+\frac{1}{\mathbb{E}\left[X\right]}{\int }_0^x(1-F\left(u\right))du,\hfill & x>0,\hfill \end{array}\right.$$

(16)

where $X^{-}:=X\mathbb{I}\{X<0\}$. This function can be written as $F^e\left(x\right)$$={\int }_{-\infty }^x{p}^e\left(u\right)du$, where

$$p^e\left(x\right)=\left\{\begin{array}{cc}-\frac{1}{\mathbb{E}\left[X\right]}F\left(x\right),\hfill & x\le 0,\hfill \\ \frac{1}{\mathbb{E}\left[X\right]}(1-F\left(x\right)),\hfill & x>0,\hfill \end{array}\right.$$

(17)

thus, $p^e$ is a density (with respect to the Lebesgue measure) of a signed measure $Q^e$ corresponding to $F^e$. In other words, Equation (17) demonstrates the Jordan decomposition (see, e.g., Sec. 29 of [44]) of $Q^e$.

Clearly, for a non-negative random variable, the functions defined in Equation (15) and Equation (16) coincide. For a nonpositive random variable, the function $F^e$ appearing in Equation (16) is a distribution function of a probability measure. In general, when X can take positive and negative values, the function introduced in Equation (16) is not a distribution function. We will call $F^e$ the generalized equilibrium distribution function. Note that $|p^e\left(x\right)|\le \frac{1}{|\mathbb{E}[X]|}$. Thus, $F^e$ is the Lipschitz function and consequently continuous ($F^e\left(x\right)$ is well defined for each $x\in \mathbb{R}$ since $\mathbb{E}\left[X\right]$ is finite and nonzero). Moreover, $F^e$ is absolutely continuous being the Lipschitz function. Each absolutely continuous function has bounded variation. If G is a function of bounded variation, then $G=G_1-G_2$, where $G_1$ and $G_2$ are nondecreasing functions (see, e.g., [42], Theorem 12.18). One can employ the canonical choice $G_1\left(x\right):=Va{r}_0^x\left(G\right)$, where $Va{r}_a^b\left(G\right)$ means the variation of G on $[a,b]$, $-\infty <a\le b<\infty$ (if $a>b$ then $Va{r}_a^b\left(G\right):=-Va{r}_b^a\left(G\right)$). If G is right-continuous (on $\mathbb{R}$), then evidently $G_1$ and $G_2$ are also right-continuous. Thus, for a right-continuous G having bounded variation, a nondecreasing function $G_i$ in its representation corresponds to a $\sigma$-finite measure $Q_i$ on $\mathcal{B}\left(\mathbb{R}\right)$, $i=1,2$. More precisely, there exists a unique $\sigma$-finite measure $Q_i$ on $\mathcal{B}\left(\mathbb{R}\right)$ such that, for each finite interval $(a,b]$, $Q_i\left((a,b]\right)=G_i\left(b\right)-G_i\left(a\right)$, $i=1,2$. Recall that one writes for the Lebesgue–Stieltjes integral with respect to a function G

$${\int }_{\mathbb{R}}f\left(u\right)dG\left(u\right):={\int }_{\mathbb{R}}f\left(u\right)d{G}_1\left(u\right)-{\int }_{\mathbb{R}}f\left(u\right)d{G}_2\left(u\right),$$

(18)

whenever the integrals in the right-hand side exist (with values in $[-\infty ,\infty ]$), and the cases $\infty -\infty$ or $-\infty +\infty$ are excluded. The integral ${\int }_{\mathbb{R}}f\left(u\right)d{G}_i\left(u\right)$ means the integration with respect to measure $Q_i$, $i=1,2$. The signed measure Q corresponding to G is $Q_1-Q_2$. Thus, ${\int }_{\mathbb{R}}f\left(u\right)dG\left(u\right)$ means the integration with respect to signed measure Q. Note that if $G=U_1-U_2$ where $U_i$ is right-continuous and nondecreasing ($i=1,2$), then

$${\int }_{\mathbb{R}}f\left(u\right)d{G}_1\left(u\right)-{\int }_{\mathbb{R}}f\left(u\right)d{G}_2\left(u\right)={\int }_{\mathbb{R}}f\left(u\right)d{U}_1\left(u\right)-{\int }_{\mathbb{R}}f\left(u\right)d{U}_2\left(u\right).$$

(19)

The left-hand side and the right-hand side of Equation (19) make sense simultaneously, and if so, are equal to each other. Indeed, for any finite interval $(a,b]$ ($a\le b$), one has $G_1\left(b\right)-G_1\left(a\right)-(G_2\left(b\right)-G_2\left(a\right))=U_1\left(b\right)-U_1\left(a\right)-(U_2\left(b\right)-U_2\left(a\right))$. Thus, the signed measures corresponding to $G_1-G_2$ and $U_1-U_2$ coincide on $\mathcal{B}\left(\mathbb{R}\right)$. We mention in passing that one can also employ the Jordan decomposition of a signed measure.

For $F^e$ introduced in Equation (16), the analog of Equation (15) has the form

$$\mathbb{E}\left[f\left(X\right)\right]-f\left(0\right)=\mathbb{E}\left[X\right]{\int }_{\mathbb{R}}{f}^{\prime }\left(x\right)d{F}^e\left(x\right).$$

(20)

Taking into account Equation (17), one can rewrite Equation (20) equivalently as follows

$$\mathbb{E}\left[f\left(X\right)\right]-f\left(0\right)={\int }_{(-\infty ,0]}{f}^{\prime }\left(x\right)(-F\left(x\right))dx+{\int }_{(0,\infty )}{f}^{\prime }\left(x\right)(1-F\left(x\right))dx.$$

(21)

The right-hand side of the latter relation does not depend on the choice of a version of $f^{\prime }$. Due to Theorem 1(d) of [22], Equation (20) is valid for any Lipschitz function f. Evidently, an arbitrary function $f\in {\mathcal{H}}_2$ need not be the Lipschitz one and vice versa.

Lemma 3.

Let X be a random variable such that $\mathbb{E}\left[X^2\right]<\infty$ and $\mathbb{E}\left[X\right]\ne 0$. Then, Equation (20) is satisfied for all $f\in {\mathcal{H}}_2$.

The proof is provided in Appendix A.

3. Limit Theorem for Geometric Sums of Independent Random Variables

Consider $N_p\sim Geom\left(p\right)$, see Equation (2). In other words, $N_p$ has a geometric distribution with parameter p. Let $X_1,X_2,\dots$ be a sequence of independent random variables such that $\mathbb{E}\left[X_k\right]=\mu$, where $\mu \in \mathbb{R}$, $\mu \ne 0$, $k\in \mathbb{N}$. Assume that $N_p$ and ${\left(X_n\right)}_{n\in \mathbb{N}}$ are independent. Consider a normalized geometric sum

$$W_p:=\frac{p}{\mu (1-p)}\sum _{k=1}^{N_p}{X}_k,$$

(22)

introduced in Equation (3). Since $N_p$ can take zero value, set, as usual, ${\sum }_{k=1}^0{X}_k:=0$. One can see that $W_p$ can be viewed as a random sum $S_p:={\sum }_{k=1}^{N_p}{X}_k$ normalized by $\mathbb{E}\left[X\right]\mathbb{E}\left[N_p\right]$.

Lemma 4.

Let $X_1,X_2,\dots$ and $N_p$, where $p\in (0,1)$, be random variables described above in this Section. Then, the following relations hold:

$$\mathbb{E}\left[W_p\right]=1,\mathbb{E}|W_p|\le \frac{{sup}_{k\in \mathbb{N}}\mathbb{E}|X_k|}{|\mu |},$$

$$\mathbb{E}\left[W_p^2\right]=\frac{p}{{\mu }^2(1-p)}\mathbb{E}\left[X_{N_p+1}^2\right]+2.$$

(23)

Proof. 

Recall that

$$\mathbb{E}\left[N_p\right]=\sum _{k=1}^{\infty }kp{(1-p)}^{k-1}=\frac{1-p}{p},$$

(24)

$$\mathbb{E}\left[N_p^2\right]=\sum _{k=1}^{\infty }{k}^{2}p{(1-p)}^{k-1}=\frac{(1-p)(2-p)}{p^2}.$$

(25)

Thus, one has

$$\mathbb{E}\left[W_p\right]=\frac{p}{\mu (1-p)}\sum _{k=1}^{\infty }k\mu \mathbb{P}(N_p=k)=\frac{p}{1-p}\mathbb{E}\left[N_p\right]=1.$$

Clearly, $\mathbb{E}|X_k|<\infty$ since $\mathbb{E}\left[X_k\right]$ is finite $(k\in \mathbb{N}$). Therefore

$$\mathbb{E}|W_p|\le \frac{p}{|\mu |(1-p)}\sum _{k=1}^{\infty }k\mathbb{E}|X_k|\mathbb{P}(N_p=k)\le \frac{{sup}_{k\in \mathbb{N}}\mathbb{E}|X_k|}{|\mu |}.$$

Set ${\nu }_k:=\mathbb{E}\left[X_k^2\right]$, $k\in \mathbb{N}$. One has

$$\mathbb{E}\left[S_p^2\right]=\sum _{k=1}^{\infty }\mathbb{P}(N_p=k)\mathbb{E}{\left(\sum _{i=1}^k{X}_i\right)}^2=\sum _{k=1}^{\infty }p{(1-p)}^k\left(\sum _{i=1}^k{\nu }_i+k(k-1){\mu }^2\right).$$

(26)

According to Equations (24) and (25) one derives the formula

$$\sum _{k=1}^{\infty }p{(1-p)}^k\left(k(k-1){\mu }^2\right)={\mu }^2\left(\frac{(1-p)(2-p)}{p^2}-\frac{1-p}{p}\right)=2{\left(\frac{\mu (1-p)}{p}\right)}^2.$$

(27)

Convergence of the series ${\sum }_{k=1}^{\infty }p{(1-p)}^k{\sum }_{i=1}^k{\nu }_i$ having non-negative terms holds simultaneously with the validity of inequality $\mathbb{E}\left[W_p^2\right]<\infty$. Changing the order of summation, we obtain

$$\sum _{k=1}^{\infty }p{(1-p)}^k\sum _{i=1}^k{\nu }_i=\sum _{i=1}^{\infty }{(1-p)}^i{\nu }_i=\left(\frac{1-p}{p}\right)\mathbb{E}\left[X_{N_p+1}^2\right].$$

The latter formula and Equations (26), (27) yield

$$\mathbb{E}\left[W_p^2\right]={\left(\frac{p}{\mu (1-p)}\right)}^2\mathbb{E}\left[S_p^2\right]={\left(\frac{p}{\mu (1-p)}\right)}^2\left(\left(\frac{1-p}{p}\right)\mathbb{E}\left[X_{N_p+1}^2\right]+2{\left(\frac{\mu (1-p)}{p}\right)}^2\right)$$

$$=\frac{p}{{\mu }^2(1-p)}\mathbb{E}\left[X_{N_p+1}^2\right]+2.$$

Equation (23) is established. □

The proof of Theorem 3.1 in [45] shows for non-negative i.i.d. random variables $X_1,X_2,\dots$ (when $\mu =1$, see Formula (3.15) in [45]) that the equilibrium transformation of $W_p$ distribution has the following form:

$$W_p^e=\frac{p}{\mu (1-p)}\left(\sum _{k=1}^{N_p}{X}_k+X_{N_p+1}^e\right)=W_p+\frac{p}{\mu (1-p)}{X}_{N_p+1}^e,$$

(28)

where $X_{N_p+1}^e$ means that we construct $X_1^e,X_2^e,\dots$ and then take a random index $N_p+1$. In other words,

$$X_{N_p+1}^e=\sum _{n=0}^{\infty }{X}_{n+1}^e\mathbb{I}\{N_p=n\}.$$

It was explained in Section 2 that a generalized equilibrium distribution function $F_{W_p}^e\left(x\right)$ (see Definition 2) need not be a distribution function when the summands $X_1,X_2,\dots$ can take values of different signs. However, employing this function, one can establish the following result.

Theorem 1.

Let $X_1,X_2,\dots$ be a sequence of independent random variables having finite $\mathbb{E}\left[X_k\right]=\mu$, where $\mu \ne 0$, $k\in \mathbb{N}$. Assume that $N_p$ and ${\left(X_n\right)}_{n\in \mathbb{N}}$ are independent, where $N_p\sim Geom\left(p\right)$, $0<p<1$. If $Z\sim Exp\left(1\right)$, then

$$d_{{\mathcal{H}}_2}(W_p,Z)=\frac{\mathbb{E}\left[X_{N_p+1}^2\right]}{2{\mu }^2}\left(\frac{p}{1-p}\right)$$

(29)

where $W_p$ was introduced in Equation (22).

Proof. 

If $\mathbb{E}\left[W_p^2\right]=\infty$, then $d_{{\mathcal{H}}_2}(W_p,Z)=\infty$ since, for a function $h\left(x\right)=x^2/2$, $x\in \mathbb{R}$, belonging to ${\mathcal{H}}_2$, one has $\mathbb{E}\left[h\left(W_p\right)\right]=\infty$, whereas $\mathbb{E}\left[h\right(Z\left)\right]<\infty$. According to Equation (23), $\mathbb{E}\left[W_p^2\right]$ and $\mathbb{E}\left[X_{N_p+1}^2\right]$ are both finite or infinite simultaneously. Consequently, Equation (29) is true when $\mathbb{E}\left[W_p^2\right]=\infty$.

Let us turn to the case $\mathbb{E}\left[W_p^2\right]<\infty$. At first, we obtain an upper bound for $d_{{\mathcal{H}}_2}(W_p,Z)$. Take $h\in {\mathcal{H}}_2$. Applying Lemmas 1 and 2 and Remark 1, one can write due to Stein’s Equation (10) that

$$|\mathbb{E}\left[h\left(W_p\right)\right]-\mathbb{E}\left[h\left(Z\right)\right]|=|\mathbb{E}\left[f_h^{\prime }\left(W_p\right)\right]-\mathbb{E}\left[f_h\left(W_p\right)\right]+f\left(0\right)|.$$

(30)

Using the generalized equilibrium distribution transformation (20) one obtains:

$$|\mathbb{E}\left[f_h^{\prime }\left(W_p\right)\right]-\mathbb{E}\left[f_h\left(W_p\right)\right]+f\left(0\right)|=\left|{\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d{F}_{W_p}\left(x\right)-{\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d{F}_{W_p}^e\left(x\right)\right|.$$

(31)

Due to Lemma 3 this is true, for $h\in {\mathcal{H}}_2$, because $f_h\in {\mathcal{H}}_2$ according to Lemma 2 (with $\lambda =1$). Next, we employ the relation

$${\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d{F}_{W_p}\left(x\right)-{\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d{F}_{W_p}^e\left(x\right)={\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d(F_{W_p}-F_{W_p}^e)\left(x\right).$$

(32)

Evidently, one can write ${\int }_{\mathbb{R}}|f_h^{\prime }\left(x\right)|d{F}_{W_p}\left(x\right)<\infty$. The notation $d{F}_{W_p}^e\left(x\right)$ in the integral refers to the Lebesgue–Stieltjes integral with respect to a function $F_{W_p}^e\left(x\right)$ of bounded variation. In fact, the integral with integrator $d{F}_{W_p}^e\left(x\right)$ means that integration employs a signed measure $Q_p^{+}-Q_p^{-}$, where $Q_p^{+}$ and $Q_p^{-}$ have the following densities with respect to the Lebesgue measure:

$$q_p^{+}\left(x\right):=(1-F_{W_p}\left(x\right))\mathbb{I}\left\{(0,\infty )\right\},q_p^{-}\left(x\right):=F_{W_p}\left(x\right)\mathbb{I}\left\{(-\infty ,0]\right\},x\in \mathbb{R},$$

we took into account that $\mathbb{E}\left[W_p\right]=1$ according to Lemma 4. Then, for any $-\infty <a<b<\infty$, one ascertains that variation of $F_{W_p}^e$ on $[a,b]$ is given by formula $Va{r}_a^b\left(F_{W_p}^e\right)={\int }_a^b|p_{W_p}^e\left(u\right)|du$ (see, e.g., Theorem 4.4.7 [46]). Note that for any $-\infty <a<b<\infty$,

$${\int }_a^b|p_{W_p}^e\left(u\right)|du\le \mathbb{E}|W_p|<\infty$$

according to Lemma 4. Thus, $F_{W_p}^e$ is a function of bounded variation. In the right-hand side of Equation (32), we take the Lebesgue–Stieltjes integral with respect to the function of bounded variation $(F_{W_p}-F_{W_p}^e)\left(x\right)$, $x\in \mathbb{R}$. Let $F_{W_p}^e\left(x\right)=F_{p,1}^e\left(x\right)-F_{p,2}^e\left(x\right)$, $x\in \mathbb{R}$, where $F_{p,i}^e$ are nondecreasing right-continuous functions (even continuous since $F_{W_p}^e$ is continuous), $i=1,2$. Thus,

$$F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right)=(F_{W_p}\left(x\right)+F_{p,2}^e\left(x\right))-F_{p,1}^e\left(x\right),x\in \mathbb{R}.$$

With the help of Equations (18) and (19) one makes sure that, for each $n\in \mathbb{N}$,

$${\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d(F_{W_p}-F_{W_p}^e)\left(x\right)={\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d(F_{W_p}\left(x\right)+F_{p,2}^e\left(x\right))-{\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d\left(F_{p,1}^e\left(x\right)\right)$$

$$={\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d{F}_{W_p}\left(x\right)+{\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d{F}_{p,2}^e\left(x\right)-{\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d{F}_{p,1}^e\left(x\right)$$

$$={\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d{F}_{W_p}\left(x\right)-{\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d(F_{p,1}^e\left(x\right)-F_{p,2}^e\left(x\right))$$

$$={\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d{F}_{W_p}\left(x\right)-{\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d{F}_{W_p^e}\left(x\right).$$

All the integrals in the latter formulas are finite. According to Lemma 2 and Remark 1, one can write $|f_h^{\prime }\left(x\right)|\le A_0|x|+B_0$, where $A_0,B_0$ are positive constants. Thus, the Lebesgue theorem on dominated convergence ensures that

$$\underset{n\to \infty }{lim}{\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d{F}_{W_p}\left(x\right)={\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d{F}_{W_p}\left(x\right),$$

where the latter integral is finite. Indeed,

$${\int }_{\mathbb{R}}(A_0|x|+B_0)d{F}_{W_p}\left(x\right)=A_0\mathbb{E}|W_p|+B_0<\infty$$

(33)

according to Lemma 4. By the same Lemma, one has $\mathbb{E}\left[W_p\right]=1$. Therefore, on account of Equation (17), the following relation holds:

$${\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d{F}_{W_p^e}\left(x\right)={\int }_{(-n,0]}{f}_h^{\prime }\left(x\right)(-F_{W_p}\left(x\right))dx+{\int }_{(0,n]}{f}_h^{\prime }\left(x\right)(1-F_{W_p}\left(x\right))dx,$$

whereas Corollary 2, Sec. 6, Ch. II of [47] and Lemma 4 entail that

$$\begin{array}{c}{\int }_{(-\infty ,0]}(A_0|x|+B_0)F_{W_p}\left(x\right)dx+{\int }_{(0,\infty )}(A_0|x|+B_0)(1-F_{W_p}\left(x\right))dx\hfill \\ \hfill \le A_0\mathbb{E}\left[W_p^2\right]+B_0\mathbb{E}|W_p|<\infty .\end{array}$$

(34)

The Lebesgue theorem on dominated convergence for $\sigma$-finite measures and Equation (34) yield

$$\underset{n\to \infty }{lim}{\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d{F}_{W_p}^e\left(x\right)={\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d{F}_{W_p}^e\left(x\right),$$

where the latter integral is finite. Now, we show that

$$\underset{n\to \infty }{lim}{\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d(F_{W_p}-F_{W_p}^e)\left(x\right)={\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d(F_{W_p}-F_{W_p}^e)\left(x\right).$$

(35)

Note that $f_h^{\prime }\left(x\right){\mathbb{I}}_{(-n,n]}\left(x\right)\to f_h^{\prime }\left(x\right)$ at each $x\in \mathbb{R}$ as $n\to \infty$. To apply the version of the Lebesgue theorem to integrals over a signed measure, it suffices (see, e.g., [48], p. 74) to verify that

$${\int }_{\mathbb{R}}|f_h^{\prime }\left(x\right)||d(F_{W_p}-F_{W_p}^e)\left(x\right)|<\infty ,$$

where $|dG|$ means that one evaluates an integral with respect to the measure corresponding to the total variation of a measure determined by a right-continuous function G of bounded variation. The extension of the Lebesgue theorem on dominated convergence for signed measures is an immediate corollary of the Jordan decomposition mentioned above. Using this decomposition, one obtains the inequality

$${\int }_{\mathbb{R}}|f_h^{\prime }\left(x\right)||d(F_{W_p}-F_{W_p}^e)\left(x\right)|\le {\int }_{\mathbb{R}}|f_h^{\prime }\left(x\right)||d{F}_{W_p}\left(x\right)|+{\int }_{\mathbb{R}}|f_h^{\prime }\left(x\right)||d{F}_{W_p}^e\left(x\right)|.$$

Due to Remark 1 one has $|f_h^{\prime }\left(x\right)|\le A_0|x|+B_0$ for all $x\in \mathbb{R}$ and some positive constants $A_0,B_0$. Then, Equations (33) and (34) yield (as $F_{W_p}$ generates probability measure)

$${\int }_{\mathbb{R}}(A_0|x|+B_0)d{F}_{W_p}\left(x\right)+{\int }_{\mathbb{R}}(A_0|x|+B_0)|d{F}_{W_p}^e\left(x\right)|<\infty .$$

The functions $f_h^{\prime }$ and $F_{W_p}-F_{W_p}^e$ are right-continuous and have bounded variation. Then each of them can be represented as the difference of right-continuous nondecreasing functions, and using for any $n\in \mathbb{N}$ the integration by parts formula (see, e.g., Theorem 11, Sec. 6, Ch. 2, [47]), one has

$$\begin{array}{c}{\int }_{(-n,n]}{f}_h^{\prime }\left(x\right)d(F_{W_p}-F_{W_p}^e)\left(x\right)\hfill \\ \hfill =f_h^{\prime }\left(x\right)(F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right)){|}_{-n}^n-{\int }_{(-n,n]}(F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right))d{f}_h^{\prime }\left(x\right).\end{array}$$

Since the integral in the right-hand side of Equation (35) is finite, it holds

$$f_h^{\prime }\left(x\right)(F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right))\to 0,x\to -\infty orx\to \infty$$

(36)

(the proof is similar to the proof of Corollary 2, Sec. 6, Ch. 2 in [47]). Then,

$${\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d(F_{W_p}-F_{W_p}^e)\left(x\right)=-\underset{n\to \infty }{lim}{\int }_{(-n,n]}(F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right))d{f}_h^{\prime }\left(x\right).$$

The function $f_h^{\prime }$ is absolutely continuous according to Lemma 2. Hence (see also Equations (36) and (A12) in Appendix A) we get

$$\begin{array}{c}\left|{\int }_{\mathbb{R}}{f}_h^{\prime }\left(x\right)d(F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right))\right|=\underset{n\to \infty }{lim}\left|{\int }_{(-n,n]}(F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right))f_h^{″}\left(x\right)dx\right|\hfill \\ \hfill \le \parallel f_h^{″}{\parallel }_{\infty }{\int }_{\mathbb{R}}\left|F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right)\right|dx\le {\int }_{\mathbb{R}}\left|F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right)\right|dx,\end{array}$$

(37)

because $\parallel f_h^{″}{\parallel }_{\infty }\le {\parallel h^{\prime }\parallel }_{\infty }\le 1$ due to Lemmas 1 and 2. Using the homogeneity of the Kantorovich metric for signed measures which is derived from formula (20) of [22] (see Lemma 1 (a) there) and applying Lemma 3 of that paper, we can write

$$\begin{array}{c}{\int }_{\mathbb{R}}\left|F_{W_p}\left(x\right)-F_{W_p}^e\left(x\right)\right|dx=\frac{p}{|\mu |(1-p)}{\int }_{\mathbb{R}}\left|F_{S_{N_p}}\left(x\right)-F_{S_{N_p}}^e\left(x\right)\right|dx\hfill \\ \hfill \le \frac{\mathbb{E}\left[X_{N_p+1}^2\right]}{2{\mu }^2}\left(\frac{p}{1-p}\right).\end{array}$$

(38)

Relations (30), (31), (32), (37), (38) and Lemmas 1 and 2 guarantee that $d_{{\mathcal{H}}_2}(W_p,Z)$ does not exceed the right-hand side of Equation (29).

Now, we turn to the lower bounds for $d_{{\mathcal{H}}_2}(W_p,Z)$. Choose $h\left(x\right)=x^2/2$ as the test function. Since $h\in {\mathcal{H}}_2$, we can write

$$d_{{\mathcal{H}}_2}(W_p,Z)\ge \left|\mathbb{E}\left[h\left(W_p\right)\right]-\mathbb{E}\left[h\left(Z\right)\right]\right|=\frac{1}{2}\left|\mathbb{E}\left[W_p^2\right]-\mathbb{E}\left[Z^2\right]\right|.$$

(39)

For a random variable Z following the exponential law $Exp\left(1\right)$, one has $\mathbb{E}\left[Z^2\right]=2$. Formula (23) of Lemma 4 yields

$$d_{{\mathcal{H}}_2}(W_p,Z)\ge \frac{\mathbb{E}\left[X_{N_p+1}^2\right]}{2{\mu }^2}\left(\frac{p}{1-p}\right).$$

Taking into account formula (38), we come to the desired statement. The proof is complete. □

Remark 3.

Evidently,

$$\mathbb{E}\left[X_{N_p+1}^2\right]=\sum _{n=0}^{\infty }\mathbb{E}\left[X_{n+1}^2\right]p{(1-p)}^n.$$

Thus, one obtains

$$\mathbb{E}\left[X_{N_p+1}^2\right]\le \underset{n\in \mathbb{N}}{sup}\mathbb{E}\left[X_n^2\right],$$

and the latter inequality becomes an equality when $\mathbb{E}\left[X_n^2\right]=\mathbb{E}\left[X_1^2\right]$ for all $n\in \mathbb{N}$. Therefore, the statement of Theorem 1 can be written as follows

$$d_{{\mathcal{H}}_2}(W_p,Z)\le \frac{{sup}_{n\in \mathbb{N}}\mathbb{E}\left[X_n^2\right]}{2{\mu }^2}\left(\frac{p}{1-p}\right),$$

and this becomes an equality when $\mathbb{E}\left[X_n^2\right]=\mathbb{E}\left[X_1^2\right]$ for all $n\in \mathbb{N}$.

Remark 4.

In [22], the authors proved the following inequality

$$d_{{\mathcal{H}}_2}(W_p,Z)\le \frac{3\mathbb{E}\left[X_{N_p+1}^2\right]}{2{\mu }^2}\left(\frac{p}{1-p}\right).$$

We established the sharp estimate with a factor $1/2$ instead of $3/2$ having employed Equation (20) for a class of functions comprising solutions of the Stein equation for $h\in {\mathcal{H}}_2$. The estimate with factor $1/2$ was also obtained in the recent paper [49] but for i.i.d. summands. The lower bounds were not provided there. In our Theorem 1, the summands have the same expectations but need not have the same distribution.

Remark 5.

If the summands of $W_p$ are non-negative, we consider $W_p^e$ appearing in Equation (28). Applying Theorem 1(i) [22] to relation (29), one obtains

$$d_{{\mathcal{H}}_1}(W_p^e,Z)=\frac{\mathbb{E}\left[X_{N_p+1}^2\right]}{2{\mu }^2}\frac{p}{1-p}.$$

For $i\in \mathbb{N}$, consider a random variable $X_i$ having distribution $Exp(1/\mu )$. Then $X_i^e\sim Exp(1/\mu )$, and, consequently, $X_{N_p+1}^e\sim Exp(1/\mu )$. We can choose $X_i^e$, $i\in \mathbb{N}$, according to Remark 2. Then, the distribution of $W_p^e$ will be the same if we change $X_{N_p+1}^e$ to $X_{N_p+1}$ in Equation (28). In such a way, $W_p^e$ is a normalized sum of a random number of independent random variables. Using the homogeneity of the Kantorovich metric, one has

$$d_{{\mathcal{H}}_1}\left(\frac{p}{\mu }\sum _{k=1}^{N_p+1}{X}_k,(1-p)Z\right)=(1-p)d_{{\mathcal{H}}_1}\left(\frac{p}{\mu (1-p)}\sum _{k=1}^{N_p+1}{X}_k,Z\right)=\frac{\mathbb{E}\left[X_{N_p+1}^2\right]}{2{\mu }^2}p.$$

(40)

Therefore, for an arbitrary sequence ${\left(X_k\right)}_{k\in \mathbb{N}}$ satisfying conditions of Theorem 1, the upper bound for the left-hand side of Equation (40) is not less than the right-hand side of Equation (40).

4. Limit Theorem for Geometric Sums of Exchangeable Random Variables

Now, we consider exchangeable random variables $X_1,X_2,\dots$ satisfying the dependence condition proposed in [35]. Namely, assume that for all $n\in \mathbb{N}$, $t_j\in \mathbb{R}$ ($j=1,\dots ,n$) and some $\rho \in [0,1]$

$$\mathbb{E}\left[e^{i(t_1{X}_1+\dots +t_n{X}_n)}\right]=\rho \mathbb{E}\left[e^{i{X}_1(t_1+\dots +t_n)}\right]+(1-\rho )\prod _{j=1}^n\mathbb{E}\left[e^{i{t}_j{X}_j}\right],$$

(41)

where $i^2=-1$. The cases of $\rho =0$ and $\rho =1$ correspond, respectively, to independent random variables and those possessing the property of comonotonicity. The latter means that for $\rho =1$ the joint behavior of $X_1,\dots ,X_n$ is strongly correlated and coincides with one of a vector $(X_1,\dots ,X_1)$.

Theorem 2.

Let $X_1,X_2\dots$ be exchangeable random variables with $\mathbb{E}\left[X_1\right]=\mu$, $\mu \ne 0$ satisfying condition (41) for some $\rho \in (0,1)$. Suppose that ${\left(X_n\right)}_{n\in \mathbb{N}}$ and $N_p$ are independent, where $N_p\sim Geom\left(p\right)$, $p\in (0,1)$. In contrast to the Rényi theorem, one has

$$W_p\stackrel{\mathcal{D}}{\to }Y,p\to 0+,$$

where the law of Y is the following mixture

$${\mathbb{P}}_Y=\rho {\mathbb{P}}_{V{X}_1/\mu }+(1-\rho ){\mathbb{P}}_Z,$$

(42)

random variables $X_1,V$ are independent and $V\sim Exp\left(1\right)$, $Z\sim Exp\left(1\right)$.

Proof. 

Let ${\tilde{X}}_1,{\tilde{X}}_2,\dots$ be independent copies of $X_1,X_2,\dots$, respectively. Suppose that ${\tilde{X}}_1,{\tilde{X}}_2,\dots$ are independent with $N_p$. Set $S_0:=0$, ${\tilde{S}}_0:=0$, ${\tilde{S}}_n:={\tilde{X}}_1+\dots +{\tilde{X}}_n$, $n\in \mathbb{N}$. Denote the characteristic function of a random variable $\xi$ by $f_{\xi }\left(t\right)$, $t\in \mathbb{R}$. For each $t\in \mathbb{R}$, using Equation (41), one has

$$f_{S_{N_p}}\left(t\right)=\sum _{n=0}^{\infty }\mathbb{E}\left[e^{it{S}_n}\right]\mathbb{P}(N_p=n)$$

$$=\mathbb{P}(N_p=0)+\sum _{n=1}^{\infty }\left(\rho \mathbb{E}\left[e^{i{X}_{1}tn}\right]+(1-\rho )\prod _{j=1}^n\mathbb{E}\left[e^{it{X}_j}\right]\right)\mathbb{P}(N_p=n)$$

$$=p+\sum _{n=0}^{\infty }\left(\rho \mathbb{E}\left[e^{i{X}_{1}tn}\right]+(1-\rho )\prod _{j=1}^n\mathbb{E}\left[e^{it{\tilde{X}}_j}\right]\right)\mathbb{P}(N_p=n)-\rho p-(1-\rho )p$$

$$=\rho f_{X_1{N}_p}\left(t\right)+(1-\rho )\sum _{n=0}^{\infty }{f}_{{\tilde{S}}_n}\left(t\right)\mathbb{P}(N_p=n)=\rho f_{X_1{N}_p}\left(t\right)+(1-\rho )f_{{\tilde{S}}_{N_p}}\left(t\right).$$

For each $t\in \mathbb{R}$, one has

$$f_{W_p}\left(t\right)=\rho f_{\frac{p}{\mu (1-p)}{X}_1{N}_p}\left(t\right)+(1-\rho )f_{{\tilde{W}}_p}\left(t\right),$$

(43)

where ${\tilde{W}}_p=\frac{p}{\mu (1-p)}{\sum }_{j=1}^{N_p}{\tilde{X}}_j$.

According to the classical Rényi theorem, ${\tilde{W}}_p\stackrel{\mathcal{D}}{\to }Z$ as $p\to 0+$, where $Z\sim Exp\left(1\right)$. Note that $T_p:=\frac{p}{1-p}{N}_p\stackrel{\mathcal{D}}{\to }V$ as $p\to 0+$, where $V\sim Exp\left(1\right)$. In fact, one can apply Theorem 1 with $X_j\equiv 1$, $j\in \mathbb{N}$ to check this. For each $t\in \mathbb{R}$, taking into account that $T_p$ and $X_1$ are independent and applying the Lebesgue theorem on dominated convergence, we see that

$$\mathbb{E}\left[e^{it{T}_p{X}_1}\right]=\mathbb{E}\left[\mathbb{E}{e}^{it{T}_p{X}_1}|X_1\right]={\int }_{\mathbb{R}}{e}^{it{T}_{p}x}d{F}_{X_1}\left(x\right)\to {\int }_{\mathbb{R}}{e}^{itVx}d{F}_{X_1}\left(x\right)=\mathbb{E}\left[e^{itV{X}_1}\right],p\to 0+,$$

since $X_1$ and V are independent. Hence,

$$\frac{p}{\mu (1-p)}{X}_1{N}_p\stackrel{\mathcal{D}}{⟶}\frac{V{X}_1}{\mu },p\to 0+$$

is true. In light of Equation (43),

$$W_p\stackrel{\mathcal{D}}{⟶}Y,p\to 0+,$$

here the law of Y is the mixture of distributions $V{X}_1/\mu$ and Z provided by Equation (42). The proof is complete. □

Theorem 3.

Assume that $N_p$ and ${\left(X_n\right)}_{n\in \mathbb{N}}$ satisfy conditions of Theorem 2. Let ${\mu }_2=\mathbb{E}\left[X_1^2\right]$. Then,

$$d_{{\mathcal{H}}_2}(W_p,Y)=\frac{{\mu }_2}{2{\mu }^2}\left(\frac{p}{1-p}\right).$$

(44)

Proof. 

Relation (43) for characteristic functions implies that the following equality of distributions holds

$$W_p\stackrel{\mathcal{D}}{=}\frac{p}{\mu (1-p)}\left((1-{\mathbb{I}}_{\rho })N_p{X}_1+{\mathbb{I}}_{\rho }{\tilde{S}}_{N_p}\right),$$

(45)

where indicator ${\mathbb{I}}_{\rho }$ equals 1 and 0 with probabilities $1-\rho$ and $\rho$, respectively, and is independent of all the variables under consideration. Assume at first that ${\mu }_2<\infty$. Then, for $h\in {\mathcal{H}}_2$,

$$\mathbb{E}\left[h\left(W_p\right)\right]=\rho \mathbb{E}\left[h\left(\frac{p}{\mu (1-p)}{N}_p{X}_1\right)\right]+(1-\rho )\mathbb{E}\left[h\left({\tilde{W}}_p\right)\right].$$

In view of Equation (42) one has

$$\mathbb{E}\left[h\left(Y\right)\right]=\rho \mathbb{E}\left[h\left(\frac{V{X}_1}{\mu }\right)\right]+(1-\rho )\mathbb{E}\left[h\left(Z\right)\right].$$

The latter two formulas and the triangle inequality yield

$$\begin{array}{c}|\mathbb{E}\left[h\left(W_p\right)\right]-\mathbb{E}\left[h\left(Y\right)\right]|\hfill \\ \hfill \le \rho \left|\mathbb{E}\left[h\left(\frac{p}{\mu (1-p)}{N}_p{X}_1\right)\right]-\mathbb{E}\left[h\left(\frac{V{X}_1}{\mu }\right)\right]\right|+(1-\rho )\left|\mathbb{E}\left[h\left({\tilde{W}}_p\right)\right]-\mathbb{E}\left[h\left(Z\right)\right]\right|.\end{array}$$

(46)

By means of Theorem 1 we have

$$\underset{h\in {\mathcal{H}}_2}{sup}|\mathbb{E}\left[h\left({\tilde{W}}_p\right)\right]-\mathbb{E}\left[h\left(Z\right)\right]|=\frac{{\mu }_2}{2{\mu }^2}\left(\frac{p}{1-p}\right).$$

(47)

For each $h\in {\mathcal{H}}_2$, taking into account the independence of $X_1$, $N_p$, V, one can write

$$\begin{array}{c}\left|\mathbb{E}\left[h\left(\frac{p}{\mu (1-p)}{N}_p{X}_1\right)\right]-\mathbb{E}\left[h\left(\frac{V{X}_1}{\mu }\right)\right]\right|\hfill \\ \hfill =\left|{\int }_{\mathbb{R}}\left(\mathbb{E}\left[h\left(\frac{p}{\mu (1-p)}{N}_p{X}_1\right)\right]-\mathbb{E}\left[h\left(\frac{xV}{\mu }\right)\right]\right)d{F}_{X_1}\left(x\right)\right|.\end{array}$$

Due to homogeneity of $d_{{\mathcal{H}}_2}$ we infer from Theorem 1 that

$$\underset{h\in {\mathcal{H}}_2}{sup}\left|\mathbb{E}\left[h\left(\frac{p}{\mu (1-p)}{N}_p{X}_1\right)\right]-\mathbb{E}\left[h\left(\frac{xV}{\mu }\right)\right]\right|=d_{{\mathcal{H}}_2}\left(\frac{px}{\mu (1-p)}{N}_p,\frac{xV}{\mu }\right)$$

$$={\left(\frac{x}{\mu }\right)}^2{d}_{{\mathcal{H}}_2}\left(\frac{p}{(1-p)}\sum _{k=1}^{N_p}1,V\right)=\frac{1}{2}{\left(\frac{x}{\mu }\right)}^2\frac{p}{1-p}.$$

Consequently, it holds

$$\begin{array}{c}\left|\mathbb{E}\left[h\left(\frac{p}{\mu (1-p)}{N}_p{X}_1\right)\right]-\mathbb{E}\left[h\left(\frac{V{X}_1}{\mu }\right)\right]\right|\hfill \\ \hfill \le \frac{p}{2(1-p)}{\int }_{\mathbb{R}}{\left(\frac{x}{\mu }\right)}^{2}d{F}_{X_1}\left(x\right)=\frac{{\mu }_2}{2{\mu }^2}\left(\frac{p}{1-p}\right).\end{array}$$

(48)

Equations (46), (47) and (48) lead to the upper bound for $d_{{\mathcal{H}}_2}(W_p,Y)$.

Note that a function $h\left(x\right)=x^2/2$, $x\in \mathbb{R}$, belongs to $\in {\mathcal{H}}_2$ and therefore

$$\underset{{\mathcal{H}}_2}{sup}\left|\mathbb{E}\left[h\left(W_p\right)\right]-\mathbb{E}\left[h\left(Y\right)\right]\right|\ge \frac{1}{2}\left(\mathbb{E}\left[W_p^2\right]-\mathbb{E}\left[Y^2\right]\right).$$

(49)

Note that $\mathbb{E}\left[Z^2\right]=\mathbb{E}\left[V^2\right]=2$ because $Z\sim Exp\left(1\right)$ and $V\sim Exp\left(1\right)$. The random variables $X_1,V,Z$ are independent. Thus, in light of Equation (42), one has

$$\mathbb{E}\left[Y^2\right]=2\rho \frac{{\mu }_2}{{\mu }^2}+2(1-\rho ).$$

(50)

By means of Equations (45), (23) and (25) we obtain

$$\begin{array}{c}\mathbb{E}\left[W_p^2\right]={\left(\frac{p}{\mu (1-p)}\right)}^2\rho \mathbb{E}\left[N_p^2\right]\mathbb{E}\left[X_1^2\right]+(1-\rho )\mathbb{E}\left[{\tilde{W}}_p^2\right]\hfill \\ ={\left(\frac{p}{\mu (1-p)}\right)}^2\rho \frac{(1-p)(2-p)}{p^2}{\mu }_2+(1-\rho )\left(\frac{p}{{\mu }^2(1-p)}{\mu }_2+2\right)=\\ \hfill =\frac{{\mu }_2}{{\mu }^2}\left(\rho \frac{2-p}{1-p}+(1-\rho )\frac{p}{1-p}\right)+2(1-\rho ).\end{array}$$

(51)

Equations (50) and (51) permit to find $\mathbb{E}\left[W_p^2\right]-\mathbb{E}\left[Y^2\right]$. Hence Equation (49) leads to the inequality

$$\begin{array}{c}\underset{{\mathcal{H}}_2}{sup}\left|\mathbb{E}\left[h\left(W_p\right)\right]-\mathbb{E}\left[h\left(Y\right)\right]\right|\hfill \\ \hfill \ge \left(\frac{1}{2}\right)\frac{{\mu }_2}{{\mu }^2}\left(\rho \left(\frac{2-p}{1-p}-2\right)+(1-\rho )\frac{p}{1-p}\right)=\left(\frac{1}{2}\right)\frac{{\mu }_2}{{\mu }^2}\frac{p}{1-p}.\end{array}$$

(52)

Now, let ${\mu }_2=\infty$. Then, $d_{{\mathcal{H}}_2}(W_p,Y)=\infty$ according to Equation (52). The proof is complete. □

5. Convergence of Random Sums of Independent Summands to Generalized Gamma Distribution

Statements concerning weak convergence of geometric sums distributions to exponential law are often just particular cases of more general results concerning the convergence of random sums of random summands to generalized gamma law when the number of summands follows the generalized negative binomial distribution, see, e.g., [27,29,49]). The recent work [29] demonstrated how it is possible to study the mentioned general case employing the estimates of proximity of geometric sums distributions to exponential law. We introduce some notation to apply Theorem 1 for analysis of the distance between the distributions of random sums and the generalized gamma law.

Introduce a random variable $G_{r,\lambda }$ such that $G_{r,\lambda }\sim G(r,\lambda )$, where $G(r,\lambda )$ is the gamma law with positive parameters r and $\lambda$, i.e., its density with respect to the Lebesgue measure has the form

$$g(z;r,\lambda )=\frac{{\lambda }^r{z}^{r-1}}{\Gamma \left(r\right)}{e}^{-\lambda z}{\mathbb{I}}_{(0,\infty )}\left(z\right),z\in \mathbb{R},$$

$\Gamma \left(r\right)$ being the gamma function. For $r=1$, one has $G(1,\lambda )=Exp\left(\lambda \right)$. Clearly, for $a>0$, $a{G}_{r,\lambda }\sim G(r,\lambda /a)$. Set $G_{r,\alpha ,\lambda }^{*}:=G_{r,\lambda }^{1/\alpha }$, where $\alpha >0$. One says that random variable $G_{r,\alpha ,\lambda }^{*}$ has the generalized gamma distribution $G^{*}(r,\alpha ,\lambda )$. According to Equation (5) of [29], the density of $G_{r,\alpha ,\lambda }^{*}$ is given by formula

$$g^{*}(z;r,\alpha ,\lambda )=\frac{|\alpha |{\lambda }^r{z}^{\alpha r-1}}{\Gamma \left(r\right)}{e}^{-\lambda z^{\alpha }}{\mathbb{I}}_{(0,\infty )}\left(z\right),z\in \mathbb{R}.$$

Also it is known (see Equation (6) in [29]) that, for $r\in (0,1)$, $\alpha \in (0,1]$ and $\lambda >0$, the following relation holds

$$g^{*}(z;r,\alpha ,\lambda )={\int }_0^1\frac{u}{1-u}{e}^{-\frac{u}{1-u}z}q(u;r,\alpha ,\lambda )du,z>0,$$

(53)

where q is a density of a specified random variable $Y_{r,\alpha ,\lambda }$ such that support of its distribution belongs to $(0,1)$ (see Remark 3 [49]). We only note that for $\alpha =1$ the density q admits a representation

$$q\left(u;r,1,\frac{b}{1-b}\right)=b^r\left(\frac{sin\pi r}{\pi }\right)\frac{{(1-u)}^{r-1}}{u{(u-b)}^r}{\mathbb{I}}_{(b,1)}\left(u\right),b\in (0,1).$$

Consider a random variable $N_{r,\alpha ,p}^{*}$ having the generalized negative binomial distribution $GNB(r,\alpha ,p)$, where $r>0$, $\alpha \ne 0$ and $p\in (0,1)$, i.e.,

$$\mathbb{P}(N_{r,\alpha ,p}^{*}=k)={\int }_0^{\infty }\frac{z^k}{k!}{e}^{-z}{g}^{*}\left(z;r,\alpha ,\frac{p}{1-p}\right)dz,k=0,1,\dots$$

(54)

Thus $GNB(r,\alpha ,p)$ has a mixed Poisson distribution. One can verify that $GNB(r,1,p)$ coincides with $NB(r,p)$, where $NB(r,p)$ is the negative binomial law. Recall that $N_{r,p}\sim NB(r,p)$ if

$$\mathbb{P}(N_{r,p}=k)=\frac{\Gamma (k+r)}{k!\Gamma \left(r\right)}{p}^r{(1-p)}^k,k=0,1,\dots$$

Note also that $N_{1,p}\sim Geom\left(p\right)$.

Introduce the random variables

$$W_{r,\alpha ,p}^{*}:=\frac{1}{\mu }{\left(\frac{p}{1-p}\right)}^{1/\alpha }\sum _{k=1}^{N_{r,\alpha ,p}^{*}}{X}_k,S_{r,\alpha ,p}^{*}:=\sum _{k=1}^{N_{r,\alpha ,p}^{*}}{X}_k,$$

(55)

where $N_{r,\alpha ,p}^{*}\sim GNB(r,\alpha ,p)$, $r>0$, $\alpha \ne 0$, $p\in (0,1)$, and $\mathbb{E}\left[X_k\right]=\mu$, $\mu \ne 0$, $k\in \mathbb{N}$. We assume that ${\left(X_n\right)}_{n\in \mathbb{N}}$ and $N_{r,\alpha ,p}^{*}$ are independent, where $r>0$, $\alpha \ne 0$, $p\in (0,1)$.

Theorem 4.

Let ${\left(X_n\right)}_{n\in \mathbb{N}}$ be a sequence of independent random variables having $\mathbb{E}\left[X_n\right]=\mu$, $\mu \ne 0$, $n\in \mathbb{N}$. Then, for $W_{r,\alpha ,p}^{*}$ introduced in Equation (55) with parameters $r\in (0,1)$, $\alpha \in (0,1]$, $p\in (0,1)$ and $G_{r,1}$ having the gamma distribution $G(r,1)$, the following relation holds

$$d_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},G_{r,1}^{1/\alpha })=\frac{1}{2{\mu }^2}{\left(\frac{p}{1-p}\right)}^{2/\alpha }{\int }_0^1\mathbb{E}\left[X_{N_u+1}^2\right]\left(\frac{1-u}{u}\right)q\left(u;r,\alpha ,\frac{p}{1-p}\right)du,$$

(56)

whenever the right-hand side of Equation (56) is finite. Here, $N_u:=N_{1,1,u}^{*}$, $N_u\sim Geom\left(u\right)$, $u\in (0,1)$ and q appeared in Equation (53).

Proof. 

Without loss of generality, we can assume that $\mu =1$; otherwise, we consider ${\tilde{X}}_n:=\frac{X_n}{\mu }$, $n\in \mathbb{N}$. For such sequence, $\mathbb{E}\left[{\tilde{X}}_{N_u+1}^2\right]=\frac{1}{{\mu }^2}\mathbb{E}\left[X_{N_u+1}^2\right]$. Note that $\frac{1-p}{p}{G}_{r,1}$ has the same distribution as $G_{r,p/(1-p)}$. Applying the homogeneity property of the ideal probability metric of order two, one has

$$d_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},G_{r,1}^{1/\alpha })={\left(\frac{p}{1-p}\right)}^{2/\alpha }{d}_{{\mathcal{H}}_2}\left(S_{r,\alpha ,p}^{*},G_{r,p/(1-p)}^{1/\alpha }\right).$$

The proof of Theorem 1 [29] starts with establishing for any bounded Borel function h, $r\in (0,1)$, $\alpha \in (0,1]$ and $p\in (0,1)$, that

$$\mathbb{E}\left[h\left(G_{r,p/(1-p)}^{1/\alpha }\right)\right]={\int }_0^1\mathbb{E}\left[h\left(\frac{1-u}{u}Z\right)\right]q\left(u;r,\alpha ,\frac{p}{1-p}\right)du,$$

(57)

where $Z\sim Exp\left(1\right)$, and

$$\mathbb{E}\left[h\left(S_{r,\alpha ,p}^{*}\right)\right]={\int }_0^1\mathbb{E}\left[h\left(S_{1,1,u}^{*}\right)\right]q\left(u;r,\alpha ,\frac{p}{1-p}\right)du.$$

(58)

Let us examine these relations for each $h\in {\mathcal{H}}_2$. Recall that in light of Remark 1 $|h\left(x\right)|\le A_0{x}^2+B_0$ for some positive constants $A_0$ and $B_0$ (which depend on h), we write $h=h^{+}-h^{-}$, where $h^{+}\left(x\right):=h\left(x\right)\mathbb{I}\{h\left(x\right)\ge 0\}$, $h^{-}\left(x\right):=-h\left(x\right)\mathbb{I}\{h\left(x\right)\le 0\}$. Set $h_n\left(x\right):=h^{+}\left(x\right){\mathbb{I}}_{(-n,n]}\left(x\right)$, $n\in \mathbb{N}$. Then, $h_n$ and $n\in \mathbb{N}$ are bounded Borel functions such that for each $x\in \mathbb{R}$, $0\le h_n\left(x\right)\nearrow h^{+}\left(x\right)$ as $n\to \infty$. Hence, the monotone convergence theorem yields

$$\mathbb{E}\left[h^{+}\left(G_{r,p/(1-p)}^{1/\alpha }\right)\right]=\underset{n\to \infty }{lim}\mathbb{E}\left[h_n\left(G_{r,p/(1-p)}^{1/\alpha }\right)\right].$$

Note that, for each $u\in (0,1)$, $\mathbb{E}\left[h_n\left(\frac{1-u}{u}Z\right)\right]\nearrow \mathbb{E}\left[h^{+}\left(\frac{1-u}{u}Z\right)\right]$. Applying the monotone convergence theorem once again, we obtain

$${\int }_0^1\mathbb{E}\left[h^{+}\left(\frac{1-u}{u}Z\right)\right]q\left(u;r,\alpha ,\frac{p}{1-p}\right)du=\underset{n\to \infty }{lim}{\int }_0^1\mathbb{E}\left[h_n\left(\frac{1-u}{u}Z\right)\right]q\left(u;r,\alpha ,\frac{p}{1-p}\right)du.$$

So, Equation (57) is valid if instead of h belonging to ${\mathcal{H}}_2$ we write $h^{+}$. Obviously, $0\le h^{+}\left(x\right)\le |h\left(x\right)|\le A_0{x}^2+B_0$, $x\in \mathbb{R}$, $n\in \mathbb{R}$. Thus,

$$\mathbb{E}\left[h^{+}{\left(G_{r,p/(1-p)}^{1/\alpha }\right)}^2\right]\le A_0\mathbb{E}\left(G_{r,p/(1-p)}^{2/\alpha }\right)+B_0<\infty .$$

According to [27] (page 8), for $\delta >0$, one has

$$\mathbb{E}\left[{\left(G_{r,\alpha ,\lambda }^{*}\right)}^{\delta }\right]=\frac{\Gamma (r+\frac{\delta }{\alpha })}{{\lambda }^{\delta /\alpha }\Gamma \left(r\right)}.$$

(59)

This permits us to write $\mathbb{E}\left(G_{r,p/(1-p)}^{2/\alpha }\right)=\mathbb{E}\left[{\left(G_{r,1,p/(1-p)}^{*}\right)}^{2/\alpha }\right]<\infty$.

In the same manner, we demonstrate that Equation (57) is valid if instead of $h\in {\mathcal{H}}_2$ we take $h^{-}$. Moreover, $\mathbb{E}\left[h^{-}\left(G_{r,p/(1-p)}^{1/\alpha }\right)\right]$ is finite. Therefore, Equation (57) holds for any $h\in {\mathcal{H}}_2$, and for such h, $\mathbb{E}\left[h\left(G_{r,p/(1-p)}^{1/\alpha }\right)\right]$ is finite.

By the monotone convergence theorem $\mathbb{E}\left[h^{+}\left(S_{r,\alpha ,p}^{*}\right)\right]={lim}_{n\to \infty }\mathbb{E}\left[h_n\left(S_{r,\alpha ,p}^{*}\right)\right]$. In a similar way, $\mathbb{E}\left[h_n\left(S_{1,1,u}^{*}\right)\right]\nearrow \mathbb{E}\left[h^{+}\left(S_{1,1,u}^{*}\right)\right]$ as $n\to \infty$, and applying this theorem once again, we obtain

$${\int }_0^1\mathbb{E}\left[h^{+}\left(S_{1,1,u}^{*}\right)\right]q\left(u;r,\alpha ,\frac{p}{1-p}\right)du=\underset{n\to \infty }{lim}{\int }_0^1\mathbb{E}\left[h_n\left(S_{1,1,u}^{*}\right)\right]q\left(u;r,\alpha ,\frac{p}{1-p}\right)du.$$

Taking into account that Equation (58) is valid for bounded Borel functions $h_n$, one ascertains that Equation (58) holds if we replace h by $h^{+}$. To show the latter integral is finite, we note that $0\le h^{+}\left(x\right)\le |h\left(x\right)|\le A_0{x}^2+B_0$, for some positive $A_0,B_0$ and all $x\in \mathbb{R}$. Formula (23) of Lemma 4 yields, for each $u\in (0,1)$,

$$\mathbb{E}\left[{\left(S_{1,1,u}^{*}\right)}^2\right]\le \frac{1-u}{u}\mathbb{E}\left[X_{N_u+1}^2\right]+2\frac{{(1-u)}^2}{u^2}.$$

It was assumed above that the right-hand side of Equation (56) is finite. So,

$${\int }_0^1\mathbb{E}\left(A_0\left(\frac{1-u}{u}\mathbb{E}\left[X_{N_u+1}^2\right]+2\frac{{(1-u)}^2}{u^2}\right)+B_0\right)q\left(u;r,\alpha ,\frac{p}{1-p}\right)du<\infty ,$$

since in light of Equation (57), taking $h\left(x\right)=1$ and $h\left(x\right)=\frac{x^2}{2}$ (these functions belong to ${\mathcal{H}}_2$), $x\in \mathbb{R}$, we obtain, respectively,

$${\int }_0^{1}q\left(u;r,\alpha ,\frac{p}{1-p}\right)du=1,$$

$$\mathbb{E}\left[Z^2\right]{\int }_0^1\frac{{(1-u)}^2}{u^2}q\left(u;r,\alpha ,\frac{p}{1-p}\right)du=\mathbb{E}\left(G_{r,p/(1-p)}^{2/\alpha }\right)<\infty .$$

(60)

We demonstrate analogously that Equation (58) holds upon replacing $h\in {\mathcal{H}}_2$ with $h^{-}$ and if the right-hand side of Equation (56) is finite, it follows that

$${\int }_0^1\mathbb{E}\left[h^{-}\left(S_{1,1,u}^{*}\right)\right]q\left(u;r,\alpha ,\frac{p}{1-p}\right)du$$

is finite as well. Consequently, Equation (58) is established for each $h\in {\mathcal{H}}_2$ (whenever the right-hand side of Equation (56) is finite) and $\mathbb{E}\left[h\left(S_{r,\alpha ,p}^{*}\right)\right]$ is finite for such h. Therefore, for $h\in {\mathcal{H}}_2$ and fixed $\alpha ,r,p$, one has

$$\begin{array}{c}\mathbb{E}\left[h\left(S_{r,\alpha ,p}^{*}\right)\right]-\mathbb{E}\left[h\left(G_{r,p/(1-p)}^{1/\alpha }\right)\right]\hfill \\ \hfill ={\int }_0^1\left(\mathbb{E}\left[h\left(S_{1,1,u}^{*}\right)\right]-\mathbb{E}\left[h\left(\frac{1-u}{u}Z\right)\right]\right)q\left(u;r,\alpha ,\frac{p}{1-p}\right)du=:J\left(h\right).\end{array}$$

By Theorem 1, for $h\in {\mathcal{H}}_2$, it holds

$$\left|\mathbb{E}\left[h\left(S_{1,1,u}^{*}\right)\right]-\mathbb{E}\left[h\left(\frac{1-u}{u}Z\right)\right]\right|\le d_{{\mathcal{H}}_2}\left(S_{1,1,u}^{*},\frac{1-u}{u}Z\right)={\left(\frac{1-u}{u}\right)}^2{d}_{{\mathcal{H}}_2}\left(\frac{u}{1-u}{S}_{1,1,u}^{*},Z\right)$$

$$\le {\left(\frac{1-u}{u}\right)}^2\frac{u}{1-u}\left(\frac{1}{2}\right)\mathbb{E}\left[X_{N_u+1}^2\right]=\left(\frac{1}{2}\right)\frac{1-u}{u}\mathbb{E}\left[X_{N_u+1}^2\right],$$

where we take into account that $N_{1,1,u}^{*}\sim NB(1,u)$, and $NB(1,u)$ coincides with $Geom\left(u\right)$. Thus, $\frac{u}{1-u}{S}_{1,1,u}^{*}$ can be written as

$$\frac{u}{1-u}\sum _{k=1}^{N_u}{X}_k,$$

where $N_u\sim Geom\left(u\right)$, $N_u$ and ${\left(X_k\right)}_{k\in \mathbb{N}}$ are independent.

Therefore, for each $h\in {\mathcal{H}}_2$, ${\left(\frac{p}{1-p}\right)}^{2/\alpha }|J\left(h\right)|$ is bounded by the right-hand side of Equation (56), and so the desired upper bound is obtained (recall that $\mu =1$).

Now, we turn to the lower bound of $d_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},G_{r,1}^{1/\alpha })$. Take $h\left(x\right)=x^2/2$ belonging to ${\mathcal{H}}_2$. Then, applying Equation (23) to evaluate $\mathbb{E}\left[{\left(S_{1,1,u}^{*}\right)}^2\right]$, one has

$$\begin{array}{c}{d}_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},G_{r,1}^{1/\alpha })\hfill \\ \ge \frac{1}{2}{\left(\frac{p}{1-p}\right)}^{2/\alpha }\left|{\int }_0^1\left(\mathbb{E}\left[{\left(S_{1,1,u}^{*}\right)}^2\right]-{\left(\frac{1-u}{u}\right)}^2\mathbb{E}\left[G_{1,1}^2\right]\right)q\left(u;r,\frac{p}{1-p}\right)du\right|\\ \hfill =\frac{1}{2}{\left(\frac{p}{1-p}\right)}^{2/\alpha }{\int }_0^1\left(\frac{1-u}{u}\right)\mathbb{E}\left[X_{N_u+1}^2\right]q\left(u;r,\frac{p}{1-p}\right)du,\end{array}$$

(61)

where $G_{1,1}=Z\sim Exp\left(1\right)$. Thus, Equation (61) completes the proof. □

Corollary 1.

Let conditions of Theorem 4 be satisfied and also ${\mu }_2={sup}_{n\in \mathbb{N}}\mathbb{E}\left[X_n^2\right]<\infty$. Then, the right-hand side of Equation (56) is finite and

$$d_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},G_{r,1}^{1/\alpha })\le \frac{{\mu }_2}{2{\mu }^2}{\left(\frac{p}{1-p}\right)}^{1/\alpha }\frac{\Gamma (r+\frac{1}{\alpha })}{\Gamma \left(r\right)}.$$

The inequality becomes an equality if ${\mu }_2=\mathbb{E}\left[X_n^2\right]$ for all $n\in \mathbb{N}$. In particular, if $\alpha =1$ then $\frac{\Gamma (r+1)}{\Gamma \left(r\right)}=r$.

Proof. 

According to Equation (57), for $h\left(x\right)=x$, $x\in \mathbb{R}$,

$$\mathbb{E}\left[G_{r,p/(1-p)}^{1/\alpha }\right]=\mathbb{E}\left[Z\right]{\int }_0^1\left(\frac{1-u}{u}\right)q\left(u;r,\alpha ,\frac{p}{1-p}\right)du.$$

Thus, the following relation is valid.

$${\int }_0^1\left(\frac{1-u}{u}\right)q\left(u;r,\alpha ,\frac{p}{1-p}\right)du=\mathbb{E}\left[G_{r,p/(1-p)}^{1/\alpha }\right].$$

(62)

Due to [27] (see page 8 there), for $\delta >0$, one has $\mathbb{E}\left[G_{r,\alpha ,\lambda }^{*}\right]=\frac{\Gamma (r+1/\alpha )}{{\lambda }^{1/\alpha }\Gamma \left(r\right)}.$ Therefore,

$$\mathbb{E}\left[G_{r,p/(1-p)}^{1/\alpha }\right]=\mathbb{E}\left[G_{r,\alpha ,p/(1-p)}^{*}\right]={\left(\frac{1-p}{p}\right)}^{\frac{1}{\alpha }}\frac{\Gamma (r+\frac{1}{\alpha })}{\Gamma \left(r\right)}.$$

For $\alpha =1$, we obtain $\mathbb{E}\left[G_{r,p/(1-p)}\right]=\frac{1-p}{p}\frac{\Gamma (r+1)}{\Gamma \left(r\right)}=r\frac{(1-p)}{p}$. □

6. Convergence of Random Sums of Exchangeable Summands to Generalized Gamma Distribution

Consider the model of exchangeable random variables $X_1,X_2,\dots$ described in Section 4. Introduce the distribution of a random variable $U_{r,\alpha ,\lambda }^{*}$ as the following mixture

$${\mathbb{P}}_{U_{r,\alpha ,\lambda }^{*}}=\rho {\mathbb{P}}_{\left(\frac{V_{r,\alpha ,\lambda }^{*}{X}_1}{\mu }\right)}+(1-\rho ){\mathbb{P}}_{Z_{r,\alpha ,\lambda }^{*}},$$

(63)

where $\rho \in [0,1]$, $\alpha >0$, $r>0$, $\mu :=\mathbb{E}\left[X_1\right]$, $\mu \ne 0$, random variables $X_1,V_{r,\alpha ,\lambda }^{*}$ are independent, $V_{r,\alpha ,\lambda }^{*}\sim G^{*}(r,\alpha ,\lambda )$, $Z_{r,\alpha ,\lambda }^{*}\sim G^{*}(r,\alpha ,\lambda )$. Since $\mathbb{E}\left[G_{r,\lambda }^{2/\alpha }\right]=\frac{\Gamma (r+2/\alpha )}{{\lambda }^{2/\alpha }\Gamma \left(r\right)}$ (see, e.g., page 8 [27]), one has

$$\mathbb{E}\left[{\left(U_{r,\alpha ,\lambda }^{*}\right)}^2\right]=\left(\rho \frac{\mathbb{E}\left[X_1^2\right]}{{\mu }^2}+(1-\rho )\right)\frac{\Gamma (r+2/\alpha )}{{\lambda }^{2/\alpha }\Gamma \left(r\right)}.$$

(64)

Due to the properties of generalized gamma distributions, for any positive number c,

$$\begin{array}{c}\frac{1}{c^{\alpha }}{U}_{r,\alpha ,\lambda }^{*}=\frac{1}{c^{\alpha }}\left((1-{\mathbb{I}}_{\rho })\frac{V_{r,\alpha ,\lambda }^{*}{X}_1}{\mu }+{\mathbb{I}}_{\rho }{Z}_{r,\alpha ,\lambda }^{*}\right)\hfill \\ \hfill =\left((1-{\mathbb{I}}_{\rho })\frac{V_{r,\alpha ,\lambda }^{*}{X}_1}{\mu }+{\mathbb{I}}_{\rho }{Z}_{r,\alpha ,c\lambda }^{*}\right)=U_{r,\alpha ,c\lambda }^{*},\end{array}$$

(65)

where indicator ${\mathbb{I}}_{\rho }$ equals 1 and 0 with probabilities $1-\rho$ and $\rho$, respectively, and is independent with all the variables under consideration. Note that $U_{1,1,1}^{*}$ has the same distribution as a random variable Y, having the law defined in Equation (42). Recall that the generalized negative binomial distribution $GNB(r,\alpha ,p)$ is the law of a random variable $N_{r,\alpha ,p}^{*}$, see Equation (54). We will use the following result.

Lemma 5.

If $r>0$, $\alpha \ne 0$, $p\in (0,1)$, then for $N_{r,\alpha ,p}^{*}\sim GNB(r,\alpha ,p)$ one has

$$\mathbb{E}\left[N_{r,\alpha ,p}^{*}\right]=\mathbb{E}\left[G_{r,\alpha ,p/(1-p)}^{*}\right],\mathbb{E}\left[N_{r,\alpha ,p}^{*}(N_{r,\alpha ,p}^{*}-1)\right]=\mathbb{E}\left[{\left(G_{r,\alpha ,p/(1-p)}^{*}\right)}^2\right].$$

(66)

Proof. 

According to Equation (54), for each $n\in \mathbb{N}$,

$$\sum _{k=1}^{n}k\mathbb{P}(N_{r,\alpha ,p}^{*}=k)={\int }_0^{\infty }z\sum _{k=1}^n\frac{z^{k-1}}{(k-1)!}{e}^{-z}{g}^{*}(z;r,\alpha ,\frac{p}{1-p})dz,$$

$$\sum _{k=2}^{n}k(k-1)\mathbb{P}(N_{r,\alpha ,p}^{*}=k)={\int }_0^{\infty }{z}^2\sum _{k=2}^n\frac{z^{k-2}}{(k-2)!}{e}^{-z}{g}^{*}(z;r,\alpha ,\frac{p}{1-p})dz.$$

The desired statement follows from the monotone convergence theorem for the Lebesgue integral by letting $n\to \infty$. □

Theorem 5.

Let $X_1,X_2\dots$ be exchangeable random variables, introduced in Section 4, such that $\mathbb{E}\left[X_1\right]=\mu$, $\mathbb{E}\left[X_1^2\right]={\mu }_2<\infty$. Assume that for some $\rho \in (0,1)$ Equation (41) holds. Suppose that ${\left(X_n\right)}_{n\in \mathbb{N}}$ and $N_{r,\alpha ,p}^{*}$ are independent, where $N_{r,\alpha ,p}^{*}\sim GNB(r,\alpha ,p)$. Then, for $W_{r,\alpha ,p}^{*}$ defined in Equation (55) with parameters $r\in (0,1)$, $\alpha \in (0,1]$, $p\in (0,1)$ and $U_{r,\alpha ,1}^{*}$ given in Equation (63), one has

$$d_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},U_{r,\alpha ,1}^{*})=\frac{{\mu }_2}{2{\mu }^2}{\left(\frac{p}{1-p}\right)}^{1/\alpha }\frac{\Gamma (1+\frac{1}{\alpha })}{\Gamma \left(r\right)}.$$

(67)

Proof. 

Without loss of generality, we can assume that $\mu =1$; otherwise, we consider ${\tilde{X}}_n:=X_n/\mu$, $n\in \mathbb{N}$. For such sequence, ${\tilde{\mu }}_2=\mathbb{E}{\tilde{X}}_1^2={\mu }_2/{\mu }^2$. Note that Equation (58) is true for dependent summands (see Theorem 1 [29]). Furthermore, for bounded $h\left(t\right)$, $t\in \mathbb{R}$, function $h_x\left(t\right)=h\left(xt\right)$ is also bounded for any $x\in \mathbb{R}$. Thus, an employment of Equation (63) gives

$$\mathbb{E}\left[h\left(U_{r,\alpha ,\lambda }^{*}\right)\right]=\rho {\int }_{\mathbb{R}}\mathbb{E}\left[h_x\left(G_{r,\lambda }^{1/\alpha }\right)\right]d{F}_{X_1}\left(x\right)+(1-\rho )\mathbb{E}\left[h\left(G_{r,\lambda }^{1/\alpha }\right)\right].$$

(68)

Now we apply Equation (57) with bounded $h_x$ and by Fubini’s theorem obtain:

$$\begin{array}{c}{\int }_{\mathbb{R}}\mathbb{E}\left[h_x\left(G_{r,\lambda }^{1/\alpha }\right)\right]d{F}_{X_1}\left(x\right)={\int }_{\mathbb{R}}{\int }_0^1\mathbb{E}\left[h_x\left(\frac{1-u}{u}{V}^{*}\right)\right]q\left(u;r,\alpha ,\lambda \right)dud{F}_{X_1}\left(x\right)\hfill \\ \hfill ={\int }_0^1\mathbb{E}\left[h\left(\frac{1-u}{u}{X}_1{V}^{*}\right)\right]q\left(u;r,\alpha ,\lambda \right)du,\end{array}$$

(69)

where $X_1$ and $V^{*}$ are independent and $V^{*}\sim Exp\left(1\right)$. Apply Equation (57) for the second summand of Equation (68). Then, Equation (69) yields

$$\begin{array}{c}\mathbb{E}\left[h\left(U_{r,\alpha ,\lambda }^{*}\right)\right]\hfill \\ =\rho {\int }_0^1\mathbb{E}\left[h\left(\frac{1-u}{u}{X}_1{V}^{*}\right)\right]q\left(u;r,\alpha ,\lambda \right)du+(1-\rho ){\int }_0^1\mathbb{E}\left[h\left(\frac{1-u}{u}{Z}^{*}\right)\right]q\left(u;r,\alpha ,\lambda \right)du\\ \hfill ={\int }_0^1\mathbb{E}\left[h\left(\frac{1-u}{u}{U}_{1,1,1}^{*}\right)\right]q\left(u;r,\alpha ,\lambda \right)du,\end{array}$$

(70)

where $Z^{*}\sim Exp\left(1\right)$ and $U_{1,1,1}^{*}$ have the same distribution as Y, see Equation (42).

Recall that, for $h\in {\mathcal{H}}_2$, an inequality $|h\left(x\right)|\le A_0{x}^2+B_0$ holds for all $x\in \mathbb{R}$ and some positive constants $A_0$, $B_0$ (see Remark 1). Moreover, $\mathbb{E}\left[{\left(U_{r,\alpha ,\lambda }^{*}\right)}^2\right]<\infty$ according to Equation (64). So, employing bounded $h_n\left(x\right)=h\left(x\right){\mathbb{I}}_{(-n,n]}\left(x\right)$ tending to $h\left(x\right)\in {\mathcal{H}}_2$ as $n\to \infty$, one can invoke the Lebesgue dominated convergence theorem to claim that ${lim}_{n\to \infty }\mathbb{E}\left[h_n\left(U_{r,\alpha ,\lambda }^{*}\right)\right]=\mathbb{E}\left[h\left(U_{r,\alpha ,\lambda }^{*}\right)\right]$. We take into account that

$${\int }_0^1\mathbb{E}\left|h_n\left(\frac{1-u}{u}{U}_{1,1,1}^{*}\right)\right|q\left(u;r,\alpha ,\lambda \right)du\le A_0\mathbb{E}\left[{\left(U_{1,1,1}^{*}\right)}^2\right]{\int }_0^1{\left(\frac{1-u}{u}\right)}^{2}q\left(u;r,\alpha ,\lambda \right)du+B_0.$$

The integral in the right-hand side of the latter formula is finite by Equation (60) and $\mathbb{E}\left[{\left(U_{1,1,1}^{*}\right)}^2\right]<\infty$ in accord with Equation (64). Thus, it is possible to apply the Lebesgue dominated convergence theorem to obtain

$$\underset{n\to \infty }{lim}{\int }_0^1\mathbb{E}\left[h_n\left(\frac{1-u}{u}{U}_{1,1,1}^{*}\right)\right]q\left(u;r,\alpha ,\lambda \right)du={\int }_0^1\mathbb{E}\left[h\left(\frac{1-u}{u}{U}_{1,1,1}^{*}\right)\right]q\left(u;r,\alpha ,\lambda \right)du$$

for any $h\in {\mathcal{H}}_2$. So, Equation (70) holds for all $h\in {\mathcal{H}}_2$.

In a similar way, ${lim}_{n\to \infty }\mathbb{E}\left[h_n\left(S_{r,\alpha ,p}^{*}\right)\right]=\mathbb{E}\left[h\left(S_{r,\alpha ,p}^{*}\right)\right]$ for $h\in {\mathcal{H}}_2$. According to the Cauchy–Bunyakovsky–Schwarz inequality for identically distributed variables $X_1,X_2,\dots$ we have $|\mathbb{E}\left[X_i{X}_j\right]|\le {\mu }_2$ for $i,j\in \mathbb{N}$ and consequently

$$\begin{array}{c}\mathbb{E}\left[{\left(S_{r,\alpha ,p}^{*}\right)}^2\right]=\sum _{k=0}^{\infty }\mathbb{P}(N_{r,\alpha ,p}^{*}=k)\mathbb{E}\left[{\left(\sum _{j=1}^k{X}_j\right)}^2\right]\hfill \\ \hfill \le {\mu }_2\sum _{k=0}^{\infty }\mathbb{P}(N_{r,\alpha ,p}^{*}=k)k^2={\mu }_2\mathbb{E}\left[{\left(N_{r,\alpha ,p}^{*}\right)}^2\right].\end{array}$$

(71)

Equations (59) and (66) entail that $\mathbb{E}\left[{\left(N_{r,\alpha ,p}^{*}\right)}^2\right]<\infty$. Thus, the dominated convergence theorem guarantees that ${lim}_{n\to \infty }\mathbb{E}\left[h_n\left(S_{r,\alpha ,p}^{*}\right)\right]=\mathbb{E}\left[h\left(S_{r,\alpha ,p}^{*}\right)\right]$. Furthermore, one can demonstrate that, for each $h\in {\mathcal{H}}_2$,

$$\underset{n\to \infty }{lim}{\int }_0^1\mathbb{E}\left[h_n\left(S_{1,1,u}^{*}\right)\right]q\left(u;r,\alpha ,\lambda \right)du={\int }_0^1\mathbb{E}\left[h\left(S_{1,1,u}^{*}\right)\right]q\left(u;r,\alpha ,\lambda \right)du.$$

(72)

For this purpose we note that Equation (71) implies

$${\int }_0^1\mathbb{E}\left|h_n\left(S_{1,1,u}^{*}\right)\right|q\left(u;r,\alpha ,\lambda \right)du\le C+A{\mu }_2{\int }_0^1\mathbb{E}\left[{\left(N_{1,1,u}^{*}\right)}^2\right]q\left(u;r,\alpha ,\lambda \right)du.$$

According to Equation (66) one has

$${\int }_0^1\mathbb{E}\left[{\left(N_{1,1,u}^{*}\right)}^2\right]q\left(u;r,\alpha ,\lambda \right)du={\int }_0^1\left(\mathbb{E}\left[{\left(G_{1,1,u/(1-u)}^{*}\right)}^2\right]+\mathbb{E}\left[G_{1,1,u/(1-u)}^{*}\right]\right)q\left(u;r,\alpha ,\lambda \right)du.$$

The latter integral is finite because one can take $h\left(x\right)=x$ and $h\left(x\right)=x^2/2$ in Equation (57) and invoke Equation (59). Then, it is possible to use the dominated convergence theorem once again to establish Equation (72).

Now, combining Equation (58) and Equation (70) leads for any $h\in {\mathcal{H}}_2$ to the relation

$$\begin{array}{c}\mathbb{E}\left[h\left(S_{r,\alpha ,p}^{*}\right)\right]-\mathbb{E}\left[h\left(U_{r,\alpha ,p/(1-p)}^{*}\right)\right]\hfill \\ \hfill ={\int }_0^1\left(\mathbb{E}\left[h\left(S_{1,1,u}^{*}\right)\right]-\mathbb{E}\left[h\left(\frac{1-u}{u}{U}_{1,1,1}^{*}\right)\right]\right)q\left(u;r,\alpha ,\frac{p}{1-p}\right)du.\end{array}$$

(73)

Note that a random variable $N_{1,1,u}^{*}$ follows the geometric distribution $Geom\left(u\right)$ with parameter $u\in (0,1)$. For each $h\in {\mathcal{H}}_2$ and any $u\in (0,1)$, by Theorem 3 and in view of $d_{{\mathcal{H}}_2}$ homogeneity, we obtain

$$\begin{array}{c}\left|\mathbb{E}\left[h\left(S_{1,1,u}^{*}\right)\right]-\mathbb{E}\left[h\left(\frac{1-u}{u}{U}_{1,1,1}^{*}\right)\right]\right|\le d_{{\mathcal{H}}_2}\left(S_{1,1,u}^{*},\frac{1-u}{u}{U}_{1,1,1}^{*}\right)\hfill \\ \hfill ={\left(\frac{1-u}{u}\right)}^2{d}_{{\mathcal{H}}_2}(W_u,Y)\le {\left(\frac{1-u}{u}\right)}^2\left(\frac{u}{1-u}\right)\frac{{\mu }_2}{2}=\left(\frac{1-u}{u}\right)\frac{{\mu }_2}{2}.\end{array}$$

(74)

Employing Equations (73), (74) and (62) one deduces

$$d_{{\mathcal{H}}_2}(S_{r,\alpha ,p}^{*},U_{r,\alpha ,p/(1-p)}^{*})\le \frac{{\mu }_2}{2}{\int }_0^1\left(\frac{1-u}{u}\right)q\left(u;r,\alpha ,\frac{p}{1-p}\right)du=\frac{{\mu }_2}{2}\mathbb{E}\left[G_{r,p/(1-p)}^{1/\alpha }\right].$$

(75)

Equation (65) implies by virtue of $d_{{\mathcal{H}}_2}$ homogeneity that

$$d_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},U_{r,\alpha ,1}^{*})={\left(\frac{p}{1-p}\right)}^{2/\alpha }{d}_{{\mathcal{H}}_2}(S_{r,\alpha ,p}^{*},U_{r,\alpha ,p/(1-p)}^{*}).$$

(76)

Combining Equations (59), (75) and (76) we conclude that the right-hand side of Equation (67) is an upper bound for $d_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},U_{r,\alpha ,1}^{*})$.

Choosing $h\left(x\right)=x^2/2$ in Equation (73), upon employing Equation (52) and Equation (62) one infers:

$$\begin{array}{c}{d}_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},G_{r,1}^{1/\alpha })\ge \hfill \\ \ge \frac{1}{2}{\left(\frac{p}{1-p}\right)}^{2/\alpha }\left|{\int }_0^1\left(\mathbb{E}\left[{\left(S_{1,1,u}^{*}\right)}^2\right]-{\left(\frac{1-u}{u}\right)}^2\mathbb{E}\left[{\left(U_{1,1,1}^{*}\right)}^2\right]\right)q\left(u;r,\alpha ,\frac{p}{1-p}\right)du\right|=\\ \hfill =\frac{{\mu }_2}{2}{\left(\frac{p}{1-p}\right)}^{2/\alpha }{\int }_0^1\left(\frac{1-u}{u}\right)q\left(u;r,\alpha ,\frac{p}{1-p}\right)du=\frac{{\mu }_2}{2}{\left(\frac{p}{1-p}\right)}^{2/\alpha }\mathbb{E}\left[G_{r,\alpha ,p/(1-p)}^{*}\right].\end{array}$$

Using Equation (59) once again, we see that the right-hand side of Equation (67) is a lower bound for $d_{{\mathcal{H}}_2}(W_{r,\alpha ,p}^{*},U_{r,\alpha ,1}^{*})$. □

7. Inverse to Equilibrium Transformation

The development of Stein’s method is closely connected with various transformations of distributions. Let a random variable $W\ge 0$ and $0<\mu =\mathbb{E}\left[W\right]<\infty$. Then, one says that a random variable $W^s$ has the W-size biased distribution if for all f such that $\mathbb{E}\left[Wf\right(W\left)\right]$ exists

$$\mathbb{E}\left[Wf\left(W\right)\right]=\mu \mathbb{E}\left[f\left(W^s\right)\right].$$

The connection of this transformation with Stein’s equation was considered in [50,51]. It was pointed out in [51] that this transformation works well for combinatorial problems, such as counting the number of vertices in a random graph having prespecified degrees, see also [52]. In [53], another transformation was introduced. Namely, if a random variable W has mean zero and variance ${\sigma }^2\in (0,\infty )$, then the authors of [53] write (Definition 1.1) that a variable $W^{*}$ has W-zero biased distribution whenever, for all differentiable f such that $\mathbb{E}Wf\left(W\right)$ exists, the following relation holds

$$\mathbb{E}\left[Wf\left(W\right)\right]={\sigma }^2\mathbb{E}\left[f^{\prime }\left(W^{*}\right)\right].$$

This definition is inspired by an equation $\mathbb{E}\left[Wf\left(W\right)\right]={\sigma }^2\mathbb{E}\left[f^{\prime }\left(W\right)\right]$ characterizing the normal law $N(0,{\sigma }^2)$. The authors of [53] explain that $W^{*}$ always exists if $\mathbb{E}\left[W\right]=0$ and $\mathsf{var}W\in (0,\infty )$. Zero-based coupling for products of normal random variables is treated in [54]. In Sec. 2 of [30], it is demonstrated that the gamma distribution is uniquely characterised by the property that its size-biased distribution is the same as its zero-biased distribution. Two generalizations of zero biasing were proposed in [55], see p. 104 of that paper for discussion of these transformations. We refer also to survey [56].

Now, we turn to the equilibrium distribution transformation introduced in [33] and concentrate on approximation of the law under consideration by means of an exponential law, see the corresponding Definition 1 in Section 2.

According to the second part of Theorem 2.1 of [33] (in our notation), for $Z\sim Exp\left(1\right)$ and non-negative random variable X with $\mathbb{E}\left[X\right]=1$ and $\mathbb{E}\left[X^2\right]<\infty$ the following estimate holds

$$d_{{\mathcal{H}}_1}(X,Z)\le 2\mathbb{E}|X^e-X|,$$

and at the same time

$$d_{{\mathcal{H}}_1}(X^e,Z)\le \mathbb{E}|X^e-X|.$$

(77)

The authors of [33] also proved that $d_{\mathcal{K}}(X^e,Z)\le \mathbb{E}|X^e-X|$. Notice that the estimate for $d_{{\mathcal{H}}_1}(X^e,Z)$ is more precise than that for $d_{{\mathcal{H}}_1}(X,Z)$.

Now we turn to Equation (77) and demonstrate how to find the distribution of X when we know the distribution of $X^e$. In other words, we concentrate on the inverse of an equilibrium distribution transformation.

Assume that $\mathbb{E}\left[X\right]>0$. Recall that a random variable $X^e$ exists if $F^e\left(x\right)$ appearing in Equation (16) is a distribution function. The latter statement for $\mathbb{E}\left[X\right]>0$ is equivalent to nonnegativity of X. Indeed, for non-negative X, $F^e\left(x\right)$ coincides with a distribution function having a density (15). If $F^e\left(x\right)$ is a distribution function and $\mathbb{E}\left[X\right]>0$ in Equation (16), then $F^e\left(x\right)\ge 0$ for $x<0$ only if $F\left(x\right)=0$ for $x<0$.

Thus a random variable $X^e$ has a (version of) density $p^e\left(x\right)$ introduced in Equation (15). Obviously, the function $p^e\left(x\right)$ has the following properties. It is nonincreasing on $[0,\infty )$ and $p^e\left(x\right)=0$ for $x<0$. This density is right-continuous on $[0,\infty )$ and consequently $p^e\left(0\right)<\infty$. Now, we are able to provide a full description of the class of densities for random variables $X^e$ relevant to all non-negative X with positive mean.

Lemma 6.

Let a non-negative random variable $X^e$ have a version of density (with respect to the Lebesgue measure)$p^e\left(x\right)$, $x\in \mathbb{R}$, such that this function is nonincreasing on $[0,\infty )$, $p^e\left(x\right)=0$ for $x<0$, and there is finite ${lim}_{x\to 0+}{p}^e\left(x\right)$. Then, there exists a unique preimage of $X^e$ distribution having the distribution function F continuous at $x=0$. Namely,

$$F\left(x\right)=\left\{\begin{array}{cc}1-\frac{p^e\left(x\right)}{p^e\left(0\right)},\hfill & x\ge 0,\hfill \\ 0,\hfill & x<0.\hfill \end{array}\right.$$

(78)

Proof. 

First of all, note that $p^e\left(0\right)>0$ as otherwise $p^e\left(x\right)=0$ for all $x\in \mathbb{R}$ ($p^e$ is a nonincreasing function on $[0,\infty )$). We also know that there exist a left-sided limit and a right-sided limit of $p^e$ at each point $x\in (0,\infty )$ as well as the right-sided limit of $p^e$ at $x=0$. The set of discontinuity points of $p^e$ is at most countable, and we can take a version which is right continuous at each point of $[0,\infty )$. Then, Equation (78) introduces a distribution function. Consider a random variable X with distribution function F and check the validity of Equation (14).

The integration by a parts formula yields, for any $b>0$,

$$1\ge {\int }_0^b{p}^e\left(x\right)dx=b{p}^e\left(b\right)+p^e\left(0\right){\int }_0^{b}xdF\left(x\right).$$

(79)

Summands in the right-hand side of Equation (79) are non-negative. Therefore, for any $b>0$, $\mathbb{E}\left[X\mathbb{I}(X\le b)\right]\le 1/p^e\left(0\right)$. Hence, the monotone convergence theorem implies that $\mathbb{E}\left[X\right]$ is finite. According to Equation (78)

$$b{p}^e\left(b\right)/p^e\left(0\right)=b(1-F\left(b\right))=b\mathbb{P}(X>b)\to 0,b\to \infty ,$$

(80)

since $\mathbb{E}\left[X\right]<\infty$. Taking in the Equation (79) limit as $b\to \infty$, one obtains $1=p^e\left(0\right)\mathbb{E}\left[X\right]$. Now, we are ready to verify Equation (14). For any Lipschitz function f, $\mathbb{E}\left[f\right(X\left)\right]$ is finite and

$$\mathbb{E}\left[f\left(X\right)\right]={\int }_0^{\infty }f\left(x\right)dF\left(x\right)=-\frac{1}{p^e\left(0\right)}{\int }_0^{\infty }f\left(x\right)d{p}^e\left(x\right).$$

Taking into account Equation (80), we infer that $f\left(b\right)p^e\left(b\right)\to 0$ as $b\to \infty$. Consequently, applying integration by parts once again (f has bounded variation), we obtain

$$\begin{array}{c}\mathbb{E}\left[X\right]\mathbb{E}\left[f^{\prime }\left(X^e\right)\right]=\frac{1}{p^e\left(0\right)}{\int }_0^{\infty }{f}^{\prime }\left(x\right)p^e\left(x\right)dx=\frac{1}{p^e\left(0\right)}{\int }_0^{\infty }{p}^e\left(x\right)df\left(x\right)\hfill \\ \hfill =\frac{1}{p^e\left(0\right)}\left[-f\left(0\right)p^e\left(0\right)-{\int }_0^{\infty }f\left(x\right)d{p}^e\left(x\right)\right]=\mathbb{E}\left[f\left(X\right)\right]-f\left(0\right).\end{array}$$

Uniqueness of X distribution corresponding to $X^e$ is a consequence of Equation (15) and continuity of $F\left(x\right)$ at $x=0$. Indeed, assume that for $X_1$ and $X_2$ one has $X_1^e=X_2^e$. Then, Equation (15) yields that for almost all $x\ge 0$,

$$\frac{1}{\mathbb{E}\left[X_1\right]}\mathbb{P}(X_1>x)=\frac{1}{\mathbb{E}\left[X_2\right]}\mathbb{P}(X_2>x),$$

(81)

and therefore $\mathbb{P}(X_1>x)=c\mathbb{P}(X_2>x)$, where c is a positive constant (the equilibrium distribution in Definition 1 is introduced for random variables with positive expectation only). Since $\mathbb{P}(X_1=0)=\mathbb{P}(X_2=0)=0$, one has $\mathbb{P}(X_1>0)=\mathbb{P}(X_2>0)$. Let $x_n\to 0+$, $n\to \infty$, where the points $x_n$ belong to the set considered in Equation (81) to ensure that $c=1$. Thus, distributions of $X_1$ and $X_2$ coincide. □

Remark 6.

Let $X_p$ be the Bernoulli random variable taking values 1 and 0 with probabilities p and $1-p$, respectively. Then, it is easily seen that the distribution of $X_p^e$ is uniform on $[0,1]$. Thus, in contrast to Lemma 6, without assumption of continuity of F at a point $x=0$ one can not guarantee, in general, the preimage uniqueness for the inverse transformation to the equilibrium one.

In the proof of Lemma 6, we find out that $\mathbb{E}\left[X\right]=1/p^e\left(0\right)$. Set $\lambda =p^e\left(0\right)$, $Z\sim Exp\left(\lambda \right)$. Then, $\mathbb{E}\left[X\right]=\mathbb{E}\left[Z\right]$. Further, we suppose that this choice of $\lambda$ is made.

Recall that random variables U and V are stochastically ordered if either $\mathbb{P}(U\le x)\le \mathbb{P}(V\le x)$, for every $x\in \mathbb{R}$, or the opposite inequality holds (for all $x\in \mathbb{R}$). Now, we clarify one of the Theorem 2.1 of [33] statements (see also Theorem 3 [22], where the result similar to Theorem 2.1 of [33] is formulated employing the generalized distributions).

Theorem 6.

Let a random variable $X^e$ satisfy conditions of Lemma 6, and $\mathbb{E}\left[X^e\right]<\infty$ and X be a preimage of the equilibrium transformation. Then, Equation (77) holds. Moreover, the inequality becomes an equality when X and $X^e$ are stochastically ordered.

Proof. 

Apply the Stein Equation (10) along with equilibrium transformation (14). Then, in light of $\mathbb{E}\left[X\right]=\frac{1}{\lambda }$ and $\mathbb{E}{f}_h\left(X\right)-f_h\left(0\right)=\frac{1}{\lambda }\mathbb{E}{f}_h^{\prime }\left(X^e\right)$, we can write

$$\begin{array}{c}\left|\mathbb{E}\left[h\left(X^e\right)\right]-\mathbb{E}\left[h\left(Z\right)\right]\right|=\left|\mathbb{E}\left(f_h^{\prime }\left(X^e\right)-\lambda f_h\left(X^e\right)\right)+\lambda f\left(0\right)\right|\hfill \\ \hfill =\lambda \left|\mathbb{E}\left(f_h\left(X^e\right)-f_h\left(X\right)\right)\right|\le \lambda ||f_h^{\prime }{||}_{\infty }\mathbb{E}|X^e-X|\le ||h^{\prime }{||}_{\infty }\mathbb{E}|X^e-X|.\end{array}$$

(82)

The last inequality in (82) is true due to Lemma 2. Now, we demonstrate that equality in (82) can be attained. Taking $h\left(x\right)=x-\frac{1}{\lambda }$, we have a solution $f_h\left(x\right)=-\frac{1}{\lambda }x$ of Equation (12). Then,

$$\left|\mathbb{E}\left[h\left(X^e\right)\right]-\mathbb{E}\left[h\left(Z\right)\right]\right|=\lambda \left|\mathbb{E}\left(f_h\left(X^e\right)-f_h\left(X\right)\right)\right|=\left|\mathbb{E}(X^e-X)\right|.$$

Employing the integration by parts formula, one can show that the expression in the right-hand side of the last equality is equal to the Kantorovich distance between X and $X^e$ when these variables are stochastically ordered. Note that $x(1-F(x\left)\right)\to 0$, $x(1-F^e\left(x\right))\to 0$ as $x\to \infty$ and $xF\left(x\right)\to 0$, $x{F}^e\left(x\right)\to 0$ as $x\to -\infty$ because $\mathbb{E}\left[X\right]$ and $\mathbb{E}\left[X^e\right]$ are finite. Thus,

$$\begin{array}{c}\left|\mathbb{E}\left[X^e\right]-\mathbb{E}\left[X\right]\right|=\left|{\int }_{\mathbb{R}}x\left(d{F}_{X^e}\left(x\right)-d{F}_X\left(x\right)\right)\right|\hfill \\ \hfill =\left|-{\int }_{\mathbb{R}}\left(F_{X^e}\left(x\right)-F_X\left(x\right)\right)dx\right|={\int }_{\mathbb{R}}\left|F_{X^e}\left(x\right)-F_X\left(x\right)\right|dx,\end{array}$$

since $F_{X^e}\left(x\right)\ge F_X\left(x\right)$ (or ≤) for all $x\in \mathbb{R}$. It is well-known that the Kantorovich distance is the minimal one for the metric $\tau (U,V)=\mathbb{E}|U-V|$ (see, e.g., [9], Ch. 1, §1.3). Therefore,

$${\int }_{\mathbb{R}}\left|F_{X^e}\left(x\right)-F_X\left(x\right)\right|dx=inf\mathbb{E}|U-V|,$$

where the infimum has taken over all joint laws $(U,V)$ such that ${\mathbb{P}}_U={\mathbb{P}}_{X^e}$ and ${\mathbb{P}}_V={\mathbb{P}}_X$ (see also Remark 2 and [10], Corollary 5.3.2). Consequently, in the framework of Theorem 6, $\left|\mathbb{E}\left[X^e\right]-\mathbb{E}\left[X\right]\right|=\mathbb{E}|X^e-X|$. □

Remark 7.

One can show that by means of Lemma 2 and Equation (82) it is possible to provide an estimate

$$d_{\mathcal{K}}(X^e,Z)\le \lambda \mathbb{E}|X^e-X|.$$

(83)

For each function h belonging to $\mathcal{K}$, in a similar way to Equation (82), one can apply Equation (10) together with equilibrium transformation. Now, it is sufficient to study the Stein equation with right derivative. Formula (13) gives a solution of the Stein equation according to Lemma 2. Note that for $f_h$, the right derivative coincides almost everywhere with the derivative, and the law of $X^e$ is absolutely continuous according to Equation (15). Thus, for the Lipschitz function $f_h$ (see Lemma 2), one can use an equilibrium transformation.

Example 1.

Consider the distribution functions $F_{\epsilon }\left(x\right)$ of random variables $X_{\epsilon }$, taking values $\epsilon$ and $2-\epsilon$ with probabilities $1/2$, $0<\epsilon <1$. Formula (15) yields that $X_{\epsilon }^e$ has the following piece-line structure

$$F_{\epsilon }^e\left(x\right)=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}x<0,\hfill \\ x,\hfill & \mathrm{if}0\le x<\epsilon ,\hfill \\ x/2+\epsilon /2,\hfill & \mathrm{if}\epsilon \le x<2-\epsilon ,\hfill \\ 1,\hfill & \mathrm{if}2-\epsilon \le x.\hfill \end{array}\right.$$

If $\epsilon \ge 1/2$ then, for all $x\in \mathbb{R}$, the following inequality holds: $F_{\epsilon }^e\left(x\right)\ge F_{\epsilon }\left(x\right)$, i.e., $X_{\epsilon }$ and $X_{\epsilon }^e$ are stochastically ordered. We see that for $\epsilon <1/2$, the inequality is violated in the right neighborhood of a point $\epsilon$. Thus, there are beside the stochastically ordered pairs (X, $X^e$) also those of a different kind.

Now, we turn to another example of stochastically ordered X and $X^e$.

Example 2.

Take $X^e$ having the Pareto distribution. The notation $X^e\sim Pareto(\alpha ,\beta )$ means that $X^e$ has a density $f^e\left(x\right)=\frac{\alpha {\beta }^{\alpha }}{{(x+\beta )}^{\alpha +1}}$ ($x\ge 0$) and the corresponding distribution function $F^e\left(x\right)=1-{\left(\frac{\beta }{x+\beta }\right)}^{\alpha }$, where $x\ge 0$, $\alpha >0,\beta >0$.

Further, we consider only $\alpha >1$, since in this case there exists finite $\mathbb{E}\left[X^e\right]=\frac{\beta }{\alpha -1}$. By means of Lemma 6, we obtain the distribution of the preimage of the equilibrium transformation

$$F\left(x\right)=1-\frac{f^e\left(x\right)}{f^e\left(0\right)}=1-\frac{\alpha {\beta }^{\alpha }}{{(x+\beta )}^{\alpha +1}}\frac{{\beta }^{\alpha +1}}{\alpha {\beta }^{\alpha }}=1-{\left(\frac{\beta }{x+\beta }\right)}^{\alpha +1},x\ge 0.$$

Thus one can state that $X\sim Pareto(\alpha +1,\beta )$. It is not difficult to see that $F^e\left(x\right)\le F\left(x\right)$ for $x\in \mathbb{R}$, i.e., the random variables $X^e$ and X are stochastically ordered. Due to Theorem 6, one has

$$d_{{\mathcal{H}}_1}(X^e,Z)=\mathbb{E}|X^e-X|=\mathbb{E}\left[X^e\right]-\mathbb{E}\left[X\right]=\frac{\beta }{\alpha -1}-\frac{\beta }{\alpha }=\frac{\beta }{\alpha (\alpha -1)},$$

(84)

$$d_{\mathcal{K}}(X^e,Z)\le \frac{\alpha }{\beta }\mathbb{E}|X^e-X|=\frac{1}{\alpha -1}.$$

In such a way we find the bound for the Kolmogorov distance between the distributions $Pareto(\alpha ,\beta )$ and $Exp(\alpha /\beta )$. This relation demonstrates the convergence rate of $d_1(X^e,Z)$ to zero as $\alpha \to \infty$. The estimate is nontrivial for $\alpha >2$.

Remark 8.

It is interesting that estimation of the proximity of the Pareto law to the Exponential one became important in signal processing, see [34] and references therein. Let $X\sim Pareto(\alpha ,\beta )$, where $\alpha >0$, $\beta >0$, and $Z\sim Exp\left(\lambda \right)$. In [34], the author indicates that the Pinsker–Csiszár inequality was employed to derive

$$d_{\mathcal{K}}(X,Z)\le \sqrt{2D_{KL}(X||Z)},$$

(85)

where $D_{KL}(X||Z)$ is the Kullback–Leibler divergence between laws of X and Z. More precisely, in the left-hand side of Equation (85) one can write the total variation distance $d_{TV}(X,Z)$ between distributions of X and Z. Clearly, $d_{\mathcal{K}}(X,Z)\le d_{TV}(X,Z)$. By evaluating $D_{KL}(X||Z)$ and performing an optimal choice of parameter $\lambda$, it was demonstrated (formula (19) in [34]) that, for $\alpha >1$ and any $\beta >0$,

$$d_{\mathcal{K}}(X,Z)\le \sqrt{\frac{2}{\alpha (\alpha -1)}}$$

(86)

if $\lambda =\frac{\alpha -1}{\beta }$. The author of [34] on page 8 writes that in his previous work [57] the inequality

$$d_{\mathcal{K}}(X,Z)\le \frac{3}{\alpha }$$

(87)

was established with the same choice of $\lambda$. Next, he also writes that “in the most cases $\alpha >2$” and notes that the estimate in Equation (86) involving the Kullback–Leibler divergence is more precise for $\alpha >\frac{9}{7}$ than the estimate in Equation (87) obtained by the Stein method. Moreover, on page 4 of [34] we read: “The problem with the Stein approach is that the bounds do not suggest a suitable way in which, for a given Pareto model, an appropriate approximating Exponential distribution can be specified”. However, we have demonstrated that application of the inverse equilibrium transformation together with the Stein method permits indicating, whenever $\alpha >2$, the corresponding Exponential distribution with proximity closer than the right-hand sides of Equation (86) and Equation (87) can provide.

8. Conclusions

Our principle goal was to find the sharp estimates of the proximity of random sums distributions to exponential and more general laws. This goal is achieved when we employ the probability metric $d_{{\mathcal{H}}_2}$. Thus, it would be valuable to find the best possible approximations of random sums distributions by means of specified laws using the metrics ${\zeta }_s$ of order $s>0$. The results of [32] provide the basis for this approach.

There are various complementary refinements of the Rényi theorem. One approach is related to the employment of Brownian motion. It is interesting that in [58] (p. 1071) the authors proposed an explanation of the Rényi theorem involving the embedding theorem. We provide a little bit different complete proof. Let $X_1,X_2,\dots$ be i.i.d. random variables with mean $\mu :=\mathbb{E}{X}_1$ and ${\sigma }^2:=\mathsf{var}{X}_1<\infty$, whereas $S_n,n\in \mathbb{N},$ denote the corresponding partial sums. According to Theorem 12.6 of [59], which is due to A.V. Skorokhod and V. Strassen, there exists a standard Brownian motion $B\left(t\right),t\ge 0,$ (perhaps it is defined on an extension of initial probability space) such that

$$\frac{1}{\sqrt{t}}\underset{0\le u\le t}{sup}|S_{\left[u\right]}-\mu u-\sigma B\left(u\right)|\stackrel{\mathbb{P}}{\to }0,t\to \infty ,$$

(88)

and

$$\underset{t\to \infty }{lim}\frac{S_{\left[t\right]}-\mu t-\sigma B\left(t\right)}{\sqrt{2tloglogt}}=0a.s.,$$

(89)

where $\stackrel{\mathbb{P}}{\to }$ stands for convergence in probability, and a.s. means almost surely. Thus, in light of Equation (89), we can write, for $t\ge 0$,

$$S_{\left[t\right]}=\mu t+\sigma B\left(t\right)+R\left(t\right),$$

(90)

where ${sup}_{0\le u\le t}R\left(u\right)/\sqrt{t}\stackrel{\mathbb{P}}{\to }0$ and $R\left(t\right)/\sqrt{2tloglogt}\to 0$ a.s. when $t\to \infty$. Substitute $N_p$ (see Equation (2)) in Equation (90) instead of t. It is easily seen that $N_p\stackrel{\mathbb{P}}{\to }\infty$ (i.e., for each $t>0$, one has $\mathbb{P}(N_p\le t)\to 0$ as $p\to 0+$) and by means of characteristic functions one can verify that $p{N}_p\stackrel{\mathcal{D}}{\to }Z$ as $p\to 0+$, where $Z\sim Exp\left(1\right)$. Therefore, $\mu p{N}_p\stackrel{\mathcal{D}}{\to }\mu Z$, $p\to 0+$. In the proof of Lemma 4, we showed (Equation (24)) that $\mathbb{E}\left[N_p\right]=(1-p)/p$. Consequently,

$$\begin{array}{c}\mathsf{var}\left[pB\left(N_p\right)\right]=p^2\mathbb{E}\left[B{\left(N_p\right)}^2\right]=p^2\sum _{k=0}^{\infty }\mathbb{E}\left[B{\left(k\right)}^2\right]p{(1-p)}^k\hfill \\ \hfill =p^2\sum _{k=0}^{\infty }kp{(1-p)}^k=p^2\mathbb{E}\left[N_p\right]=p^2\frac{1-p}{p}=p(1-p)\to 0,p\to 0+.\end{array}$$

Hence, $p\sigma B\left(N_p\right)\stackrel{\mathbb{P}}{\to }0$ as $p\to 0+$. Now, we demonstrate that $pR\left(N_p\right)\stackrel{\mathbb{P}}{\to }0,p\to 0+.$ For any $\epsilon >0$ and any $t>0$,

$$\begin{array}{c}\mathbb{P}(p|R\left(N_p\right)|>\epsilon )\le \mathbb{P}(p|R\left(N_p\right)|>\epsilon ,N_p\le t)+\mathbb{P}(N_p>t)\hfill \\ \hfill \le \mathbb{P}(p\underset{0\le u\le t}{sup}|R\left(u\right)|>\epsilon )+\mathbb{P}(N_p>t).\end{array}$$

In light of Equation (88), for arbitrary $\gamma >0$ and $\epsilon >0$, one can take $t_0=t_0\left(\gamma \right)$ such that $\mathbb{P}({sup}_{0\le u\le t_0}|R\left(u\right)|>\epsilon \sqrt{t_0})<\gamma /2$. Then, for any $0<p\le 1/\sqrt{t_0}$, we obtain

$$\mathbb{P}(p\underset{0\le u\le t_0}{sup}|R\left(u\right)|>\epsilon )<\gamma /2.$$

Since $N_p\stackrel{\mathbb{P}}{\to }\infty$, we can find $p_0>0$ such that $\mathbb{P}(N_p>t_0)<\gamma /2$ if $0<p\le p_0$. Therefore, $R\left(N_p\right)\stackrel{\mathbb{P}}{\to }0$ as $p\to 0+$. The Slutsky lemma yields the desired relation

$$p{S}_{N_p}\stackrel{\mathcal{D}}{\to }\mu Z,p\to 0+,$$

which implies Equation (3). However, it seems that there is no clear intuitive reason why the law of the random sum converges to an exponential in the Rényi theorem. Moreover, in Ch. 3, Sec. 2 “The Rényi Limit Theorem” of [20] (see Sec. 2.1 “Motivation”), one can find examples demonstrating that intuition behind the Rényi theorem is poor.

Actually, relation (90) leads to refinements of Equation (3). In [58], it is proved that if $X_1$ has finite exponential moments and other specified conditions are satisfied then there exists a more sophisticated approximation for distribution of $W_p$, and its accuracy is estimated. The results are applied to the study of $M/G/1$ queue for both light-tailed and heavy-tailed service time distributions. Note that in [58], Section 5, the authors study the model where the distribution of $X_1$ can depend on p. For future research, it would be desirable to establish analogues of our theorems for such a model.

The results concerning the accuracy of approximating a distribution under consideration by an exponential law are applicable to some queuing models. Let, for a queue $M/G/1$, the inter-arrival times follow $Exp\left(\lambda \right)$ distribution and S stand for the general service time. Introduce the stationary waiting time W and define $\rho :=\lambda \mathbb{E}\left[S\right]$ to be its load. Due to [60], if $\mathbb{E}\left[S^3\right]<\infty$ then $(1-\rho )W\stackrel{\mathcal{D}}{\to }Z$ as $\rho \to 1$, where $Z\sim Exp\left(1\right)$. Theorem 3.1 of [45] contains an upper bound of $d_{{\mathcal{H}}_1}(W_p,Z)$, where $Z\sim Exp\left(1\right)$. This estimate is used by the authors for analysis of queueing systems with a single server. It would be interesting to obtain the sharp approximations in the framework of queueing systems.

For the model of exchangeable random variables, Theorem 2 in Section 2 ensures the weak convergence of distributions under consideration to specified mixture of explicitly indicated laws. Theorem 3 proves the sharp convergence rate estimate to this limit law by means of the ideal probability metric of the second order. It would be worthwhile to establish such an estimate of the distributions proximity applying the Lévy–Prokhorov distance because convergence in this metric is equivalent to the weak convergence of distributions of random variables. All the more, at present there is no unified theory of probability metrics. In this regard, one can mention Proposition 1.2 of [17] stating that if a random variable Z has the Lebesgue density bounded by C then, for any random variable Y,

$$d_{\mathcal{K}}(Y,Z)\le \sqrt{C{d}_{{\mathcal{H}}_1}(Y,Z)}.$$

However, this estimate only gives the sub-optimal convergence rates. We also highlight the important total variation distance $d_{TV}$. The authors of [61] study the sum $W:={\sum }_{j\in J}{X}_j$, where $\{X_j,j\in J\}$ is a family of locally dependent non-negative integer-valued random variables. Using the perturbations of Stein’s operator, they establish the upper bounds for $d_{TV}(W,M)$ where the law of M is a mixture of Poisson distribution and either binomial or negative binomial distribution. It would be desirable to obtain the sharp estimates and, moreover, consider a more general model where the set of summation is random. In this connection, it seems helpful to employ the paper [62], where the authors proved results concerning the weak convergence of distributions of statistics constructed from samples of random size. In addition, it would be interesting to extend these results to stratified samples by invoking Lemma 1 of [63].

Special attention is paid to various generalizations of the geometric sums. In Theorem 3.3 of [64], the authors consider random sums with summation index $T_n:=Y_1+\dots +Y_n$, where $Y_1,Y_2,\dots$ are i.i.d. random variables following the geometric law $Geom\left(p\right)$, see Equation (2). Then, they show that $S_{T_n}/\mathbb{E}\left[S_{T_n}\right]$ converge in distribution to the gamma law with certain parameters as $p\to 0+$. In [62], it is demonstrated that the Linnik and the Mittag–Leffler laws arise naturally in the framework of limit theorems for random sums. Hopefully, in future the complete picture of limit laws involving general theory of distributions mixtures will appear. In addition, it is desirable to study various models of random sums of dependent random variables. On this track, it could be useful to consider the decompositions of exchangeable random sequences extending the fundamental de Finetti theorem, see, e.g., [65].

One can try to generalize the results of Section 7 for accumulative laws proposed in [66]. These laws are akin to both the Pareto distribution and the lognormal distribution. In addition, we refer to [43] where the “variance-gamma distributions” were studied. These distributions form a four-parameter family and comprise as special and limiting cases the normal, gamma and Laplace distributions. Employment of these distributions permits enlarging a range of applications in modeling and fitting real data.

To complete the indication of further research directions, we note that the next essential and nontrivial step is to establish the limit theorem in functional spaces for processes generated by a sequence of random sums of random variables. For such stochastic processes, one can obtain the analogues of the classical invariance principles.