Any Distribution on the Positive Real Line as a Limit of Random Sums

Abstract

We prove that any positive random variable may be the limit of the random sums of independent identically distributed random variables. Thus, the limit distribution in the case of summing a random number of random variables may not be stable. The lack of stability in the limit distribution significantly distinguishes the summation scheme for a random number of random variables from the classical summation scheme. Moreover, the analogs of stable distributions in the summation of a random number of terms also turn out to be limit distributions in a suitable summation scheme.

1. Introduction

Limit theorems in probability theory probably originated with the result now recognized as the De Moivre–Laplace theorem, originally obtained by De Moivre in 1733. Further developments in this area led to the emergence of a certain class of theorems under the general name “central limit theorem” and numerous generalizations of these theorems. The main results in this direction are presented in the monographs by B.V. Gnedenko and A.N. Kolmogorov [1], V.V. Petrov [2], and others. Most results related to limit theorems for sums of independent identically distributed random variables are devoted specifically to the description of limit laws. It turns out that limit laws possess a well-known stability property. Roughly speaking, if we define an operator on the space of probability distributions by passing from a distribution to its convolution with itself and the required normalization and centralization, then the limit distribution turns out to be a fixed point of this operator.

Later, interest arose in limit theorems for sums of a random number of independent identically distributed random variables. Let us note here the articles by R.L. Dobrushin [3] and B.V. Gnedenko [4,5], as well as the monograph by B.V. Gnedenko and V.Yu. Korolev [6]. More recent results are given in [7].

For the sums of random variables, the topic of this communication, many results are similar to classical limit theorems in probability theory. However, there are, nevertheless, fundamental differences, one of which we would like to point out. It turns out that in the case of the summation of positive random variables with finite mean, a suitably chosen sequence of random sums yields a class of limit laws that is large and coincides with all probability distributions concentrated on the positive semiaxis. As noted above, the classical summation scheme primarily focuses on finding limit distributions. It is later proven that limit distributions are stable. In this paper, we demonstrate that the stability of a distribution implies it is also a limit distribution, but the converse is not true. (Since several centralization methods are possible when summing a random number of random variables, we will only allow for normalization of the values. When discussing stability, we refer to strict stability. Thus, a random variable X is called stable when summing a random number $\nu$ of independent identically distributed random variables if and only if $X\stackrel{d}{=}a\left(\nu \right){\sum }_{j=1}^{\nu }{X}_j$. Here $\stackrel{d}{=}$ denotes equality in distribution, and normalization $a\left(\nu \right)$ is a positive number depending on the mean of $\nu$.) As a consequence, we obtain that the limit distributions for sums of a random number of random variables may no longer be fixed points of the corresponding operators.

2. Main Result

Theorem 1.

Let $X=(X_1,X_2,\dots ,X_n,\dots )$ be a sequence of independent identically distributed (i.i.d.) positive random variables with unit first moment; i.e., $\mathrm{I}\mathrm{E}{X}_i=1,i=1,2,\dots ,n,\dots$. Suppose that Y is an arbitrary positive random variable. Then there exists a family $\{{\nu }_m,m\ge 1\}$ of non-negative integer-valued random variables ${\nu }_m$, depending on the distribution of Y only, such that

1.

The family $\{{\nu }_m,m\ge 1\}$ and X are independent;

2.

${\nu }_m$ tends in probability to infinity as $m\to \infty$;

3.

The random sum ${\sum }_{k=1}^{{\nu }_m}{X}_k$ tends to Y in distribution as $m\to \infty$.

For the proof, we need the following two lemmas.

Lemma 1.

Let $\psi \left(s\right)$ be the Laplace transform of a distribution on the positive semiaxis. Then $\mathcal{P}\left(z\right)=\psi (1-z)$ is a probability-generating function.

Proof. 

(see [8]). Suppose that

$$\psi \left(s\right)={\int }_0^{\infty }exp(-sx)d\mathcal{A}\left(x\right),$$

where $\mathcal{A}\left(x\right)$ is a probability distribution function. Therefore

$$\mathcal{P}\left(z\right)=\psi (1-z)={\int }_0^{\infty }exp(-(1-z)x)d\mathcal{A}\left(x\right)=\sum _{k=0}^{\infty }{\displaystyle \frac{z^k}{k!}}{\int }_0^{\infty }{x}^{k}exp(-x)d\mathcal{A}\left(x\right).$$

The series on the right-hand side has non-negative coefficients, and their sum is 1. □

Lemma 2.

Let $\mathcal{Q}\left(z\right)$ be a probability-generating function (p.g.f.). The function $\mathcal{Q}(1-A+Az)$ is a p.g.f. for all $A>1$ if and only if $\mathcal{Q}\left(z\right)=\psi (1-z)$, where $\psi \left(s\right)$ is the Laplace transform of a probability distribution on the positive semiaxis. (The transformation $\mathcal{Q}\left(z\right)\to \mathcal{Q}(1-A+Az)$ with $0<A<1$ can be interpreted as thinning of a random variable with p.g.f. $\mathcal{Q}\left(z\right)$ [9]. While this transform can be applied to every p.g.f., the inverse case with $A>1$ is limited to scalable p.g.f.s only. The relation for the corresponding probability mass functions was obtained in [10]. The class of scalable distributions includes, among others, negative binomial, shifted logarithmic, and discrete stable distributions.)

Proof. 

1. Suppose that $\mathcal{Q}\left(z\right)=\psi (1-z)$, where $\psi \left(s\right)$ is a Laplace transform of a probability distribution on the positive semiaxis. We have

$$\mathcal{Q}(1-A+Az)=\psi \left(A(1-z)\right)={\int }_0^{\infty }exp(-(1-z)x)d\mathcal{A}(x/A)$$

and the statement follows from Lemma 1.

2. Suppose now that the function $\mathcal{Q}(1-A+Az)$ is probability-generating function for all $A>1$. Let $\xi \left(s\right)$ be the Laplace transform of a distribution with unit mean. Then $\mathcal{Q}(1-A(1-\xi (s/A)\left)\right)$ is the Laplace transform of a distribution on the non-negative semiaxis for all $A>1$. Its limit as $A\to \infty$ equals $\mathcal{Q}(1-s)$ and is continuous at the point $s=0$. The function $\psi \left(s\right)=\mathcal{Q}(1-s)$ is continuous at point $s=0$, too. Therefore, $\psi \left(s\right)=\mathcal{Q}(1-s)$ is the Laplace transform of a distribution $\mathcal{A}\left(x\right)$ on the non-negative semiaxis as a limit of Laplace transforms in view of the continuity theorem for Laplace transforms. □

Proof of Theorem 1.

Denote by $\phi \left(s\right)$ the Laplace transform of the positive random variable Y. According to Lemma 1, define a family of p.g.f.s

$${\mathcal{P}}_m\left(z\right)=\phi \left(m(1-z)\right),m>1,$$

(1)

and let $\{{\nu }_m,m>1\}$ be a family of corresponding random variables taking non-negative integer values. Clearly, we can construct the family to be independent of X.

Let $B>0$ be an arbitrary positive number and calculate the probability $\mathrm{I}\mathrm{P}\{{\nu }_m\ge B\}$. We have

$${\mathcal{P}}_m\left(z\right)=\phi \left(m(1-z)\right)=\sum _{k=0}^{\infty }{\displaystyle \frac{m^k{z}^k}{k!}}{\int }_0^{\infty }{x}^{k}exp(-mx)d\mathcal{A}\left(x\right),$$

where $\mathcal{A}\left(x\right)$ is the probability distribution function of Y. Therefore,

$$\mathrm{I}\mathrm{P}\{{\nu }_m=k\}={\displaystyle \frac{m^k}{k!}}{\int }_0^{\infty }{x}^{k}exp(-mx)d\mathcal{A}\left(x\right).$$

From ${lim}_{u\to +\infty }{u}^{k}exp(-u)=0$ it follows that for any fixed $k\ge 0$ and $x>0$,

$$\underset{m\to \infty }{lim}{\displaystyle \frac{m^k{x}^{k}exp(-(m-1)x)}{k!}}=0.$$

Therefore,

$$\sum _{k=0}^{\left[B\right]}\mathrm{I}\mathrm{P}\{{\nu }_m=k\}⟶0\mathrm{as}m\to \infty ,$$

where $\left[B\right]$ denotes the integer part of B. In other words with $\mathrm{I}\mathrm{P}\{{\nu }_m\ge B\}\to 1$ as $m\to \infty$, the random variable ${\nu }_m$ converges to infinity in probability.

Consider the sum ${\sum }_{k=1}^{{\nu }_m}{X}_k/m$ of i.i.d random variables $X_k$. The Laplace transform of its distribution is

$${\mathcal{P}}_m\left(f(s/m)\right)=\phi \left(m(1-f(s/m))\right)⟶\phi \left(s\right).$$

Here $f\left(s\right)$ is the Laplace transform of the distribution of $X_k$, and we have used the fact that $\mathrm{I}\mathrm{E}{X}_k=1$. □

In most applied problems, the number of terms in a sum is random. For example, the number of stock exchange transactions per unit of time is a random variable. The number of insurance claims over a fixed period of time is random. The number of cases during an epidemic is random. Theorem 1 shows that the limit distribution for sums of a random number of terms depends on the distribution of the number of terms, but not on the distribution of the terms themselves. This appears to be a complete analog of the central limit theorem. However, what could these limit distributions be? Theorem 1 shows that, depending on the distribution of the number of positive terms, this distribution can be an arbitrary law on the positive half-line.

Note that Theorem 1 gives possible limit distributions for the random sums, which are not necessarily stable. Finding analogs of stable distributions for sums of a random number of random variables is more complicated. Some results in this direction are given in [11].

In the proofs of Lemma 2 and Theorem 1, some well-known facts from the theories of characteristic functions and Laplace transforms are used. These facts may be found in [12]. The interested reader may find more details, for example, in the books [13,14].

3. Discussion

The first question for discussion concerns our assertion that not every distribution on the positive semiaxis can be stable. Here we present a simple example to support this assertion. For this purpose, we will need one result from citeKKRT: Let $\phi \left(s\right)$ be the Laplace transform of a stable distribution connected to sums of a random number of random variables with p.g.f. ${\mathcal{P}}_m\left(z\right)$. Then

$${\mathcal{P}}_m\left(z\right)=\phi (m{\phi }^{-1}\left(z\right)).$$

(2)

Let us now consider the following function:

$$\phi \left(s\right)={(1+s)}^{-2},s\ge 0.$$

This function is the Laplace transform of probability density $xexp\{-x\}$, $x\ge 0$. We have

$${\phi }^{-1}\left(z\right)=1/\sqrt{z}-1.$$

Define

$${\mathcal{P}}_m\left(z\right)=\phi (m{\phi }^{-1}\left(z\right))={(1+m(1/\sqrt{z}-1))}^{-2}.$$

It is easy to see that

$${(1+m(1/\sqrt{z}-1))}^{-2}=\sum _{k=1}^{\infty }{\displaystyle \frac{(k-1){(m-1)}^{k-2}}{m^k}}{z}^{k/2}.$$

Since the series expansion is performed in half-integer powers, ${\mathcal{P}}_m\left(z\right)$ is not a gfp. Thus, $\phi \left(s\right)$ is not the Laplace transform of the stable distribution for sums of a random number of random variables; it is a limiting distribution by virtue of Theorem 1.

The second question related to the problem under consideration concerns the sums of a random number of random variables with arbitrary signs. Combining B.V. Gnedenko’s [5] results with Theorem 1, we find that the class of all limit distributions in the above-mentioned scheme for summing a random number of i.i.d. random variables coincides with the set of all possible scale mixtures (we consider the non-negative scale only) of classical strictly stable laws. However, this does not allow us to find a simple criterion for a fixed distribution to belong to the class under consideration. In particular, such characteristic functions may have zeros on the real axis. It would be of definite interest to consider the marginal distributions for all types of possible centralizations.

The third question concerns the physical interpretation of Lemma 2. In particular, the value X of a positive integer random variable can be viewed as the number of particles of a certain type with probability-generating function $Q\left(z\right)$. Then, a transformation from $Q\left(z\right)$ to $Q(1-A+Az)$ could mean, for example, that these particles interacted with an absorbing screen that transmitted the particle with probability A and absorbed it with probability $1-A$. In other words, we are dealing with thinning. If $Q(1-A+Az)$ is a probability-generating function for $A>1$, this means that $Q\left(z\right)$ is obtained from it by thinning. Lemma 2 provides a condition that a generating function can be obtained from others by thinning with any absorption probability. What is the role of such distributions in physics?

4. Conclusions

We can now say that the problem of describing limit distributions for sums of random variables is much less interesting than finding analogs of stable laws. From a methodological standpoint, it seems more appropriate to study stable laws, as they are more important, before considering limit distributions in the classical summation scheme.

At the same time, studying the relationship between arbitrary distributions on the half-line and integer laws is of particular interest. The relations proposed by Lemmas 1 and 2 can be used to easily transfer the properties of probability laws from one class to another.