On Estimation of the Remainder Term in New Asymptotic Expansions in the Central Limit Theorem ^†

Abstract

We offer a new asymptotic expansion with an explicit remainder estimate in the central limit theorem. The results obtained are essentially based on new forms of asymptotic expansions in the central limit theorem. We also present a more accurate estimation of the CLT-expansions remainder which is rigorously proved and backed up numerically. It is shown that our approach can be used for further refinement of allied asymptotic expansions.

1. Introduction

The central limit theorem (CLT) states that under fairly broad conditions the sum of independent (or weakly dependent) identically distributed (i.i.d.) random variables is approximately normally distributed. In this paper, we deal with CLT for i.i.d. variables $X_1,X_2, ...$ with zero mean and unit variance each. Let $X_1$ follow a distribution $\mathrm{P}$ with cumulative distribution function $F\left(x\right)$ and characteristic function $f\left(t\right)$.

We assume that $X_1$ has a finite absolute moment of order $m+2$ and for some $\nu >0$ the function ${\left|f\left(t\right)\right|}^{\nu }$ is integrable on $\mathbb{R}$. Since for any $T>0$ the integral

$${\displaystyle \underset{T}{\overset{+\infty }{\int }}}{\left|f\left(t\right)\right|}^{\nu }dt<+\infty$$

converges and

$$\alpha \left(T\right)=sup\left\{\left|f\left(t\right)\right|:t⩾T\right\}<1$$

(see [1], p. 43), it follows that for $n⩾\nu$ the distribution ${\mathrm{P}}_n$ of the rescaled sum $\left(X_1+...+X_n\right)n^{-\frac{1}{2}}$ has a continuous density $p_n\left(x\right)$. In it is its turn, this implies that for this distribution ${\mathrm{P}}_n$ (with cumulative distribution function $F_n\left(x\right)$) the CLT holds [1], $p_n\left(x\right)\to \phi \left(x\right)$ as $n\to +\infty$ for any $x\in \mathbb{R}$; here $\phi \left(x\right)=\frac{1}{\sqrt{2\pi }}{e}^{-x^2/2}$ is the density of the standard normal distribution $\Phi \left(x\right)$. The density $p_n\left(x\right)$ can be represented as the inverse Fourier transform [1] (pp. 42, 147) of the characteristic function $f^n$ of the convolution of n copies of the original distribution.

Here the question naturally arises concerning the accuracy of the CLT-approximation. One of the main results on the subject is the Berry–Esseen theorem (see, for example, [2]). Nevertheless, this theorem is too general (and not attempting to take any specific properties of the original distribution into play) and, therefore, provides rather crude estimates of the convergence rate [3].

One way to improve the accuracy of CLT-approximations is to use asymptotic expansions. Until quite recently, most of such expansions only gave an estimate of the order of approximation at best and thus were of little use for, say, numeric computations.

Yu.V. Prohorov wrote [4] (p. 7) that “it was P.L. Chebyshev’s idea to explore the asymptotic behavior of the difference $F_n\left(x\right)-\Phi \left(x\right)$ and it was them to give a formal expansion of the difference”. A number of such expansions under different restrictions on the original distribution were obtained later by H. Cramer [5] and investigated by C.-G. Esseen [6]. Moreover, H. Cramer [7] claimed that the featured series expansions were introduced by F. Edgeworth [8]. For example, if there exists an integer $m⩾1$, such that $\mathrm{M}{\left|X_1\right|}^{m+2}<\infty$ and Cramer’s (C) condition ${\displaystyle \underset{\left|t\right|\to \infty }{lim}}sup\left|f\left(t\right)\right|<1$ is fulfilled, then

$$F_n\left(x\right)=\Phi \left(x\right)+{\displaystyle \sum _{k=1}^m}\frac{P_k(-\Phi )}{n^{k/2}}+O\left(\frac{1}{n^{m/2}}\right),n\to \infty ,$$

where $P_k(-\Phi )=L_{3k-1}\left(x\right)\phi \left(x\right)$ and $L_{3k-1}\left(x\right)$ is a polynomial of degree $3k-1$ in x. Explicit formulas for $P_k\left(-\Phi \right)$ in terms of semi-invariants were obtained by V.V. Petrov [9] in 1962.

Seeking more efficient forms of such expansions V.V. Senatov took what can be justly characterized a revolutionary step; he offered expansions that allowed for explicit estimation of the remainder (not just barely indicating big-Oh approximation order). At the moment only one type of CLT-asymptotic expansions, namely the Gram–Charlier expansion, was widely known, that is,

$$p_n\left(x\right)=\phi \left(x\right)+{\displaystyle \sum _{k=3,k\ne 3m-4}^{3m-3}}{\theta }_k\left(P_n\right)H_k\left(x\right)\phi \left(x\right)+O\left(\frac{1}{n^{m/2}}\right),n\to \infty ,$$

where $H_k\left(x\right)={\left(-1\right)}^k\frac{{\phi }^{\left(k\right)}\left(x\right)}{\phi \left(x\right)}$ are Chebyshev–Hermite polynomials and

$${\theta }_k\left(P_n\right)=\frac{1}{k!}{\displaystyle \underset{-\infty }{\overset{+\infty }{\int }}}{H}_k\left(x\right)p_n\left(x\right)dx,k⩾0$$

are normalized moments [10].

As mentioned by V.V. Senatov [1] (p. 124), already, in 1920, H. Cramer [7] noticed that this accuracy can be attained with only $m+2$ moments at hand (the Gram–Charlier expansion provides the same accuracy only when $3m-3$ moments are used). That was the reason, in Senatov’s opinion, why researchers’ primary focus was on the Edgeworth–Cramer (not Gram–Charlier) expansion.

At the same, it turns out that there exists an expansion of the Gram–Charlier type that makes use of only $m+2$ absolute moment of variable $X_1$. V.V. Senatov to construct this kind of asymptotic expansions proposed together with the moments:

$${\theta }_k={\displaystyle \sum _{j=0}^{\left[\frac{k}{2}\right]}}{(-1)}^j{\alpha }_{k-2j}{\alpha }_{2j}\left(\phi \right),$$

use the so-called [10] Senatov incomplete moments:

$${\theta }_k^{\left(l\right)}={\displaystyle \sum _{j=0}^{\left[\frac{l}{2}\right]}}{(-1)}^j{\alpha }_{k-2j}{\alpha }_{2j}\left(\phi \right),l⩽k,$$

where

$${\alpha }_j=\frac{\mathrm{M}{X}_1^j}{j!},{\beta }_j=\frac{\mathrm{M}{\left|X_1\right|}^j}{j!},{\alpha }_{2l}\left(\phi \right)=\frac{1}{2^{l}l!}.$$

V.V. Senatov came up [11] with what he dubbed a shortened Gram–Charlier expansion:

$$p_n\left(x\right)=\phi \left(x\right)+{\displaystyle \sum _{k=3}^{m+1}}{\theta }_k\left(P_n\right)H_k\left(x\right)\phi \left(x\right)+{\displaystyle \sum _{k=m+2,k\ne 3m-4}^{3m-3}}{\theta }_k^{(m+1)}\left(P_n\right)H_k\left(x\right)\phi \left(x\right)+R.$$

He used arbitrary measures (not necessarily probabilistic measures) in their derivation, which imposes additional limitations on the moments of the original distribution.

In attempts to remove the limitations and make the estimation of the remainder more accurate, V.V. Senatov and V.N. Sobolev [12] suggested a novel form of asymptotic expansion that does not impose any additional restrictions on the momenta (as compared to [1]).

The Gram–Charlier expansion differs architecturally from the Edgeworth–Cramer expansion, the former is in powers of the Chebyshev–Hermite polynomials, while the latter is in powers of n. Expansions of the third type [12,13] are obtained as follows, the terms in such an expansion are to be ordered with respect to the number of factors, which are Senatov’s moments.

Before the results of V.V. Senatov and V.N. Sobolev [12,13], only two types of such expansions were widely known, the Gram–Charlier and Edgeworth–Cramer expansions. In the former, the terms are grouped in the order of the Chebyshev–Hermite polynomials, and in the latter, they are grouped in powers of n. In [12], V.V. Senatov and V.N. Sobolev proposed grouping the terms according to the number of factors of Senatov’s moments

$$p_n\left(x\right)=\phi \left(x\right)+{\displaystyle \sum _{s=1}^{m-1}}{C}_n^s{\displaystyle \sum _{l=3s}^{m-1+2s}}\frac{{\Theta }_{s,l}}{n^{l/2}}{H}_{l}x)\phi \left(x\right)+O\left(\frac{1}{n^{m/2}}\right),n\to \infty ,$$

where

$${\Theta }_{s,l}={\displaystyle \sum _{t_1+...+t_s=l}}{\theta }_{t_1}...{\theta }_{t_s}$$

the summation is carried out over tuples of natural numbers $t_1$, …, $t_s$, such that $t_j⩾3$, $j=1,\cdots ,m-1$ and $t_1+\cdots +t_s=l$.

2. Main Result

To improve the accuracy of the expansions from [12] note first that the values

$${\alpha }_0=1,{\alpha }_1=0,{\alpha }_2=\frac{1}{2},{\alpha }_{2l}\left(\phi \right)=\frac{1}{2^{l}l!}$$

are known. Then, we introduce the following non-negative quantities:

$$∥{\theta }_s∥={\beta }_{s+2}+{\displaystyle \sum _{j=1}^{\left[\frac{s}{2}\right]-2}}{\alpha }_{2j}\left(\phi \right)\left|{\alpha }_{s-2j}\right|+\left|{\displaystyle \sum _{j=\left[\frac{s}{2}\right]-1}^{\left[\frac{s}{2}\right]}}{(-1)}^j{\alpha }_{2j}\left(\phi \right){\alpha }_{s-2j}\right|,$$

for which the inequalities

$$∥{\theta }_s∥⩽{\beta }_{s+2}+{\displaystyle \sum _{j=1}^{\left[\frac{s}{2}\right]}}{\alpha }_{2j}\left(\phi \right)\left|{\alpha }_{s-2j}\right|$$

are valid. Therefore, using $∥{\theta }_s∥$ in the estimates instead of the right side of the last inequality increases accuracy of asymptotic expansions.

Let $∥{\theta }_s^{\left(l\right)}∥$ be the abbreviated versions of $∥{\theta }_s∥$: they are calculated by the same formulas in which ${\alpha }_k=0$ for $k>l.$ There is also an improvement here.

Another improvement consists of preserving the minus signs between the summands when evaluating the estimates. The following quantities naturally arise

$$S_{0,2}(\Theta )=∥{\theta }_{m+2}∥,S_{0,3}(\Theta )=∥{\theta }_{m+3}^{\left(m+1\right)}∥$$

which for $l⩾1$ read

$$S_{l,2}(\Theta )={\displaystyle \sum _{k_1,k_2,...,k_l}^m}\left|{\theta }_{3+k_1}{\theta }_{3+k_2}...{\theta }_{3+k_l}\right|∥{\theta }_{m+2-\left(l+k_1+k_2+...+k_l\right)}∥,$$

$$S_{l,3}(\Theta )={\displaystyle \sum _{k_1,k_2,...,k_l}^m}\left|{\theta }_{3+k_1}{\theta }_{3+k_2}...{\theta }_{3+k_l}\right|∥{\theta }_{m+3-l-k_1-k_2-...-k_l}^{\left(m+2-l-k_1-k_2-...-k_l\right)}∥.$$

The notation ${\displaystyle \sum _{k_1,k_2,...,k_l}^m}$ implies summation over all sets $k_1, k_2, ..., k_l$ of non-negative numbers, such that $0⩽k_1+k_2+...+k_l⩽m-l-1$.

We will also use the following quantities of the moment type

$$L_l\left(u\right)=\frac{1}{2\pi }{\displaystyle \underset{\left|t\right|⩾u}{\int }}{\left|t\right|}^l{e}^{-\frac{t^2}{2}}dt,B_{l,n-k}=\frac{1}{2\pi }{\displaystyle \underset{-T\sqrt{n}}{\overset{+T\sqrt{n}}{\int }}}{\left|t\right|}^l{\mu }^{n-k}\left(\frac{t}{\sqrt{n}}\right)dt,$$

where the function $\mu \left(t\right)=\max \left\{\left|f\left(t\right)\right|,e^{-t^2/2}\right\}$ was introduced by V.Yu. Korolev.

Below, we formulate our main theorem that provides a CLT-asymptotic expansion with an improved estimation of the remainder under fairly general conditions.

Theorem 1.

Let identically distributed independent random variables $X_1, X_2, ...$ with zero mean and unit variance each follow the same distribution P. Suppose that P has a finite absolute moment of order $m+2$ and ${\displaystyle {\int }_{-\infty }^{\infty }}{\left|f\left(t\right)\right|}^{\nu }dt<\infty .$ Where $f\left(t\right)$ is the characteristic function of P. Then, for any $n⩾max\left(\nu ,m+1\right)$ and for all $x\in \mathbb{R}$

$$p_n\left(x\right)=\phi \left(x\right){\displaystyle \sum _{l=0}^{m-1}}\frac{C_n^l}{{\left(\sqrt{n}\right)}^{3l}}{\displaystyle \sum _{k_1,k_2,...,k_l}^m}\frac{{\theta }_{3+k_1}{\theta }_{3+k_2}...{\theta }_{3+k_l}}{{\left(\sqrt{n}\right)}^{k_1+k_2+...+k_l}}{H}_{3l+k_1+...+k_l}\left(x\right)+R_{n,m}\left(x\right),$$

where

$$\left|R_{n,m}\left(x\right)\right|⩽\frac{1}{{\left(\sqrt{n}\right)}^{m+2}}{\displaystyle \sum _{l=0}^{m-2}}\frac{C_n^{l+1}}{n^l}\left(B_{m+2+2l,n-1}{S}_{l,2}(\Theta )+B_{m+3+2l,n-1}{S}_{l,3}(\Theta )\frac{1}{\sqrt{n}}\right)$$

$$+{\Lambda }_n\left(T\right)+{\overline{\Lambda }}_n\left(T\right).$$

and two last terms in the remainder’s estimate are

$${\Lambda }_n\left(T\right)=\frac{\sqrt{n}}{\pi }{\alpha }^{n-\nu }\left(T\right){\displaystyle \underset{T}{\overset{+\infty }{\int }}}{\left|f\left(t\right)\right|}^{\nu }dt,$$

$${\overline{\Lambda }}_n\left(T\right)={\displaystyle \sum _{l=0}^{m-1}}{C}_n^l{\displaystyle \sum _{k_1,k_2,...,k_l}^m}\frac{\left|{\theta }_{3+k_1}{\theta }_{3+k_2}...{\theta }_{3+k_l}\right|}{{\left(\sqrt{n}\right)}^{3l+k_1+k_2+...+k_l}}{L}_{3l+k_1+k_2+...+k_l}\left(T\sqrt{n}\right),$$

which decay exponentially fast.

3. Conclusions

We obtain new explicit estimates for accuracy of approximation in the CLT-expansions. Our approach can be used for further refinement of allied asymptotic expansions.