The Distribution and Quantiles of the Sample Mean from a Stationary Process

Abstract

Edgeworth–Cornish–Fisher expansions are hugely important, as they give the distribution, density and quantiles of any standard estimate. Here we show that the sample mean of a univariate or multivariate stationary process is a standard estimate, so that all the known results for standard estimates can be applied. We also show how to allow for missing data and weighted means.

1. Introduction and Summary

Finding the distribution, density and quantiles of an estimate is of great importance. This has been accomplished for standard estimates by extending the expansions of Edgeworth, Cornish, and Fisher. In this section we summarize these results for a univariate standard estimate.

Section 2 gives our main result: we show that the mean of a sample from a univariate stationary process satisfies a special truncated form of the cumulant expansion (1) below, so that all the results of this section can be applied. It also considers the case where observations are not sequential, as for the case of missing data. And it considers unbiased weighted sample means.

In Section 4, we extend this to the mean of a sample from a multivariate stationary process, after summarising the multivariate Edgeworth expansions for a standard estimate in Section 3.

We now summarise the known results for the distribution and quantiles of a univariate standard estimate. Let $\widehat{w}$ be an estimate of an unknown $w\in R$ based on a sample of size n. References [1,2] gave expansions in $n^{-1/2}$ for its distribution and quantiles when its cumulants satisfied an artifical assumption removed by [3]. We call $\widehat{w}$ a standard estimate with respect to n, if $E\widehat{w}\to w$ as $n\to \infty$, and its rth cumulant can be expanded as

$$\begin{array}{cc}& {\kappa }_r\left(\widehat{w}\right)\approx \sum _{i=r-1}^{\infty }{a}_{ri}{n}^{-i},r\ge 1,\hfill \end{array}$$

(1)

where ≈ indicates an asymptotic expansion that need not converge. For example, in [4], this holds if $\widehat{w}$ is a smooth function of the mean of independent identically distributed (i.i.d.) random variables. The cumulant coefficients of $\widehat{w},a_{ri}$, may depend on n but must be bounded as $n\to \infty$, and $a_{21}$ must be bounded away from 0 as $n\to \infty$. For $\widehat{w}$ non-lattice, the distribution, density and quantiles of

$$Y_n={(n/a_{21})}^{1/2}(\widehat{w}-w)$$

have asymptotic expansions in powers of $n^{-1/2}$:

$$\begin{array}{c}{P}_n\left(x\right)=Prob.(Y_n\le x)\approx \Phi \left(x\right)-\varphi \left(x\right)\sum _{r=1}^{\infty }{h}_{0r}\left(x\right)n^{-r/2},\end{array}$$

(2)

$$\begin{array}{c}{p}_n\left(x\right)=d{P}_n\left(x\right)/dx\approx \varphi \left(x\right)[1+\sum _{r=1}^{\infty }{h}_{1r}\left(x\right)n^{-r/2}],\end{array}$$

(3)

$$\begin{array}{c}{\Phi }^{-1}\left(P_n\left(x\right)\right)\approx x-\sum _{r=1}^{\infty }{f}_r\left(x\right)n^{-r/2},P_n^{-1}(\Phi \left(x\right))\approx x+\sum _{r=1}^{\infty }{g}_r\left(x\right)n^{-r/2},\end{array}$$

(4)

where $\Phi$ and $\varphi$ are the distribution and density of a unit normal random variable $N\sim \mathcal{N}(0,1)$, and $h_{kr}\left(x\right),f_r\left(x\right),g_r\left(x\right)$ are polynomials of degrees $k+3r-1,r+1,r+1$ both in x and in the standardized cumulant coefficients

$$\begin{array}{cc}& A_{ri}=a_{ri}/a_{21}^{r/2}.\hfill \end{array}$$

(5)

In practice, one truncates these expansions for the distribution, density and quantiles of $\widehat{w}$. The leading $h_{kr}\left(x\right),f_r\left(x\right),g_r\left(x\right)$ given below suffice to see if divergence has begun. For $k\ge 0,$ the kth Hermite polynomial is

$$\begin{array}{cc}& H_k=H_k\left(x\right)=\varphi {\left(x\right)}^{-1}{(-d/dx)}^k\varphi \left(x\right)=E{(x+iN)}^k\mathrm{where}i=\sqrt{-1}:\hfill \\ & H_0=1,H_1=x,H_2=x^2-1,H_3=x^3-3x,H_4=x^4-6x^2+3,\hfill \\ & H_5=x^5-10x^3+15x,H_6=x^6-15x^4+45x^2-15,\hfill \end{array}$$

(6)

and so on. See [5] for (6). For the expansions to $O\left(n^{-5/2}\right)$ of (2) and (4), see [3]. $h_{kr}\left(x\right)$ of (2), (3) has the form

$$\begin{array}{cc}& h_{kr}\left(x\right)=\sum [P_{rj}{H}_{k+j-1}:1\le j\le 3r,r-j \mathrm{even}]\hfill \end{array}$$

where $P_{rj}$ are polynomials in $\left\{A_{rj}\right\}$ of (5), given for $r\le 4$ in the appendix of [6], or easily derived from [3]. The terms needed in expansions (2)–(4) to $O\left(n^{-3/2}\right)$ are

$$\begin{array}{cc}& h_{01}\left(x\right)=f_1\left(x\right)=g_1\left(x\right)=A_{11}+A_{32}{H}_2/6,h_{11}\left(x\right)=A_{11}{H}_1+A_{32}{H}_3/6,\hfill \\ & h_{k2}\left(x\right)=(A_{11}^2+A_{22})H_{1+k}/2+(A_{43}+4A_{11}{A}_{32})H_{3+k}/24+A_{32}^2{H}_{5+k}/72,\hfill \end{array}$$

(7)

$$\begin{array}{c}{f}_2\left(x\right)=(A_{22}/2-A_{11}{A}_{32}/3)H_1+A_{43}{H}_3/24-A_{32}^2(4x^3-7x)/36,\hfill \\ g_2\left(x\right)=A_{22}{H}_1/2+A_{43}{H}_3/24-A_{32}^2(2x^3-5x)/36.\hfill \end{array}$$

(8)

By [7], the log density has a simpler form than the density:

$$\begin{array}{cc}& ln[p_n\left(x\right)/\varphi \left(x\right)]=\sum _{r=1}^{\infty }{b}_r\left(x\right)n^{-r/2}\mathrm{where}{b}_1\left(x\right)=h_{11}\left(x\right),\hfill \\ & b_2\left(x\right)=-A_{11}^2/2+(A_{22}-A_{32}{A}_{11})H_2/2+A_{43}{H}_4/24-A_{32}^2(3x^4-12x^2+5)/24.\hfill \end{array}$$

(9)

For $r>1,b_r\left(x\right)$ is a polynomial of an order only $r+2$, while $h_{1r}\left(x\right)$ is of an order $3r$. See [6] for other $h_{kr}\left(x\right)$ and $b_r\left(x\right)$. If $E\widehat{w}=w$, then $A_{11}=0$ so that (2)–(4) and (9) hold with

$$\begin{array}{cc}& h_{01}\left(x\right)=f_1\left(x\right)=g_1\left(x\right)=b_1\left(x\right)=A_{32}{H}_2/6,h_{11}\left(x\right)=A_{32}{H}_3/6,\hfill \\ & h_{k2}\left(x\right)=A_{22}{H}_{1+k}/2+A_{43}{H}_{3+k}/24+A_{32}^2{H}_{5+k}/72\mathrm{for} k=0,1,\hfill \end{array}$$

(10)

$$\begin{array}{c}{f}_2\left(x\right)=A_{22}{H}_1/2+A_{43}{H}_3/24-A_{32}^2(4x^3-7x)/36,\hfill \\ b_2\left(x\right)=A_{22}{H}_2/2+A_{43}{H}_4/24-A_{32}^2(3x^4-12x^2+5)/24,\hfill \end{array}$$

(11)

and $g_2\left(x\right)$ of (8) is unchanged.

Note 1.

The original Edgeworth expansion was for $\widehat{w}$ the mean of n i.i.d. random variables from a distribution with the rth cumulant ${\kappa }_r$. So (1) holds with $a_{r,r-1}={\kappa }_r,a_{ri}=0,i\ge r$. An explicit formula for its general term was given in [8].

Edgeworth–Cornish–Fisher expansions were first extended to general parametric and non-parametric standard estimates in [3,4] and to functions of them in [9].

In [10], we gave the extended Edgeworth-Cornish-Fisher expansions for smooth functions of the sample cross-moments of a linear process. We now show that this extends easily to a stationary process.

2. The Cumulants of a Stationary Sample Mean

When $\widehat{w}$ is the mean of i.i.d. random variables, the cumulant expansion (1) has only one term. Remarkably, for the sample mean of a stationary process, its cumulant expansion has exactly two terms.

Suppose that $\overline{X}$ is the mean of a sample $X_1,\dots ,X_n$ from a real stationary process $\left\{X_i\right\}$. So $\overline{X}$ is an unbiased estimate of $\mu =E{X}_0$. We now show that its rth cumulant has the form

$$\begin{array}{cc}& {\kappa }_r\left(\overline{X}\right)=a_{nr,r-1}{n}^{1-r}+a_{nrr}{n}^{-r},r>1,\hfill \end{array}$$

(12)

where $a_{nri}$ are bounded as n increases, and $a_{n21}$ is bounded away from 0. This makes it a special case of a standard estimate, so that Section 1 applies with $a_{ri}=a_{nri}$ for $i=r-1$ and $i=r$, and $a_{ri}=0$ for $i>r$ and $a_{11}=0$.

If $a_{nri}=a_{ri}+O\left(e^{-n{\lambda }_r}\right)$ where ${\lambda }_{ri}>0,$ then $a_{nri}$ can be replaced by $a_{ri}.$ Here, $x_n=O\left(y_n\right)$ means that $x_n/y_n$ is bounded.

Suppose that the stationary process has cross-cumulants,

$$\begin{array}{cc}& k(i_1\dots i_r)={\kappa }_{i_1,\dots ,i_r}=\kappa (X_{i_1},\dots ,X_{i_r}).\hfill \end{array}$$

(13)

Given integers $i_1,\dots ,i_r,$ set

$$\begin{array}{c}\hfill i_0=\underset{k=1}{\overset{r}{min}}{i}_k,I_k=i_k-i_0\ge 0,I_0=\underset{k=1}{\overset{r}{max}}{I}_k=\underset{k=1}{\overset{r}{max}}{i}_k-i_0.\end{array}$$

(14)

Since $\left\{X_i\right\}$ is stationary,

$$\begin{array}{c}\hfill k(i_1\dots i_r)=k(I_1\dots I_r).\end{array}$$

(15)

These are not changed by permuting subscripts. Also, at least one $I_k$ is zero. For $r\ge 2,$ transforming from $i_k$ to $T_k=i_k-i_1$ for $k=2,\dots ,r$,

$$\begin{array}{c}\hfill n^r{\kappa }_r\left(\overline{X}\right)=\sum _{i_1,\dots ,i_r=1}^{n}k(i_1\dots i_r)=\sum _{|T_k|<n,k=2,\dots ,r}[n-{\delta }_r\left(\mathbf{T}\right)]k(0T_2\dots T_r)\end{array}$$

(16)

$$\begin{array}{c}\hfill \mathrm{where}{\delta }_r\left(\mathbf{T}\right)=max(0,T_2,\dots ,T_r)-min(0,T_2,\dots ,T_r).\end{array}$$

(17)

For example, ${\delta }_2\left(\mathbf{T}\right)=\left|T_2\right|$,

$$\begin{array}{c}\hfill {\delta }_3\left(\mathbf{T}\right)=T_{3}I(0\le T_2<T_3)+(T_3-T_2)I(T_2\le 0<T_3)-T_{2}I(T_2<T_3\le 0).\end{array}$$

So for $r\ge 2,{\kappa }_r\left(\overline{X}\right)={\sum }_{i=r-1}^r{a}_{nri}{n}^{-i}$, where

$$\begin{array}{c}\hfill a_{n21}=\sum _{\left|T\right|<n}k\left(0T\right),a_{nr,r-1}=\sum _{|T_i|<n,i=2,\dots ,r}k(0T_2\dots T_r),\end{array}$$

(18)

$$\begin{array}{c}\hfill a_{nrr}=-\sum _{|T_i|<n,i=2,\dots ,r}{\delta }_r\left(\mathbf{T}\right)k(0T_2\dots T_r).\end{array}$$

(19)

This proves that (12) holds. So the Edgeworth–Cornish–Fisher expansions (2)–(4) and (9)–(11) apply to $(\widehat{w},w)=(\overline{X},\mu )$ with $a_{ri}$ in (1) replaced by these $a_{nri}$, so that $A_{ri}=a_{nri}/a_{n21}^{r/2}.$

If the cross-cumulants $k(0T_2\dots T_r)$ decrease exponentially in $T_2$, as is true for a stationary ARMA process by [10], then for $r\ge 2$,

$$\begin{array}{c}{a}_{nr,r-1}=a_{r,r-1}+O\left(e^{-n{\lambda }_r}\right),a_{nrr}=a_{rr}+O\left(e^{-n{\lambda }_r}\right)\mathrm{where}{\lambda }_r>0,\hfill \\ a_{21}=\sum _{\left|T\right|<\infty }k\left(0T\right),a_{r,r-1}=\sum _{|T_i|<\infty ,i=2,\dots ,r}k(0T_2\dots T_r),\hfill \end{array}$$

(20)

$$\begin{array}{c}{a}_{rr}=-\sum _{|T_i|<\infty ,i=2,\dots ,r}{\delta }_r\left(\mathbf{T}\right)k(0T_2\dots T_r),\end{array}$$

(21)

so that for $X_0$ non-lattice, these Edgeworth–Cornish–Fisher expansions apply to $(\widehat{w},w)=(\overline{X},\mu )$ with these $a_{ri}$. According to [3] or the appendix to [6], (10)–(8), $h_{kr},f_r,g_r$ simplify for $r=3,4$. This expands the results given in [10] for a linear process.

For convergence in law of $n^{1/2}(\overline{X}-\mu )$ to $\mathcal{N}(0,a_{21})$ with $a_{21}$ of (20), under mixing conditions on a stationary process, see Sections 18.4 and 18.5 of [11]. They also show how to express $a_{21n}$ and $a_{21}$ in terms of the spectral distribution and density.

Missing values. Now suppose that we only have observations at times $t_1,\dots ,t_n$. Our estimate of $\mu =E{X}_0$ is then

$$\begin{array}{cc}& {\widehat{\mu }}_t=n^{-1}\sum _{i=1}^n{X}_{t_i}.\mathrm{So}E{\widehat{\mu }}_t=\mu ,\mathrm{and}\mathrm{for}{S}_k=t_{i_k}-t_{i_1},r\ge 2,\hfill \\ & n^r{\kappa }_r\left({\widehat{\mu }}_t\right)=\sum _{i_1,\dots ,i_r=1}^{n}k(t_{i_1}\dots t_{i_r})=\sum _{i_1,\dots ,i_r=1}^{n}k(0S_2\dots S_r)=n{a}_{nr,r-1} \mathrm{say}.\hfill \end{array}$$

So if $a_{n21}$ is bounded away from 0 and $a_{nr,r-1}$ is bounded in n, we can apply Section 1 with $a_{ri}=0$ for $i\ge r\ge 1$.

Weighted means. Let $w_{n1},\dots ,w_{nn}$ be given numbers adding to n. For example, the standardized form of the Chernoff weight $i/n$, giving more weight to more recent observations, is $w_{ni}=2i/(n+1).$ See [12]. An unbiased estimate of $\mu$ is the weighted sample mean, ${\widehat{\mu }}_w=n^{-1}{\sum }_{i=1}^n{w}_{ni}{X}_i.$

$$\begin{array}{cc}& \mathrm{For} r\ge 2,n^r{\kappa }_r\left({\widehat{\mu }}_w\right)=\sum _{i_1,\dots ,i_r=1}^n{w}_{n{i}_1}\dots w_{n{i}_r}k(i_1,\dots ,i_r)=n{a}_{nr,r-1}\hfill \end{array}$$

So if $a_{n21}$ is bounded away from 0 and $a_{nr,r-1}$ is bounded in n, we can apply Section 1 with $a_{ri}=0$ for $i\ge r\ge 1$. Missing values can also be treated by giving them a weight of 0.

3. Multivariate Edgeworth Expansions

Ordinary Bell polynomials. For a sequence from R, say $e=(e_1,e_2,\dots ),$ the partial ordinary Bell polynomial ${\tilde{B}}_{rs}={\tilde{B}}_{rs}\left(e\right)$, is defined by the identity

$$\begin{array}{cc}& S^s=\sum _{r=s}^{\infty }{z}^r{\tilde{B}}_{rs}\left(e\right)\mathrm{where}S=\sum _{r=1}^{\infty }{z}^r{e}_r,z\in R.\hfill \\ & \mathrm{So}, {\tilde{B}}_{r0}={\delta }_{r0},{\tilde{B}}_{r1}=e_r,{\tilde{B}}_{rr}=e_1^r,{\tilde{B}}_{21}=2e_1{e}_2,\hfill \end{array}$$

where ${\delta }_{00}=1,{\delta }_{r0}=0$ for $r\ne 0.$ They are tabled on p. 309 of [13]. The complete ordinary Bell polynomial, ${\tilde{B}}_r\left(e\right)$, is defined in terms of S by

$$\begin{array}{c}\hfill e^S=\sum _{r=0}^{\infty }{z}^r{\tilde{B}}_{rs}\left(e\right).\mathrm{So}{\tilde{B}}_r\left(e\right)=\sum _{s=0}^r{\tilde{B}}_{rs}\left(e\right)/s!:\end{array}$$

(22)

$$\begin{array}{c}{\tilde{B}}_0\left(e\right)=1,{\tilde{B}}_1\left(e\right)=e_1,{\tilde{B}}_2\left(e\right)=e_2+e_1^2/2,{\tilde{B}}_3\left(e\right)=e_3+e_1{e}_2+e_1^3/6.\end{array}$$

(23)

Now suppose that

$$\begin{array}{cc}& e_j\left(s\right)=\sum _{r=1}^{j+2}{\overline{b}}_{r+j}^{1-r}{\overline{s}}_1\dots {\overline{s}}_r/r!\mathrm{where}{\overline{s}}_k=s_{i_k},{\overline{b}}_{r+j}^{1-r}=b_{r+j}^{i_1\dots i_r},b_{2d+1}^{i_1\dots i_r}=0,\hfill \end{array}$$

(24)

for some constants $b_j^{i_1\dots i_r}$. Then for $r\ge 1$, we can write

$$\begin{array}{cc}& {\tilde{B}}_r\left(e\left(s\right)\right)=\sum _{k=1}^{3r}[{\overline{P}}_r^{1-k}{\overline{s}}_1\dots {\overline{s}}_k:k-r \mathrm{even}],\hfill \end{array}$$

(25)

where (24) and (25) use the tensor summation convention of implicitly summing $i_1,i_2,\dots$ over their range $1,\dots ,p$, and ${\overline{P}}_r^{1-k}$ is a polynomial in $\left\{{\overline{b}}_j^{1-r}\right\}$. For $r=1,2,{\tilde{B}}_r\left(e\left(s\right)\right)$ is given by (25) in terms of

$$\begin{array}{cc}& {\overline{P}}_1^1={\overline{b}}_2^1,{\overline{P}}_1^{1-3}={\overline{b}}_4^{1-3}/6,{\overline{P}}_2^{12}={\overline{b}}_2^1{\overline{b}}_2^2/2+{\overline{b}}_4^{12}/2,\hfill \\ & {\overline{P}}_2^{1-4}={\overline{b}}_6^{1-4}/24+\mathcal{S}{\overline{b}}_2^1{\overline{b}}_4^{2-4}/6,{\overline{P}}_2^{1-6}=\mathcal{S}{\overline{b}}_4^{1-3}{\overline{b}}_4^{4-6}/72,\hfill \end{array}$$

and the operator $\mathcal{S}$ symmetrises over $i_1,\dots ,i_k$.

Multivariate estimates. Suppose that $\widehat{w}$ is a standard estimate of $w\in R^p$ with respect to n. That is, $E\widehat{w}\to w$ as $n\to \infty$, and for $r\ge 1$, $1\le i_1,\dots ,i_r\le p,$ the rth order cumulants of $\widehat{w}$ can be expanded as

$$\begin{array}{c}\hfill {\overline{k}}^{1-r}=\kappa ({\widehat{w}}^{i_1},\dots ,{\widehat{w}}^{i_r})\approx \sum _{j=r-1}^{\infty }{\overline{k}}_j^{1-r}{n}^{-j}\mathrm{where}{\overline{k}}_j^{1-r}=k_j^{i_1\dots i_r},\end{array}$$

(26)

and the cumulant coefficients ${\overline{k}}_j^{1-r}=k_j^{i_1\dots i_r}$ may depend on n but are bounded as $n\to \infty$. So ${\overline{k}}_0^1=w^{i_1}.$

$$Y_n=n^{1/2}(\widehat{w}-w)$$

converges in law to the multivariate normal ${\mathcal{N}}_p(0,V)$ with $p\times p$ covariance $V=\left(k_1^{i_1{i}_2}\right),p\times p$, and distribution and density ${\Phi }_V\left(x\right)$ and ${\varphi }_V\left(x\right)$. So V may depend on n, but we assume that $det\left(V\right)$ is bounded away from 0 as $n\to \infty$. By [14], for $\widehat{w}$ non-lattice, the density and distribution of $Y_n$ can be expanded as

$$\begin{array}{c}\hfill p_{Y_n}\left(x\right)\approx \sum _{r=0}^{\infty }{n}^{-r/2}{p}_r\left(x\right),Prob.(Y_n\le x)\approx \sum _{r=0}^{\infty }{n}^{-r/2}{P}_r\left(x\right),x\in R^p,\end{array}$$

(27)

$$\begin{array}{c}\mathrm{where}{p}_0\left(x\right)={\varphi }_V\left(x\right),P_0\left(x\right)={\Phi }_V\left(x\right),\end{array}$$

(28)

$$\begin{array}{c}{p}_r\left(x\right)={\tilde{B}}_r\left(e(-\partial /\partial x)\right){\varphi }_V\left(x\right)\mathrm{for}r\ge 1,\hfill \\ P_r\left(x\right)={\tilde{B}}_r\left(e(-\partial /\partial x)\right){\Phi }_V\left(x\right)={\int }_{-\infty }^x{p}_r\left(x\right){\varphi }_V\left(x\right)dx\mathrm{for}r\ge 1,\hfill \end{array}$$

(29)

and $b_{2d}^{i_1\dots i_r}=k_d^{i_1\dots i_r}$ in (24), so that for example,

$$\begin{array}{c}\hfill e_1\left(s\right)={\overline{k}}_1^1{\overline{s}}_1+{\overline{k}}_2^{1-3}{\overline{s}}_1{\overline{s}}_2{\overline{s}}_3/6,e_2\left(s\right)={\overline{k}}_2^{12}{\overline{s}}_1{\overline{s}}_2/2+{\overline{k}}_3^{1-4}{\overline{s}}_1\dots {\overline{s}}_4/24.\end{array}$$

This gives the Edgeworth expansion for the distribution of $Y_n$ to $O\left(n^{-3/2}\right)$. See [15] for more terms. According to (25),

$$\begin{array}{c}{p}_r\left(x\right)/{\varphi }_V\left(x\right)=\sum _{k=1}^{3r}[{\overline{P}}_r^{1-k}{\overline{H}}^{1-k}(x,V):k-r\mathrm{even}],\hfill \\ P_r\left(x\right)=\sum _{k=1}^{3r}[{\overline{P}}_r^{1-k}{\overline{H}}_{*}^{1-k}(x,V):k-r\mathrm{even}],\mathrm{where}\hfill \\ {\overline{P}}_1^1={\overline{k}}_1^1,{\overline{P}}_1^{1-3}={\overline{k}}_2^{1-3}/6,{\overline{P}}_2^{12}={\overline{k}}_1^1{\overline{k}}_1^2+{\overline{k}}_2^{12}/2,\hfill \\ {\overline{P}}_2^{1-4}={\overline{k}}_3^{1-4}/24+\mathcal{S}{\overline{k}}_1^1{\overline{k}}_2^{2-4}/6,{\overline{P}}_2^{1-6}=\mathcal{S}{\overline{k}}_2^{1-3}{\overline{k}}_2^{4-6}/36,\hfill \end{array}$$

(30)

${\overline{H}}^{1-k}={\overline{H}}^{1-k}(x,V)$ is the multivariate Hermite polynomial,

$$\begin{array}{cc}& {\overline{H}}^{1-k}(x,V)={\varphi }_V{\left(x\right)}^{-1}(-{\overline{\partial }}_1)\dots (-{\overline{\partial }}_k){\varphi }_V\left(x\right)\mathrm{where}{\partial }_i=\partial /\partial x_i,{\overline{\partial }}_k={\partial }_{i_k},\hfill \\ & \mathrm{and} {\overline{H}}_{*}^{1-k}={\overline{H}}_{*}^{1-k}(x,V)=(-{\overline{\partial }}_1)\dots (-{\overline{\partial }}_k){\Phi }_V\left(x\right)={\int }_{-\infty }^x{\overline{H}}^{1-k}(x,V){\varphi }_V\left(x\right)dx.\hfill \end{array}$$

(${\overline{H}}_{*}^{1-k}$ deserves a name. Let us call it the multivariate Hermite function.) So $P_r\left(x\right)$ is just $p_r\left(x\right)$ with ${\overline{H}}^{1-k}{\varphi }_V\left(x\right)$ replaced by ${\overline{H}}_{*}^{1-k}$. For example, according to (27), the leading corrections to the Central Limit Theorem are given by

$$\begin{array}{cc}& p_1\left(x\right)=e_1(-\partial /\partial x){\varphi }_V\left(x\right)=({\overline{k}}_1^1{\overline{H}}^1+{\overline{k}}_2^{1-3}{\overline{H}}^{1-3}/6){\varphi }_V\left(x\right),\hfill \\ & P_1\left(x\right)=e_1(-\partial /\partial x){\Phi }_V\left(x\right)={\overline{k}}_1^1{\overline{H}}_{*}^1+{\overline{k}}_2^{1-3}{\overline{H}}_{*}^{1-3}/6,\hfill \end{array}$$

(31)

$$\begin{array}{c}{p}_2\left(x\right)={\varphi }_V\left(x\right)\sum _{k=2,4,6}{\overline{P}}_2^{1-k}{\overline{H}}^{1-k}(x,V),\hfill \\ P_2\left(x\right)=\sum _{k=2,4,6}{\overline{P}}_2^{1-k}{\overline{H}}_{*}^{1-k}(x,V),\hfill \end{array}$$

(32)

for ${\overline{P}}_2^{1-k}$ of (30). (So $h_{1r}\left(x\right)$ of Section 1 is a one-dimensional form of $p_r\left(x\right)$). By [5], for $i=\sqrt{-1}$,

$$\begin{array}{cc}& {\overline{H}}^{1-k}(x,V)=E{\Pi }_{j=1}^k({\overline{y}}_j+i{\overline{Y}}_j)\mathrm{where}{\overline{y}}_j=y_{i_j},{\overline{Y}}_j=Y_{i_j},y=V^{-1}x,\hfill \\ & Y\sim {\mathcal{N}}_p(0,V^{-1}).\mathrm{So}, H^1=y_1,{\overline{H}}^1={\overline{y}}_1,H^{1-3}=y_1{y}_2{y}_3-\sum ^3{V}^{12}{y}_3,\hfill \end{array}$$

where $V^{i_1{i}_2}$ is the $(i_1,i_2)$ element of $V^{-1},$ and ${\sum }^3{V}^{12}{y}_3=V^{12}{y}_3+V^{13}{y}_2+V^{23}{y}_1.$

The $H^{1-k}$ needed for $p_2\left(x\right)$ are

$$\begin{array}{cc}& H^{12}=y_1{y}_2-V^{12},\mathrm{so} \mathrm{that} {\overline{H}}^{12}={\overline{y}}_1{\overline{y}}_2-{\overline{V}}^{12}for{\overline{V}}^{12}=V^{i_1{i}_2},\hfill \\ & H^{1-4}=y_1\dots y_4-\sum ^6{V}^{12}{y}_3{y}_4+\sum ^3{V}^{12}{V}^{34},\hfill \\ & H^{1-6}=y_1\dots y_6-\sum ^{15}{V}^{12}{y}_3\dots y_6+\sum ^{45}{V}^{12}{V}^{34}{y}_5{y}_6-\sum ^{45}{V}^{12}{V}^{34}{V}^{56}.\hfill \end{array}$$

According to [7], the log density can be expanded as

$$\begin{array}{cc}& ln[p_n\left(x\right)/{\varphi }_V\left(x\right)]\approx \sum _{r=1}^{\infty }{n}^{-r/2}{b}_r\left(x\right).\hfill \end{array}$$

(33)

$$\begin{array}{cc}& \mathrm{So} p_n\left(x\right)/{\varphi }_V\left(x\right)\approx \sum _{r=0}^{\infty }{n}^{-r/2}{\tilde{B}}_r\left(b\left(x\right)\right)\mathrm{where}b=(b_2,b_2,\dots ),\hfill \end{array}$$

(34)

for ${\tilde{B}}_r\left(e\right)$ of (22). If $E\widehat{w}=w$, then ${\overline{k}}_1^1=0$, so that for $r=1,2,p_r\left(x\right),P_r\left(x\right)$ are given by (31)–(32) in terms of

$$\begin{array}{cc}& {\overline{P}}_1^1=0,{\overline{P}}_1^{1-3}={\overline{k}}_2^{1-3}/6,{\overline{P}}_2^{12}={\overline{k}}_2^{12}/2,{\overline{P}}_2^{1-4}={\overline{k}}_3^{1-4}/24,{\overline{P}}_2^{1-6}=\mathcal{S}{\overline{k}}_2^{1-3}{\overline{k}}_2^{4-6}/36.\hfill \end{array}$$

Note 2.

The term Hermite function is also used by [16] for the non-polynomial solution of the 2nd order differential equation for $H_n\left(x\right)$ of Section 1, given by [17] in terms of confluent hypergeometric functions.

4. Application to Stationary Vector $\overline{X}$

Suppose that $\dots ,X_{-1},X_0,X_1,\dots$ lie in $R^p$ and are stationary with mean $\mu =E{X}_0$ and finite moments. Suppose that $\overline{X}$ is the mean of a sample $X_1,\dots ,X_n$. So $E\overline{X}=\mu .$ For $j=1,\dots ,p$, denote the jth component of $X_i$ and $\overline{X}$ by $X_i^j$ and ${\overline{X}}^j,$ and the cross-cumulants of $\left\{X_i\right\}$, by

$$\begin{array}{cc}\hfill & k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{i_1\dots i_r}}\right)=\kappa (X_{i_1}^{j_1},\dots ,X_{i_r}^{j_r}) \mathrm{for} 1\le j_1,\dots ,j_r\le p.\hfill \end{array}$$

(35)

Given a sequence of integers $i_1,\dots ,i_r$, define $i_0,I_k$ as in (14), and again transform from $i_k$ to $T_k=i_k-i_1$ for $k=2,\dots ,r$. (15) becomes

$$\begin{array}{c}\hfill k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{i_1\dots i_r}}\right)=k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{I_1\dots I_r}}\right).\end{array}$$

(36)

In general, $k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1{j}_2}{0I}}\right)\ne k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1{j}_2}{0-I}}\right)$. For $r\ge 2$ and ${\delta }_r\left(\mathbf{T}\right)$ of (17), by (16),

$$\begin{array}{c}{n}^r\kappa ({\overline{X}}^{j_1},\dots ,{\overline{X}}^{j_r})=\sum _{i_1,\dots ,i_r=1}^{n}k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{i_1\dots i_r}}\right)\hfill \\ =\sum _{|T_k|<n,k=2,\dots ,r}[n-{\delta }_r\left(\mathbf{T}\right)]k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{0T_2\dots T_r}}\right).\hfill \end{array}$$

(37)

So $forr\ge 2,\kappa ({\overline{X}}^{j_1},\dots ,{\overline{X}}^{j_r})={\sum }_{e=r-1}^r{k}_{ne}^{j_1\dots j_r}{n}^{-e}$ where

$$\begin{array}{cc}& k_{n1}^{j_1{j}_2}=\sum _{\left|T\right|<n}k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1{j}_2}{0T}}\right),k_{n,r-1}^{j_1\dots j_r}=\sum _{|T_i|<n,i=2,\dots ,r}k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{0T_2\dots T_r}}\right),\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill & \mathrm{and}{k}_{nr}^{j_1\dots j_r}=-\sum _{|T_i|<n,i=2,\dots ,r}{\delta }_r\left(\mathbf{T}\right)k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{0T_2\dots T_r}}\right).\hfill \end{array}$$

(39)

This proves that a two-term version of (26) holds. So the expansions (27)–(29) and (33) hold for the density and distribution of $n^{1/2}(\overline{X}-\mu )$ with $V=\left(k_{n1}^{j_1{j}_2}\right)$ of (38), $k_i^{j_1\dots j_r}=k_{ni}^{j_1\dots j_r}$ of (38), (39), and ${\overline{k}}_1^1=0$. If the cross-cumulants $k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{0T_2\dots T_r}}\right)$ decrease exponentially in $T_2$, then for $r\ge 2$,

$$\begin{array}{c}{k}_{n,r-1}^{j_1\dots j_r}=k_{r-1}^{j_1\dots j_r}+O\left(e^{-n{\lambda }_r}\right), \mathrm{and} k_{nr}^{j_1\dots j_r}=k_r^{j_1\dots j_r}+O\left(e^{-n{\lambda }_r}\right)\mathrm{where}{\lambda }_r>0,\hfill \\ k_1^{j_1{j}_2}=\sum _{\left|T\right|<\infty }k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1{j}_2}{0T}}\right),k_{r-1}^{j_1\dots j_r}=\sum _{|T_i|<\infty ,i=2,\dots ,r}k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{0T_2\dots T_r}}\right),\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill & \mathrm{and}{k}_r^{j_1\dots j_r}=-\sum _{|T_i|<\infty ,i=2,\dots ,r}{\delta }_r\left(\mathbf{T}\right)k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{0T_2\dots T_r}}\right),\hfill \end{array}$$

(41)

so that for $X_0$ non-lattice, the expansions (27)–(29) and (33) hold for the density and distribution of $Y_n=n^{1/2}(\overline{X}-\mu )$ with $V=\left(k_1^{j_1{j}_2}\right)$ of (40), and ${\overline{k}}_1^1=0$.

These results extend to missing data and weighted means as in Section 2.

5. Discussion and Conclusions

References [1,2] showed that their quantile expansion, updated here as (4), can give great accuracy, in fact to many decimal places. However, for a given n and a large enough x, or for a given x and a small enough n, the expansions (2)–(4) will clearly diverge.

We have shown that the sample mean from a stationary process is a standard estimate of the mean of the process, so that we can apply the Edgeworth–Cornish–Fisher expansions given in Section 1 and Section 3 for any standard estimate. We also showed that remarkably, its cumulant expansion has only two terms. This simplifies the forms for $h_{kr}\left(x\right),f_r\left(x\right),g_r\left(x\right),$ and $r\ge 3$ of Section 1, and for $p_r\left(x\right),P_r\left(x\right),$ and $r\ge 3$ of Section 3. And we showed that these results extend to missing data and to weighted means.

As $E\overline{X}=\mu ,a_{11}=0$. So the expansions of Section 1 to $O\left(n^{-3/2}\right)$ require $n,x$ and $a_{32},a_{22},a_{43}$ of (18) and (19) as input, or alternatively, $a_{32},a_{22},a_{43}$ of (20) and (21). That is, one needs $n,x$ and the first three cross-cumulants of (13), $k(0T_2\dots T_r)$, for $r=2,3,4.$

Similarly, for multivariate series, the Edgeworth expansions of (27) to $O\left(n^{-3/2}\right)$ for $Y_n=n^{1/2}(\overline{X}-\mu )$ require $p_r\left(x\right),P_r\left(x\right),r=1,2$ of (31)–(32) with ${\overline{k}}_1^1=0$. These require $V=\left(k_1^{j_1{j}_2}\right),{\overline{k}}_1^1=0,{\overline{k}}_2^{1-3},{\overline{k}}_3^{1-4}$ of (38), (39) as inputs, or their limits in (40), (41). That is, one needs $n,x$ and the cross-cumulants, $k\left({\displaystyle \genfrac{}{}{0pt}{}{j_1\dots j_r}{0T_2\dots T_r}}\right)$ of (35) for $r=2,3,4$.

In general, non-parametric methods are to be preferred to parametric methods, as a wrong parametric model will give wrong results, even asymptotically. However, most econometric models are parametric. Reference [10] considered the case of the stationary linear process

$$X_i=\sum _{j=0}^{\infty }{\rho }_j{e}_{i-j},$$

where ${\rho }_j,j\ge 0,$ are given constants, and $\left\{e_i\right\}$ are i.i.d. random variables with finite cumulants ${\tau }_r,r\ge 1.$ For this very general class of semi-parametric models,

$$k(i_1,\dots ,i_r)=\alpha (i_1,\dots ,i_r){\tau }_r\mathrm{where}\alpha (i_1,\dots ,i_r)=\sum _{j=0}^{\infty }{\rho }_{j+i_1}\dots {\rho }_{j+i_r}.$$

Example 1.

For the AR(1) model, $X_i-\phi X_{i-1}=e_i$ with $\left|\phi \right|<1$,

$${\rho }_j={\phi }^j,\alpha (i_1,\dots ,i_r)={\phi }^{I_1+\dots +I_r}$$

for $I_k$ of (14). So the $k(i_1,\dots ,i_r)$ we need are given by φ and ${\tau }_r,r=2,3,4.$ For related work, see [18,19].

For a particular case like this, one can write a general computational code to plot graphs or make tables for various scenarios.

The theory of linear processes is well developed. See, for example, [20,21,22,23].

6. Four Examples of Future Directions

Traditionally, analysis of time series relies on parametric models, such as the autoregressive model of Example 1, or much more generally, ARIMA models, or the use of spectral theory that moves considerations from the time domain to the frequency domain. But as noted, a non-parametric approach is much to be preferred as the wrong parametric model gives incorrect results. A non-parametric estimate of the asymptotic variance $a_{21}={\sum }_{\left|T\right|<\infty }k\left(0T\right)$ of (20), is

$${\widehat{a}}_{21}=\sum _{\left|T\right|<N_n}\widehat{k}\left(0T\right)$$

where $\widehat{k}\left(0T\right)$ is the empirical estimate of $k\left(0T\right)$ and $N_n\to \infty .$ This should give one- and two-sided confidence intervals for $\mu$ of error $O\left(n^{-1/2}\right)$ and $O\left(n^{-1}\right)$. It should also be possible to reduce the one-sided error to $O\left(n^{-1}\right)$ or $O\left(n^{-3/2}\right)$ with (more complicated) confidence intervals, as performed for the i.i.d. case in [4].

A second area where an extension should be possible is to sample central moments and smooth functions of them, as achieved for i.i.d. observations for non-parametric and parametric statistics in [3,4,15] and for linear processes in [10]. Again, it should be possible to develop confidence intervals.

A third, more difficult extension would be a small sample version, using the method of [14]. This would remove the problem of series divergence.

A fourth extension would be to a kernel estimate of the marginal density of $X_0$. The authors of [24] gave Cornish–Fisher expansions for these based on a random sample. The expansions are in powers of ${\left(nh\right)}^{-1/2}$, not $n^{-1/2}$, where $h=c{n}^{-\alpha }$ and $\alpha >0$ can be made as small as desired by choice of kernel.