Lacunary Series and Strong Approximation ^†

Abstract

Strong approximation, introduced by Strassen (1964), is one of the most powerful methods to prove limit theorems in probability and statistics. In this paper we use strong approximation of lacunary series with conditionally independent sequences to prove uniform and permutation-invariant limit theorems for such series.

1. Introduction

It is known that sufficiently thin subsequences of any (dependent) sequence of random variables behave like independent random variables. Révész [1] proved that if ${\left(X_n\right)}_{n\ge 1}$ is a sequence of random variables satisfying ${sup}_n\mathbb{E}\left(X_n^2\right)<\infty$, then there exists a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and a random variable $X\in L_2$ such that the series ${\sum }_{k=1}^{\infty }{a}_k(X_{n_k}-X)$ converges a.s. for any coefficient sequence ${\left(a_k\right)}_{k\ge 1}$ with ${\sum }_{k=1}^{\infty }{a}_k^2<\infty$. Komlós [2] proved that from any sequence ${\left(X_n\right)}_{n\ge 1}$ of random variables satisfying ${sup}_n\mathbb{E}\left|X_n\right|<\infty$, one can select a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ such that

$$\underset{N\to \infty }{lim}{\displaystyle \frac{1}{N}}\sum _{k\le N}{X}_{n_k}=X\mathrm{a}.\mathrm{s}.$$

(1)

for some $X\in L_1$. Gaposhkin [3] and Chatterji [4,5] proved that if ${\left(X_n\right)}_{n\ge 1}$ is a sequence of random variables satisfying ${sup}_n\mathbb{E}{X}_n^2<\infty$, then there exist a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and random variables $X\in L_2$, $Y\in L_1$, $Y\ge 0$ such that

$${\displaystyle \frac{1}{\sqrt{N}}}\sum _{k\le N}(X_{n_k}-X)\stackrel{d}{⟶}N(0,Y)$$

(2)

and

$$\underset{N\to \infty }{lim\; sup}{\displaystyle \frac{1}{\sqrt{2N\log \log N}}}\sum _{k\le N}(X_{n_k}-X)=Y^{1/2}\mathrm{a}.\mathrm{s}.,$$

(3)

where $N(0,Y)$ denotes the distribution of $\sqrt{Y}g$; here g is an $N(0,1)$ variable independent of Y. Chatterji [6] formulated the following heuristic principle:

Subsequence Principle. 

Let T be a probability limit theorem valid for all sequences of i.i.d. random variables belonging to an integrability class L defined by the finiteness of a norm ${\parallel ·\parallel }_L$. Then, if ${\left(X_n\right)}_{n\ge 1}$ is an arbitrary (dependent) sequence of random variables satisfying ${sup}_n{\parallel X_n\parallel }_L<\infty$, then there exists a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ satisfying T in a mixed form.

In a profound study, Aldous [7] proved the validity of the subsequence principle for all distributional and almost sure limit theorems subject to minor technical conditions. Dacunha-Castelle [8] showed that every tight sequence ${\left(X_n\right)}_{n\ge 1}$ has a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ whose finite dimensional distributions are close to those of an exchangeable sequence ${\left(Y_k\right)}_{k\ge 1}$, defined on a possibly different probability space. However, the closeness of the finite dimensional distributions of ${\left(X_{n_k}\right)}_{k\ge 1}$ and ${\left(Y_k\right)}_{k\ge 1}$ is not enough to transfer limit theorems from ${\left(X_{n_k}\right)}_{k\ge 1}$ to ${\left(Y_k\right)}_{k\ge 1}$ and Aldous [7] used a delicate subsequence extraction technique tailored to the individual limit theorem we want to establish for $\left(X_{n_k}\right)$.

An alternative way to prove Aldous’ theorem would be to use strong approximation and to show that every tight sequence ${\left(X_n\right)}_{n\ge 1}$ has a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ which is close to an exchangeable sequence ${\left(Y_k\right)}_{k\ge 1}$ defined on the same probability space in the sense that

$$\sum _{k=1}^{\infty }|X_{n_k}-Y_k|<\infty \mathrm{a}.\mathrm{s}.$$

(4)

Clearly, by passing to a further subsequence, one can make the speed of a.s. convergence of $X_{n_k}-Y_k$ to 0 as rapid as one wishes; then, transferring limit theorems from ${\left(Y_k\right)}_{k\ge 1}$ to ${\left(X_{n_k}\right)}_{k\ge 1}$ becomes easy. Unfortunately, however, the approximation (4) is not valid in general, as is shown by the examples in [7,9]. For a necessary and sufficient condition on $\left(X_n\right)$ to have a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ satisfying (4) with an exchangeable ${\left(Y_k\right)}_{k\ge 1}$, see [9]. However, in [10] we showed that a slightly weaker version of (4) is nevertheless true, namely we have

Theorem 1.

Let ${\left(X_n\right)}_{n\ge 1}$ be an arbitrary (not necessarily tight) sequence of random variables with tail σ-field $\mathcal{T}$. Then, after suitably enlarging the probability space, there exists a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and a sequence ${\left(Y_k\right)}_{k\ge 1}$ of random variables such that $Y_k$ are conditionally independent with respect to $\mathcal{T}$ and (4) holds.

Theorem 1 is an almost sure invariance principle in the sense of Strassen [11], but it yields a.s. approximation of the individual r.v’s $X_{n_k}$, instead of their partial sums.

By De Finetti’s theorem, the exchangeability of a sequence ${\left(Y_k\right)}_{k\ge 1}$ can be “split” into two properties, namely, to the conditional independence of ${\left(Y_k\right)}_{k\ge 1}$ relative to its tail $\sigma$-field $\mathcal{T}$ and the conditional identical distribution of ${\left(Y_k\right)}_{k\ge 1}$ relative to $\mathcal{T}$. As Theorem 1 shows, the conditional independence of ${\left(Y_k\right)}_{k\ge 1}$ can always be guaranteed in (4) and the difficulties are caused by the second, seemingly much simpler property, the conditional identical distribution of the $Y_k$’s. In [10] we gave an example showing that even in the case when the sequence ${\left(X_n\right)}_{n\ge 1}$ is uniformly bounded, the behavior of the conditional distribution functions ${\mathcal{F}}_k\left(t\right)=\mathbb{P}(Y_k<t|\mathcal{T})$ of $Y_k$ can be extremely irregular, with no subsequence ${\mathcal{F}}_{m_k}$ converging in any useful sense. However, even without the conditional identical distribution of the $Y_k$’s, the approximation (4) has many useful consequences. As the examples in the next section will show, Theorem 1 provides important information for nonstationary lacunary series, e.g., for the a.s. convergence and asymptotic properties of series $\sum a_k{X}_{n_k}$. For example, while the Aldous–Chatterji subsequence theory yields the Hartman–Wintner-type LIL (3) for sufficiently thin subsequences ${\left(X_{n_k}\right)}_{k\ge 1}$ of $L_2$-bounded sequences ${\left(X_n\right)}_{n\ge 1}$, Theorem 1 yields the Kolmogorov-type LIL

$$\underset{N\to \infty }{lim\; sup}{\displaystyle \frac{{\sum }_{k=1}^N{a}_k(X_{n_k}-X)}{\sqrt{2A_N^2\log \log A_N}}}=Y^{1/2}\mathrm{a}.\mathrm{s}.$$

(5)

for suitable subsequences ${\left(X_{n_k}\right)}_{k\ge 1}$ of uniformly bounded sequences ${\left(X_n\right)}_{n\ge 1}$, where ${\left(a_n\right)}_{n\ge 1}$ is a weight sequence satisfying

$$A_N={\left(\sum _{k=1}^N{a}_k^2\right)}^{1/2}⟶\infty ,\underset{k\le N}{max}\left|a_k\right|=o\left({\displaystyle \frac{A_N}{\sqrt{\log \log A_n}}}\right).$$

(6)

In conclusion we note that in [12] it is proved that for any tight sequence ${\left(X_n\right)}_{n\ge 1}$, there is a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and a sequence ${\left(Y_k\right)}_{k\ge 1}$ of random variables which is ‘strongly exchangeable at infinity’ in a certain technical sense and (4) holds. Thus, we have an a.s. approximation theorem even in the context of the Chatterji–Aldous subsequence theory, but it applies only for limit theorems for i.i.d. or near-i.i.d. sequences.

2. Applications

By the result of Révész [1] cited in the Introduction, if ${\left(X_n\right)}_{n\ge 1}$ is a sequence of r.v.’s with ${sup}_n\mathbb{E}\left(X_n^2\right)<\infty$, then there exist a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and a random variable $X\in L_2$ such that ${(X_{n_k}-X)}_{k\ge 1}$ is a convergence system, i.e., for any numerical sequence ${\left(c_k\right)}_{k\ge 1}$ with ${\sum }_{k=1}^{\infty }{c}_k^2<\infty$, the series ${\sum }_{k=1}^{\infty }{c}_k(X_{n_k}-X)$ converges almost surely. Komlós [13] proved that with a suitable choice of $\left(n_k\right)$ and X, ${(X_{n_k}-X)}_{k\ge 1}$ will actually be an unconditional convergence system (i.e., it will be a convergence system after any permutation of its terms), settling a long-standing open problem in the theory of orthogonal series, see [14], p. 54. Another, equally elaborate proof of Komlós’ theorem was given by Aldous [7]. Since conditional independence is a permutation-invariant property, the two-series version of the Kolmogorov three-series criterion and Beppo Levi’s theorem imply that a conditionally independent sequence with conditional mean 0 and bounded second moments is an unconditional convergence system. Thus, observing that (4) and ${\sum }_{k=1}^{\infty }{c}_k^2<\infty$ imply

$$\sum _{k=1}^{\infty }|c_k\left|\right|X_{n_k}-Y_k|\le {\left(\sum _{k=1}^{\infty }{c}_k^2\right)}^{1/2}{\left(\sum _{k=1}^{\infty }{|X_{n_k}-Y_k|}^2\right)}^{1/2}<\infty \mathrm{a}.\mathrm{s}.,$$

the following version of Komlós’ theorem follows immediately from Theorem 1:

Theorem 2.

Let ${\left(X_n\right)}_{n\ge 1}$ be a sequence of r.v.’s with tail σ-algebra $\mathcal{T}$ such that ${sup}_n\mathbb{E}\left(X_n^2\right)<\infty$. Then ${(X_n-\mathbb{E}\left(X_n\right|\mathcal{T}))}_{n\ge 1}$ contains an unconditional convergence system.

By passing to a further subsequence and using a weak compactness theorem, the centering sequence $\mathbb{E}\left(X_n\right|\mathcal{T})$ can be replaced by a single random variable $X\in L_2$.

Proving the unconditionality of a convergence system is a difficult problem of analysis, with a number of famous results and open problems. By Carleson’s theorem, ${(cosnx)}_{n\ge 1}$ is a convergence system, but, as Kolmogorov pointed out in [15], this property breaks down after a suitable permutation of the sequence. In the opposite direction, Garsia [16] showed that if ${\left(f_n\right)}_{n\ge 1}$ is an orthonormal system and ${\sum }_{n=1}^{\infty }{c}_n^2<\infty$, then the sum ${\sum }_{n=1}^{\infty }{c}_n{f}_n$ is a.e. convergent after a suitable permutation of its terms. But whether there is a universal permutation $\sigma :\mathbb{N}\to \mathbb{N}$ of the positive integers such that ${\sum }_{n=1}^{\infty }{c}_n{f}_{\sigma \left(n\right)}$ is a.e. convergent for any coefficient sequence ${\left(c_n\right)}_{n\ge 1}$ with ${\sum }_{n=1}^{\infty }{c}_n^2<\infty$ (i.e., ${\left(f_{\sigma \left(n\right)}\right)}_{n\ge 1}$ is a convergence system) is still an open question.

As mentioned above, the subsequence ${\left(n_k\right)}_{k\ge 1}$ constructed in the proof of Aldous’ theorem depends on the limit theorem we want to verify and different limit theorems require different subsequences. For example, Aldous’ theorem implies that if $\left(X_n\right)$ is a sequence of r.v.’s with ${sup}_n\mathbb{E}\left(X_n^2\right)<\infty$, then for any fixed numerical sequence ${\left(c_k\right)}_{k\ge 1}$ with ${\sum }_{k=1}^{\infty }{c}_k^2<\infty$, there exist a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and $X\in L_2$ such that ${\sum }_{k=1}^{\infty }{c}_k(X_{n_k}-X)$ converges a.e., but the theorem does not yield a universal subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ working for all square summable sequences ${\left(c_k\right)}_{k\ge 1}$ simultaneously, and thus Révész’ theorem mentioned in the Introduction does not follow from Aldous’ theorem. The same applies for the weighted CLT and LIL for lacunary sequences proved by Gaposhkin (see [17], Theorem 1.5.1 and Theorem 1.6.1). However, both results are immediate consequences of Theorem 1 and since conditional independence is a permutation-invariant property, we automatically get their permutation-invariance as well:

Theorem 3.

Let ${\left(X_n\right)}_{n\ge 1}$ be a uniformly bounded sequence of random variables with ${\parallel X_n\parallel }_2=1$$(n=1,2,\dots )$. Then, there exists a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and bounded random variables X and $Y\ge 0$ such that

$$\underset{N\to \infty }{lim}{A}_N^{-1}\sum _{k=1}^N{a}_k(X_{n_k}-X)\stackrel{d}{⟶}N(0,Y)$$

(7)

for any positive numerical sequence ${\left(a_n\right)}_{n\ge 1}$ satisfying

$$A_N={\left(\sum _{k=1}^N{a}_k^2\right)}^{1/2}⟶\infty ,\underset{k\le N}{max}\left|a_k\right|=o\left(A_N\right).$$

Moreover, relation (7) remains valid after any permutation of the sequence ${\left(X_{n_k}\right)}_{k\ge 1}$.

Theorem 4.

Let ${\left(X_n\right)}_{n\ge 1}$ be a uniformly bounded sequence of random variables with ${\parallel X_n\parallel }_2=1$$(n=1,2,\dots )$. Then, there exist a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and bounded random variables X and $Y\ge 0$ such that (5) holds for any positive numerical sequence ${\left(a_n\right)}_{n\ge 1}$ satisfying (6). Moreover, relation (5) remains valid after any permutation of the sequence ${\left(X_{n_k}\right)}_{k\ge 1}.$

The unpermuted forms of Theorems 3 and 4 are due to Gaposhkin [17]. To deduce the permutation-invariant forms from Theorem 1, it suffices to use the version of the CLT and LIL for independent random variables due to Kolmogorov [18] and Lévy ([19], p. 105), the permutation-invariance of conditional independence and the observation that by (4) we have

$$\sum _{k=1}^N|a_k\left|\right|X_{n_k}-Y_k|\le (\underset{k\le N}{max}|a_k\left|\right)\sum _{k=1}^N|X_{n_k}-Y_k|=o\left(A_N\right).$$

Next, we formulate a version of the Kolmogorov–Erdos–Feller–Petrowski upper–lower class test for lacunary series.

Theorem 5.

Let ${\left(X_n\right)}_{n\ge 1}$ be a uniformly bounded sequence of random variables with ${\parallel X_n\parallel }_2=1$$(n=1,2,\dots )$. Then there exists a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and bounded random variables X and $Y\ge 0$ such that for any numerical sequence ${\left(a_n\right)}_{n\ge 1}$ satisfying

$$A_N={\left(\sum _{k=1}^N{a}_k^2\right)}^{1/2}⟶\infty ,\underset{k\le N}{max}\left|a_k\right|=o\left({\displaystyle \frac{A_N}{{(loglog{A}_n)}^{3/2}}}\right)$$

(8)

we have

$$\mathbb{P}\left(\sum _{k=1}^n{a}_k{X}_{n_k}>Y^{1/2}{A}_n\phi \left(A_n\right)i.o.\right)=1\mathrm{or}0$$

(9)

according as

$$\sum _{n=1}^{\infty }{\displaystyle \frac{\phi \left(n\right)}{n}}{e}^{-\phi {\left(n\right)}^2/2}=\infty \mathrm{or}<\infty .$$

(10)

Moreover, the result remains valid after any permutation of the sequence ${\left(X_{n_k}\right)}_{k\ge 1}$.

Again, the result follows from the corresponding result for independent random variables in [20], Theorem 1, and the observation that

$$\sum _{k=1}^N|a_k\left|\right|X_{n_k}-Y_k|\le (\underset{k\le N}{max}|a_k\left|\right)\sum _{k=1}^N|X_{n_k}-Y_k|=o(A_N/{(loglog{A}_N)}^{3/2}).$$

Replacing the exponent $3/2$ in (8) by $1/2<\rho <3/2$, the upper–lower class test (9)–(10) will continue to hold with new terms appearing gradually in the exponent of $e^{-\phi {\left(n\right)}^2/2}$ as $\rho$ approaches 1/2, according to the hierarchy of results described in [20].

In conclusion, we formulate a weighted strong law for lacunary sequences.

Theorem 6.

Let ${\left(X_n\right)}_{n\ge 1}$ be a sequence of random variables, let $p,q>1$ satisfying $1/p+1/q=1$, and assume that there exists a random variable $X\in L_p$ such that

$$\mathbb{P}\left(\right|X_n|>t)\le C\mathbb{P}\left(\right|X|>t)\mathit{for}\mathit{some}\mathit{constant}C>0\mathit{and}n=1,2,\dots .$$

(11)

Then there exist a subsequence ${\left(X_{n_k}\right)}_{k\ge 1}$ and a random variable $X\in L_p$ such that for any array ${\left(a_{N,i}\right)}_{N\ge 1,1\le i\le N}$ satisfying

$$\underset{N}{sup}{\left({\displaystyle \frac{1}{N}}\sum _{i=1}^N{\left|a_{N,i}\right|}^q\right)}^{1/q}<\infty$$

(12)

we have

$$\underset{N\to \infty }{lim}{\displaystyle \frac{1}{N}}\sum _{i=1}^N{a}_{N,i}(X_{n_i}-X)=0a.s.$$

Moreover, the result remains valid after any permutation of the sequence ${\left(X_{n_k}\right)}_{k\ge 1}$.

This follows immediately from Theorem 1 and the conditional version of Theorem 1.1 of [21] upon observing that (4) and (12) imply

$${\displaystyle \frac{1}{N}}\sum _{i=1}^N{a}_{N,i}|X_{n_i}-Y_i|\le {\left({\displaystyle \frac{1}{N}}\sum _{i=1}^N{\left|a_{N,i}\right|}^q\right)}^{1/q}{\left(\sum _{i=1}^N{|X_{n_i}-Y_i|}^p\right)}^{1/p}{N}^{-(q-1)/q}$$

and noting that the identical distribution of r.v.’s in Theorem 1.1 of [21] can be replaced by the stochastic domination condition (11). Finally, the permutation-invariance statement follows from the permutation-invariance of conditional independence.

Note that the the assumption (11) and $X\in L_p$ made on $X_n$ in Theorem 6 are stronger than the moment boundedness condition ${sup}_n\mathbb{E}{\left|X_n\right|}^p<\infty$, typically assumed in the theory of lacunary series. Whether Theorem 6 remains valid under ${sup}_n\mathbb{E}{\left|X_n\right|}^p<\infty$ remains open.

In conclusion we mention an interesting recent paper of Karatzas and Schachermayer [22] on the weak law of large numbers for lacunary series. The main result of [22] can also be deduced by using Theorem 1 and the classical criteria for the weak law of large numbers for independent random variables, but since such a proof would not be simpler than the original proof in [22], we do not discuss this problem here.