A Central Limit Theorem for Predictive Distributions

Abstract

Let S be a Borel subset of a Polish space and F the set of bounded Borel functions $f:S\to \mathbb{R}$. Let $a_n(·)=P(X_{n+1}\in ·\mid X_1,\dots ,X_n)$ be the n-th predictive distribution corresponding to a sequence $\left(X_n\right)$ of S-valued random variables. If $\left(X_n\right)$ is conditionally identically distributed, there is a random probability measure $\mu$ on S such that $\int fd{a}_n\stackrel{a.s.}{⟶}\int fd\mu$ for all $f\in F$. Define $D_n\left(f\right)=d_n\left\{\int fd{a}_n-\int fd\mu \right\}$ for all $f\in F$, where $d_n>0$ is a constant. In this note, it is shown that, under some conditions on $\left(X_n\right)$ and with a suitable choice of $d_n$, the finite dimensional distributions of the process $D_n=\left\{D_n\left(f\right):f\in F\right\}$ stably converge to a Gaussian kernel with a known covariance structure. In addition, $E\left\{\phi \left(D_n\left(f\right)\right)\mid X_1,\dots ,X_n\right\}$ converges in probability for all $f\in F$ and $\phi \in C_b\left(\mathbb{R}\right)$.

1. Introduction

All random elements appearing in the sequel are defined on a common probability space, say $(\Omega ,\mathcal{A},P)$. We denote by S a Borel subset of a Polish space and by $\mathcal{B}$ the Borel $\sigma$-field on S. We let

$$\begin{array}{c}\mathcal{P}=\left\{\mathrm{probability}\mathrm{measures}\mathrm{on}\mathcal{B}\right\}\mathrm{and}\\ F=\left\{\mathrm{real}\mathrm{bounded}\mathrm{Borel}\mathrm{functions}\mathrm{on}S\right\}.\end{array}$$

Moreover, if $\lambda \in \mathcal{P}$ and $f\in F$, we write $\lambda \left(f\right)$ to denote

$$\lambda (f)=\int fd\lambda .$$

In other terms, depending on the context, $\lambda$ is regarded as a function on $\mathcal{B}$ or a function on F. This slight abuse of notation is quite usual (see, e.g., [1,2]) and very useful for the purposes of this note.

Let

$$X=(X_1,X_2,\dots )$$

be a sequence of S-valued random variables and

$${\mathcal{F}}_0=\{\varnothing ,\Omega \}\mathrm{and}{\mathcal{F}}_n=\sigma (X_1,\dots ,X_n).$$

The predictive distributions of X are the random probability measures on $(S,\mathcal{B})$ given by

$$a_n(·)=P(X_{n+1}\in ·\mid {\mathcal{F}}_n)\mathrm{for}\mathrm{all}n\ge 0.$$

Under some conditions, there is a further random probability measure $\mu$ on $(S,\mathcal{B})$ such that

$$\mu \left(f\right)\stackrel{a.s.}{=}\underset{n}{lim}{a}_n\left(f\right)\mathrm{for}\mathrm{each}f\in F.$$

(1)

For instance, condition (1) holds if X is exchangeable. More generally, it holds if X is conditionally identically distributed (c.i.d.), as defined in Section 2. Note also that, since S is separable, condition (1) implies $a_n\to \mu$ weakly. Regarding $a_n$ and $\mu$ as measurable functions from $\Omega$ into $\mathcal{P}$, one obtains

$$P\left(\{\omega \in \Omega :a_{n,\omega }\to {\mu }_{\omega }\mathrm{weakly}\}\right)=1.$$

Assume condition (1), fix a sequence $d_n$ of positive constants, and define

$$D_n\left(f\right)=d_n\left\{a_n\left(f\right)-\mu \left(f\right)\right\}\mathrm{for}\mathrm{each}f\in F.$$

This note deals with the process

$$D_n=\left\{D_n\left(f\right):f\in F\right\}.$$

Our goal is to show that, under some conditions on X and with a suitable choice of the constants $d_n$, the finite-dimensional distributions of $D_n$ stably converge, as $n\to \infty$, to a certain Gaussian limit.

To be more precise, we recall that a kernel on $(S,\mathcal{B})$ is a measurable map $\alpha :S\to \mathcal{P}$. This means that $\alpha \left(x\right)\in \mathcal{P}$, for each $x\in S$, and the function $x\mapsto \alpha \left(x\right)\left(A\right)$ is $\mathcal{B}$-measurable for each $A\in \mathcal{B}$. In what follows, we write

$$\alpha (x)(f)=\int f(y)\alpha (x)(dy)\mathrm{for}\mathrm{all}x\in S\mathrm{and}f\in F.$$

Next, as in [3], suppose the predictive distributions of X satisfy the recursive equation

$$a_{n+1}=q_n{a}_n+(1-q_n)\alpha \left(X_{n+1}\right)\mathrm{a}.\mathrm{s}.\mathrm{for}\mathrm{all}n\ge 0,$$

(2)

where $q_0,q_1,\dots \in (0,1)$ are constants and $\alpha$ is a kernel on $(S,\mathcal{B})$. Moreover, let

$$\nu (·)=P(X_1\in ·)$$

be the marginal distribution of $X_1$. Under condition (2), X is c.i.d. whenever $\alpha$ is a regular conditional distribution for $\nu$ given a sub-$\sigma$-field $\mathcal{G}\subset \mathcal{B}$; see ([3] Section 5). Hence, we assume

$$\alpha (·)\left(A\right)=E_{\nu }(1_A\mid \mathcal{G}),\nu -\mathrm{a}.\mathrm{s}.,$$

(3)

for all $A\in \mathcal{B}$ and some sub-$\sigma$-field $\mathcal{G}\subset \mathcal{B}$. For instance, condition (3) holds if

$$\alpha \left(x\right)={\delta }_x\mathrm{for}\mathrm{all}x\in S$$

where ${\delta }_x$ denotes the unit mass at the point x (just let $\mathcal{G}=\mathcal{B}$). In addition, we assume

$$\sum _n{(1-q_n)}^2<\infty \mathrm{and}\underset{n}{lim}{d}_n\underset{k\ge n}{sup}(1-q_{k-1})=0$$

where

$$d_n={\left(\sum _{k\ge n}{(1-q_k)}^2\right)}^{-1/2}.$$

In this framework, it is shown that

$$\left(D_n\left(f_1\right),\dots ,D_n\left(f_p\right)\right)⟶{\mathcal{N}}_p(0,\Sigma )\mathrm{stably}$$

(4)

for all $p\ge 1$ and all $f_1,\dots ,f_p\in F$, where $\Sigma$ is the random covariance matrix with entries

$${\sigma }_{jk}=\int \alpha \left(x\right)\left(f_j\right)\alpha \left(x\right)\left(f_k\right)\mu \left(dx\right)-\mu \left(f_j\right)\mu \left(f_k\right).$$

We actually prove something more than (4). Let $C_b\left(\mathbb{R}\right)$ denote the set of real bounded continuous functions on $\mathbb{R}$. Then, it is shown that

$$E\left\{\phi \left(D_n\left(f\right)\right)\mid {\mathcal{F}}_n\right\} \stackrel{P}{⟶}\mathcal{N}(0,{\sigma }^2)\left(\phi \right)$$

(5)

for all $f\in F$ and $\phi \in C_b\left(\mathbb{R}\right)$, where

$${\sigma }^2=\int \alpha \left(x\right){\left(f\right)}^2\mu \left(dx\right)-\mu {\left(f\right)}^2.$$

Based on (5), it is not hard to deduce condition (4).

Before concluding the Introduction, several remarks are in order.

(i)

A remarkable special case is $\alpha \left(x\right)={\delta }_x$ for all $x\in S$. Indeed, Equation (2) holds with $\alpha =\delta$ in some meaningful situations, including Dirichlet sequences; see ([3] Section 4) for other examples. Thus, suppose $\alpha =\delta$. Then, the above formulae reduce to ${\sigma }_{jk}=\mu \left(f_j{f}_k\right)-\mu \left(f_j\right)\mu \left(f_k\right)$ and ${\sigma }^2=\mu \left(f^2\right)-\mu {\left(f\right)}^2$. Moreover, if $\nu$ is non-atomic and

$$\prod _{j=0}^n{q}_j\to 0\mathrm{and}\sum _n\prod _{j=0}^n{q}_j=\infty ,$$

then $\mu$ takes the form

$$\mu \stackrel{a.s.}{=}\sum _n{V}_n{\delta }_{Y_n}$$

where $\left(V_n\right)$ and $\left(Y_n\right)$ are independent sequences and $\left(Y_n\right)$ is i.i.d. with $Y_1\sim \nu$; see ([3] Theorem 20) and [4] for details.

(ii)

Let $l^{\infty }\left(G\right)$ be the set of real bounded functions on G, where G is any subset of F. For instance, if $S=\mathbb{R}$, one could take $G=\left\{1_{(-\infty ,x]}:x\in \mathbb{R}\right\}$. In view of (4), a natural question is whether $D_n$ has a limit in distribution when $l^{\infty }\left(G\right)$ is equipped with a suitable distance. As an example, $l^{\infty }\left(G\right)$ could be equipped with the uniform distance (as in [1,2]) or with some weaker distance (as in [5]). Even if natural, this question is neglected in this note. We hope and plan to investigate it in a forthcoming paper.

(iii)

For fixed $f\in F$, condition (4) provides some information on the convergence rate of $a_n\left(f\right)$ to $\mu \left(f\right)$. Define $L_n=u_n\left|a_n\left(f\right)-\mu \left(f\right)\right|$ where $u_n>0$ is any sequence of constants. Then, condition (4) yields $L_n\stackrel{P}{⟶}0$ whenever $u_n/d_n\to 0$. Furthermore, $L_n\stackrel{P}{⟶}\infty$ provided $u_n/d_n\to \infty$ and ${\sigma }^2>0$ a.s.

(iv)

The condition ${lim}_n{d}_n{sup}_{k\ge n}(1-q_{k-1})=0$ is just a technical assumption which guarantees that, asymptotically, there are no dominating terms. In a sense, this condition is analogous to the weak Lindeberg’s condition in the classical CLT for independent summands.

(v)

From a Bayesian point of view, $\mu$ can be seen as a random parameter of the data sequence X. This is quite clear if X is exchangeable, for, in this case, X is conditionally i.i.d. given $\mu$. If X is only c.i.d., the role of $\mu$ is not as crucial, but $\mu$ still contributes to specify the probability distribution of X; see ([3] Section 2.1). Thus, in a Bayesian framework, conditions (4)–(5) may be useful to make (asymptotic) inference about $\mu$. To this end, an alternative could be proving a limit theorem for $W_n=w_n({\mu }_n-\mu )$, where $w_n$ is a suitable constant and ${\mu }_n=(1/n){\sum }_{j=1}^n{\delta }_{X_j}$ the empirical measure. However, $D_n$ has two advantages with respect to $W_n$. It usually converges at a better rate and the variance of the limit distribution is smaller; see, e.g., Example 3.

(vi)

Conditions (4)–(5) are our main results. They can be motivated in at least two ways. Firstly, from the theoretical perspective, conditions (4)–(5) fit into the results concerning the asymptotic behavior of conditional expectations (see, e.g., [6,7,8] and references therein). Secondly, from the practical perspective, conditions (4)–(5) play a role in all those fields where predictive distributions are basic objects. The main example is Bayesian predictive inference. Indeed, the predictive distributions investigated in this note have been introduced in connection with Bayesian prediction problems; see [3]. Another example is the asymptotic behavior of certain urn schemes. Related subjects, where (4)–(5) are potentially useful, are empirical processes for dependent data, Glivenko-Cantelli-type theorems and merging of opinions. Without any claim of being exhaustive, a list of references is: [3,5,9,10,11,12,13,14,15,16,17,18,19,20,21].

2. Preliminaries

In this note, ${\mathcal{N}}_p(0,C)$ denotes the Gaussian law on the Borel sets of ${\mathbb{R}}^p$ with mean 0 and covariance matrix C, where C is symmetric and semidefinite positive. If $p=1$ and $c\ge 0$ is a scalar, we write $\mathcal{N}(0,c)$ instead of ${\mathcal{N}}_1(0,c)$ and

$$\mathcal{N}(0,c)(\phi )=\int \phi (x)\mathcal{N}(0,c)(dx)$$

for all bounded measurable $\phi :\mathbb{R}\to \mathbb{R}$. Note that, if $\Sigma$ is a random covariance matrix, ${\mathcal{N}}_p(0,\Sigma )$ is a random probability measure on the Borel sets of ${\mathbb{R}}^p$.

Let us briefly recall stable convergence. Let ${\mathcal{A}}^{+}=\{H\in \mathcal{A}:P\left(H\right)>0\}$. Fix a random probability measure K on $(S,\mathcal{B})$ and define

$${\lambda }_H\left(A\right)=E\left\{K\left(A\right)\mid H\right\}\mathrm{for}\mathrm{all}A\in \mathcal{B}\mathrm{and}H\in {\mathcal{A}}^{+}.$$

Each ${\lambda }_H$ is a probability measure on $\mathcal{B}$. Then, $X_n$ converges stably to K, written $X_n\to K$ stably, if

$$P(X_n\in ·\mid H)⟶{\lambda }_H\mathrm{weakly}\mathrm{for}\mathrm{all}H\in {\mathcal{A}}^{+}.$$

In particular, $X_n$ converges in distribution to ${\lambda }_{\Omega }$. However, stable convergence is stronger than convergence in distribution. To see this, take a further random variable $X:\Omega \to S$. Then, $X_n\stackrel{P}{⟶}X$ if, and only if, $X_n\to {\delta }_X$ stably. Thus, stable convergence is strictly connected to convergence in probability. Moreover, $(X_n,X)\to K\times {\delta }_X$ stably whenever $X_n\to K$ stably. Therefore, if $X_n$ converges stably, $(X_n,X)$ still converges stably for any S-valued random variable X.

We next turn to conditional identity in distribution. Say that X is conditionally identically distributed (c.i.d.) if

$$P\left(X_k\in ·\mid {\mathcal{F}}_n\right)=P\left(X_{n+1}\in ·\mid {\mathcal{F}}_n\right)\mathrm{a}.\mathrm{s}.\mathrm{for}\mathrm{all}k>n\ge 0.$$

Thus, at each time n, the future observations $(X_k:k>n)$ are identically distributed given the past. This is actually weaker than exchangeability. Indeed, X is exchangeable if, and only if, it is stationary and c.i.d.

C.i.d. sequences were introduced in [9,22] and then investigated in various papers; see, e.g., [3,4,5,11,23,24,25,26,27,28,29].

The asymptotics of c.i.d. sequences is similar to that of exchangeable ones. To see this, suppose X is c.i.d. and define the empirical measures

$${\mu }_n=\frac{1}{n}\sum _{j=1}^n{\delta }_{X_j}.$$

Then, there is a random probability measure $\mu$ on $(S,\mathcal{B})$ such that

$$\mu \left(A\right)\stackrel{a.s.}{=}\underset{m}{lim}{\mu }_m\left(A\right)\mathrm{for}\mathrm{each}\mathrm{fixed}A\in \mathcal{B}.$$

It follows that

$$\begin{array}{c}E\left\{\mu \left(A\right)\mid {\mathcal{F}}_n\right\}=\underset{m}{lim}E\left\{{\mu }_m\left(A\right)\mid {\mathcal{F}}_n\right\}\\ =\underset{m}{lim}\frac{1}{m}\sum _{j=n+1}^{m}P\left(X_j\in A\mid {\mathcal{F}}_n\right)=P\left(X_{n+1}\in A\mid {\mathcal{F}}_n\right)\mathrm{a}.\mathrm{s}.\end{array}$$

for all $n\ge 0$ and $A\in \mathcal{B}$. Therefore, as in the exchangeable case, the predictive distributions can be written as

$$a_n(·)=P\left(X_{n+1}\in ·\mid {\mathcal{F}}_n\right)=E\left\{\mu (·)\mid {\mathcal{F}}_n\right\}\mathrm{a}.\mathrm{s}.$$

Using the martingale convergence theorem, this implies

$$\mu \left(f\right)\stackrel{a.s.}{=}\underset{n}{lim}E\left\{\mu \left(f\right)\mid {\mathcal{F}}_n\right\}=\underset{n}{lim}{a}_n\left(f\right)\mathrm{for}\mathrm{all}f\in F.$$

Furthermore, X is asymptotically exchangeable, in the sense that the probability distribution of the shifted sequence $(X_n,X_{n+1},\dots )$ converges weakly to an exchangeable probability measure on $(S^{\infty },{\mathcal{B}}^{\infty })$.

Finally, we state a technical result to be used later on.

Lemma 1.

Let $\left(Y_n\right)$ be a sequence of real integrable random variables, adapted to the filtration $\left({\mathcal{F}}_n\right)$, and

$$Z_n=E(Y_{n+1}\mid {\mathcal{F}}_n).$$

Let V be a real non-negative random variable and $0<b_1<b_2<\dots$ an increasing sequence of constants, such that $b_n\uparrow \infty$ and $b_n/b_{n+1}\to 1$. Suppose $\left(Y_n^2\right)$ is uniformly integrable, $Z_n\stackrel{a.s.}{⟶}Z$ for some random variable Z, and define

$$T_n=b_n(Z_n-Z).$$

Then,

$$E\left\{\phi \left(T_n\right)\mid {\mathcal{F}}_n\right\} \stackrel{P}{⟶}\mathcal{N}(0,V)\left(\phi \right)\mathrm{for}\mathrm{all}\phi \in C_b\left(\mathbb{R}\right)$$

provided

$$b_n^2\sum _{k\ge n}{(Z_k-Z_{k-1})}^2\stackrel{P}{⟶}V;$$

(6)

$$\underset{n}{lim}{b}_{n}E\left\{\underset{k\ge n}{sup}\left|Z_k-Z_{k-1}\right|\right\}=0;$$

(7)

$$\sum _{k\ge n}E\left|E(Z_{k+1}\mid {\mathcal{F}}_k)-Z_k\right|=\mathrm{o}(1/b_n).$$

(8)

Proof. 

Just repeat the proof of ([10] Theorem 1) with $b_n$ in the place of $\sqrt{n}$. □

3. Main Result

Let us go back to the notation of Section 1. Recall that $q_n\in (0,1)$ is a constant for each $n\ge 0$ and $d_n={\left({\sum }_{k\ge n}{(1-q_k)}^2\right)}^{-1/2}$. We aim to prove the following CLT.

Theorem 1.

Assume conditions (2)–(3) and

$$\sum _n{(1-q_n)}^2<\infty and\underset{n}{lim}{d}_n\underset{k\ge n}{sup}(1-q_{k-1})=0.$$

Then, there is a random probability measure μ on $(S,\mathcal{B})$ such that

$$\mu \left(f\right)\stackrel{a.s.}{=}\underset{n}{lim}{a}_n\left(f\right)andE\left\{\phi \left(D_n\left(f\right)\right)\mid {\mathcal{F}}_n\right\}\stackrel{P}{⟶}\mathcal{N}(0,{\sigma }^2)\left(\phi \right)$$

for all $f\in F$ and $\phi \in C_b\left(\mathbb{R}\right)$, where

$${\sigma }^2=\int \alpha \left(x\right){\left(f\right)}^2\mu \left(dx\right)-\mu {\left(f\right)}^2.$$

As a consequence,

$$\left(D_n\left(f_1\right),\dots ,D_n\left(f_p\right)\right)⟶{\mathcal{N}}_p(0,\Sigma )stably$$

for all $p\ge 1$ and all $f_1,\dots ,f_p\in F$ where the covariance matrix Σ has entries

$${\sigma }_{jk}=\int \alpha \left(x\right)\left(f_j\right)\alpha \left(x\right)\left(f_k\right)\mu \left(dx\right)-\mu \left(f_j\right)\mu \left(f_k\right).$$

Proof. 

Due to conditions (2)–(3), X is c.i.d.; see ([3] Section 5). Hence, as noted in Section 2, there is a random probability measure $\mu$ on $(S,\mathcal{B})$ such that

$$a_n\left(f\right)\stackrel{a.s.}{=}E\left\{\mu \left(f\right)\mid {\mathcal{F}}_n\right\}\mathrm{for}\mathrm{all}f\in F.$$

By martingale convergence, it follows that $a_n\left(f\right)\stackrel{a.s.}{⟶}\mu \left(f\right)$ for all $f\in F$.

We next prove condition (5). Fix $f\in F$ and define

$$b_n=d_n,Y_n=a_n\left(f\right),Z=\mu \left(f\right)\mathrm{and}V={\sigma }^2.$$

Then, $\left(Y_n^2\right)$ is uniformly integrable (for f is bounded) and $b_n$ satisfies the conditions of Lemma 1. Moreover,

$$Z_n=E(Y_{n+1}\mid {\mathcal{F}}_n)=E\left\{E\left(\mu \left(f\right)\mid {\mathcal{F}}_{n+1}\right)\mid {\mathcal{F}}_n\right\}=E\left\{\mu \left(f\right)\mid {\mathcal{F}}_n\right\}=a_n\left(f\right)\mathrm{a}.\mathrm{s}.$$

so that $Z_n\stackrel{a.s.}{⟶}Z$. Therefore, Lemma 1 applies. Hence, to prove (5), it suffices to check conditions (6)–(8).

Let $c=sup\left|f\right|$. Since $E(Z_{k+1}\mid {\mathcal{F}}_k)=Z_k$ a.s., condition (8) is trivially true. Moreover, condition (2) implies

$$\begin{array}{ccc}{Z}_k-Z_{k-1}& =& a_k\left(f\right)-a_{k-1}\left(f\right)\hfill \\ & =& q_{k-1}{a}_{k-1}\left(f\right)+(1-q_{k-1})\alpha \left(X_k\right)\left(f\right)-a_{k-1}\left(f\right)\hfill \\ & =& (1-q_{k-1})\left\{\alpha \left(X_k\right)\left(f\right)-a_{k-1}\left(f\right)\right\}\mathrm{a}.\mathrm{s}.\mathrm{for}\mathrm{all}k\ge 1.\hfill \end{array}$$

Hence, condition (7) holds, since

$$d_{n}E\left\{\underset{k\ge n}{sup}\left|Z_k-Z_{k-1}\right|\right\}\le 2c{d}_n\underset{k\ge n}{sup}(1-q_{k-1})⟶0.$$

It remains to prove condition (6), namely

$$d_n^2\sum _{k\ge n}{(1-q_{k-1})}^2{\left\{\alpha \left(X_k\right)\left(f\right)-a_{k-1}\left(f\right)\right\}}^2\stackrel{P}{⟶}{\sigma }^2.$$

First note that, since $a_{k-1}{\left(f\right)}^2\stackrel{a.s.}{⟶}\mu {\left(f\right)}^2$ as $k\to \infty$, one obtains

$$d_n^2\sum _{k\ge n}{(1-q_{k-1})}^2{a}_{k-1}{\left(f\right)}^2=\frac{{\sum }_{k\ge n}{(1-q_{k-1})}^2{a}_{k-1}{\left(f\right)}^2}{{\sum }_{k\ge n}{(1-q_k)}^2}\stackrel{a.s.}{⟶}\mu {\left(f\right)}^2.$$

Next, define

$$R_k=\alpha \left(X_k\right){\left(f\right)}^2\mathrm{and}{M}_n=d_n^2\sum _{k\ge n}{(1-q_{k-1})}^2\left\{R_k-E(R_k\mid {\mathcal{F}}_{k-1})\right\}.$$

Then,

$$\begin{array}{ccc}E\left(M_n^2\right)& =& d_n^4\sum _{k\ge n}{(1-q_{k-1})}^{4}E\left\{{\left(R_k-E(R_k\mid {\mathcal{F}}_{k-1})\right)}^2\right\}\hfill \\ & \le & 4c^4{d}_n^4\sum _{k\ge n}{(1-q_{k-1})}^4\hfill \\ & \le & 4c^4{d}_n^2\underset{k\ge n}{sup}{(1-q_{k-1})}^2·d_n^2\sum _{k\ge n}{(1-q_{k-1})}^2\hfill \\ & ⟶& 0.\hfill \end{array}$$

Moreover,

$$E(R_k\mid {\mathcal{F}}_{k-1})=E\left\{\int \alpha \left(x\right){\left(f\right)}^2\mu \left(dx\right)\mid {\mathcal{F}}_{k-1}\right\}\stackrel{a.s.}{⟶}\int \alpha \left(x\right){\left(f\right)}^2\mu \left(dx\right).$$

Therefore,

$$d_n^2\sum _{k\ge n}{(1-q_{k-1})}^2{R}_k=M_n+d_n^2\sum _{k\ge n}{(1-q_{k-1})}^{2}E(R_k\mid {\mathcal{F}}_{k-1})\stackrel{P}{⟶}\int \alpha \left(x\right){\left(f\right)}^2\mu \left(dx\right).$$

By the same argument, it follows that

$$d_n^2\sum _{k\ge n}{(1-q_{k-1})}^2\alpha \left(X_k\right)\left(f\right)a_{k-1}\left(f\right)\stackrel{P}{⟶}\mu \left(f\right)\int \alpha \left(x\right)\left(f\right)\mu \left(dx\right).$$

In addition, as proved in the Claim below,

$$\int \alpha \left(x\right)\left(f\right)\mu \left(dx\right)\stackrel{a.s.}{=}\mu \left(f\right).$$

Collecting all pieces together, one finally obtains

$$d_n^2\sum _{k\ge n}{(1-q_{k-1})}^2{\left\{\alpha \left(X_k\right)\left(f\right)-a_{k-1}\left(f\right)\right\}}^2\stackrel{P}{⟶}\mu {\left(f\right)}^2+\int \alpha \left(x\right){\left(f\right)}^2\mu \left(dx\right)-2\mu {\left(f\right)}^2={\sigma }^2.$$

Hence, condition (6) holds.

This concludes the proof of (5). We next prove that (5) ⇒ (4). Let $p\ge 1$ and $f_1,\dots ,f_p\in F$. Fix $u_1,\dots ,u_p\in \mathbb{R}$ and define

$$U_n=\sum _{j=1}^p{u}_j{D}_n\left(f_j\right)\mathrm{and}{\sigma }_u^2=\sum _{j,k}{u}_j{u}_k{\sigma }_{jk}.$$

Moreover, for each $H\in {\mathcal{A}}^{+}$, define the probability measure

$${\lambda }_H\left(A\right)=E\left\{\mathcal{N}(0,{\sigma }_u^2)\left(A\right)\mid H\right\}\mathrm{for}\mathrm{each}\mathrm{Borel}\mathrm{set}A\subset \mathbb{R}.$$

We have to show that

$$P(U_n\in ·\mid H)⟶{\lambda }_H\mathrm{weakly}\mathrm{for}\mathrm{each}H\in {\mathcal{A}}^{+}.$$

(9)

To this end, call ${\varphi }_H$ the characteristic function of ${\lambda }_H$, namely

$${\varphi }_H\left(t\right)=E\left(\int e^{itx}\mathcal{N}(0,{\sigma }_u^2)\left(dx\right)\mid H\right)=E\left(e^{-t^2{\sigma }_u^2/2}\mid H\right)\mathrm{for}\mathrm{all}t\in \mathbb{R}.$$

Letting $f={\sum }_{j=1}^p{u}_j{f}_j$, one obtains

$$U_n=D_n\left(f\right)\mathrm{and}{\sigma }_u^2=\int \alpha \left(x\right){\left(f\right)}^2\mu \left(dx\right)-\mu {\left(f\right)}^2.$$

Therefore, condition (5) yields

$$E\left(e^{it{U}_n}\right)=E\left(E\left\{e^{it{D}_n\left(f\right)}\mid {\mathcal{F}}_n\right\}\right)⟶E\left(e^{-t^2{\sigma }_u^2/2}\right)={\varphi }_{\Omega }\left(t\right)$$

for each $t\in \mathbb{R}$. Hence, condition (9) holds for $H=\Omega$. Next, suppose $H\in {\bigcup }_n{\mathcal{F}}_n$ and $P\left(H\right)>0$. Then, for large n, one obtains

$$E\left(1_H{e}^{it{U}_n}\right)=E\left(1_{H}E\left\{e^{it{D}_n\left(f\right)}\mid {\mathcal{F}}_n\right\}\right).$$

Hence, for each $t\in \mathbb{R}$, condition (5) still implies

$$P\left(H\right){\varphi }_H\left(t\right)=E\left(1_H{e}^{-t^2{\sigma }_u^2/2}\right)=\underset{n}{lim}E\left(1_{H}E\left\{e^{it{D}_n\left(f\right)}\mid {\mathcal{F}}_n\right\}\right)=\underset{n}{lim}E\left(1_H{e}^{it{U}_n}\right).$$

Therefore, condition (9) holds whenever $H\in {\bigcup }_n{\mathcal{F}}_n$ and $P\left(H\right)>0$. Based on this fact, by standard arguments, condition (9) easily follows for each $H\in {\mathcal{A}}^{+}$.

To conclude the proof of the Theorem, it remains only to show that:

Claim: 

$\int \alpha \left(x\right)\left(f\right)\mu \left(dx\right)\stackrel{a.s.}{=}\mu \left(f\right)$ for all $f\in F$.

Proof of the Claim:

By (3), $\alpha$ is a regular conditional distribution for $\nu$ given a sub-$\sigma$-field of $\mathcal{B}$, where $\nu$ is the marginal distribution of $X_1$. Therefore, as proved in ([3] Lemma 6), there is a set $A\in \mathcal{B}$ such that $\nu \left(A\right)=1$ and

$$\int \alpha (z)(f)\alpha (x)(dz)=\alpha (x)(f)\mathrm{for}\mathrm{all}x\in A\mathrm{and}f\in F.$$

Since X is c.i.d. (and, thus, identically distributed) one also obtains $P(X_n\in A)=\nu \left(A\right)=1$ for all $n\ge 1$.

Having noted these facts, fix $f\in F$. Since $a_0=\nu$ and $\alpha$ is a regular conditional distribution for $\nu$,

$$\int \alpha \left(x\right)\left(f\right)a_0\left(dx\right)=a_0\left(f\right).$$

Moreover, if $\int \alpha \left(x\right)\left(f\right)a_n\left(dx\right)=a_n\left(f\right)$ a.s. for some $n\ge 0$, then

$$\begin{array}{ccc}\int \alpha \left(x\right)\left(f\right)a_{n+1}\left(dx\right)& =& q_n\int \alpha \left(x\right)\left(f\right)a_n\left(dx\right)+(1-q_n)\int \alpha \left(x\right)\left(f\right)\alpha \left(X_{n+1}\right)\left(dx\right)\hfill \\ & =& q_n{a}_n\left(f\right)+(1-q_n)\alpha \left(X_{n+1}\right)\left(f\right)\hfill \\ & =& a_{n+1}\left(f\right)\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

By induction, one obtains $\int \alpha \left(x\right)\left(f\right)a_n\left(dx\right)=a_n\left(f\right)$ a.s. for each $n\ge 0$. Hence,

$$\int \alpha \left(x\right)\left(f\right)\mu \left(dx\right)=\underset{n}{lim}\int \alpha \left(x\right)\left(f\right)a_n\left(dx\right)=\underset{n}{lim}{a}_n\left(f\right)=\mu \left(f\right)\mathrm{a}.\mathrm{s}.$$

. □

We do not know whether $E\left\{\phi \left(D_n\left(f\right)\right)\mid {\mathcal{F}}_n\right\}$ converges a.s. (and not only in probability) under the conditions of Theorem 1. However, it can be shown that $E\left\{\phi \left(D_n\left(f\right)\right)\mid {\mathcal{F}}_n\right\}$ converges a.s. under slightly stronger conditions on $q_n$.

Under conditions (2)–(3), for Theorem 1 to work, it suffices that

$$\underset{n}{lim}{n}^b(1-q_n)=c\mathrm{for}\mathrm{some}b>1/2\mathrm{and}c>0.$$

(10)

In addition, if (10) holds, then

$$\frac{n^{b-1/2}}{d_n}\to \frac{c}{\sqrt{2b-1}}.$$

Hence, letting $D_n^{*}=n^{b-1/2}(a_n-\mu )$, one obtains

$$\left(D_n^{*}\left(f_1\right),\dots ,D_n^{*}\left(f_p\right)\right)⟶{\mathcal{N}}_p\left(0,\frac{c^2}{2b-1}\Sigma \right)\mathrm{stably},$$

for all $p\ge 1$ and all $f_1,\dots ,f_p\in F$, provided conditions (2), (3) and (10) hold.

We close this note with some examples.

Example 1.

Let

$$q_n=\frac{n+{\theta }_n}{n+1+{\theta }_{n+1}}$$

where $\left({\theta }_n\right)$ is a bounded increasing sequence with ${\theta }_0>0$. Then, X is c.i.d. (because of (2)–(3)) but is exchangeable if and only if ${\theta }_n={\theta }_0$ for all n. In any case, since condition (10) holds with $b=c=1$, Theorem 1 applies and $d_n$ can be replaced by $\sqrt{n}$. Letting $D_n^{*}=\sqrt{n}(a_n-\mu )$, it follows that

$$\left(D_n^{*}\left(f_1\right),\dots ,D_n^{*}\left(f_p\right)\right)⟶{\mathcal{N}}_p\left(0,\Sigma \right)\mathrm{stably}.$$

It is worth noting that, in the special case ${\theta }_n={\theta }_0$ for all n, the predictive distributions of X reduce to

$$a_n=\frac{{\theta }_0\nu +{\sum }_{i=1}^n\alpha \left(X_i\right)}{n+{\theta }_0}.$$

Therefore, X is a Dirichlet sequence if $\alpha =\delta$. The general case, where α is any kernel satisfying condition (3), is investigated in [30]. It turns out that X satisfies most properties of Dirichlet sequences. In particular, μ has the same distribution as

$${\mu }^{*}=\sum _n{V}_n\alpha \left(Y_n\right),$$

where $\left(V_n\right)$ and $\left(Y_n\right)$ are independent sequences, $\left(Y_n\right)$ is i.i.d. with $Y_1\sim \nu$, and $\left(V_n\right)$ has the stick breaking distribution. Nevertheless, as shown in the next example, X can behave quite differently from a Dirichlet sequence.

Example 2

(Example 1 continued). Let $\mathcal{H}$ be a countable partition of S such that $H\in \mathcal{B}$ and $\nu \left(H\right)>0$ for all $H\in \mathcal{H}$. Define

$$\alpha \left(x\right)=\sum _{H\in \mathcal{H}}{1}_H\left(x\right)\nu (·\mid H)=\nu (·\mid H_x)\mathit{for}\mathit{all}x\in S$$

where $H_x$ is the only element of the partition $\mathcal{H}$, such that $x\in H$. Then, α is a regular conditional distribution for ν given $\sigma \left(\mathcal{H}\right)$ (i.e., condition (3) holds). If the $q_n$ are as in Example 1 with ${\theta }_n={\theta }_0$ for all n, one obtains

$$a_n=\frac{{\theta }_0\nu +{\sum }_{i=1}^n\nu (·\mid H_{X_i})}{n+{\theta }_0}.$$

Therefore,

$$a_n\ll \nu \mathit{for}\mathit{all}n\ge 0.$$

(11)

This is a striking difference with respect to Dirichlet sequences. For instance, if ν is non-atomic, condition (11) yields

$$P(X_i=X_{j}forsomei\ne j)=0$$

while $P(X_i=X_{j}forsomei\ne j)=1$ if X is a Dirichlet sequence. Note also that, for each $f\in F$,

$${\sigma }^2=\int \alpha \left(x\right){\left(f\right)}^2\mu \left(dx\right)-\mu {\left(f\right)}^2=\sum _{H\in \mathcal{H}}\nu {(f\mid H)}^2\mu \left(H\right)-\mu {\left(f\right)}^2$$

while ${\sigma }^2=\mu \left(f^2\right)-\mu {\left(f\right)}^2$ if X is a Dirichlet sequence. Other choices of α, which make X quite different from a Dirichlet sequence, are in [30].

Example 3.

A meaningful special case is ${\sum }_n(1-q_n)<\infty$. In this case,

$$\prod _{j=0}^{\infty }{q}_j:=\underset{n}{lim}\prod _{j=0}^n{q}_j$$

exists and is strictly positive. Hence, μ admits the representation

$$\mu =\nu \prod _{j=0}^{\infty }{q}_j+\sum _{i=1}^{\infty }\alpha \left(X_i\right)(1-q_{i-1})\prod _{j=i}^{\infty }{q}_j.$$

As an example, under conditions (2)–(3), Theorem 1 applies whenever

$$q_n{=exp\{-(c+n)}^{-2}\}forsomeconstantc>0.$$

With this choice of $q_n$, one obtains $(1-q_n){(c+n)}^2\to 1$, so that ${\sum }_n(1-q_n)<\infty$ and μ can be written as above. Note also that

$$\underset{n}{lim}\frac{d_n}{{(c+n)}^{3/2}}=\sqrt{3}.$$

Therefore, for fixed $f\in F$, the rate of convergence of $a_n\left(f\right)$ to $\mu \left(f\right)$ is $n^{-3/2}$ and not the usual $n^{-1/2}$.