Predictive Constructions Based on Measure-Valued Pólya Urn Processes

Abstract

Measure-valued Pólya urn processes (MVPP) are Markov chains with an additive structure that serve as an extension of the generalized k-color Pólya urn model towards a continuum of possible colors. We prove that, for any MVPP ${\left({\mu }_n\right)}_{n\ge 0}$ on a Polish space $\mathbb{X}$, the normalized sequence ${({\mu }_n/{\mu }_n\left(\mathbb{X}\right))}_{n\ge 0}$ agrees with the marginal predictive distributions of some random process ${\left(X_n\right)}_{n\ge 1}$. Moreover, ${\mu }_n={\mu }_{n-1}+R_{X_n}$, $n\ge 1$, where $x\mapsto R_x$ is a random transition kernel on $\mathbb{X}$; thus, if ${\mu }_{n-1}$ represents the contents of an urn, then $X_n$ denotes the color of the ball drawn with distribution ${\mu }_{n-1}/{\mu }_{n-1}\left(\mathbb{X}\right)$ and $R_{X_n}$—the subsequent reinforcement. In the case $R_{X_n}=W_n{\delta }_{X_n}$, for some non-negative random weights $W_1,W_2,\dots$, the process ${\left(X_n\right)}_{n\ge 1}$ is better understood as a randomly reinforced extension of Blackwell and MacQueen’s Pólya sequence. We study the asymptotic properties of the predictive distributions and the empirical frequencies of ${\left(X_n\right)}_{n\ge 1}$ under different assumptions on the weights. We also investigate a generalization of the above models via a randomization of the law of the reinforcement.

1. Introduction

Let ${\left(X_n\right)}_{n\ge 1}$ be a sequence of homogeneous random observations, taking values in a Polish space $\mathbb{X}$. The central assumption in the Bayesian approach to inductive reasoning is that ${\left(X_n\right)}_{n\ge 1}$ is exchangeable, that is, its law is invariant under finite permutations. Then, by de Finetti’s theorem, there exists a random probability measure $\tilde{P}$ on $\mathbb{X}$ such that, given $\tilde{P}$, the random variables $X_1,X_2,\dots$ are conditionally independent and identically distributed with marginal distribution $\tilde{P}$ (see [1], Section 3), denoted

$$X_n\mid \tilde{P}\stackrel{i.i.d.}{\sim }\tilde{P}.$$

(1)

Furthermore, $\tilde{P}$ is the almost sure (a.s.) weak limit of the predictive distributions and the empirical frequencies,

$$\mathbb{P}(X_{n+1}\in ·\mid X_1,\dots ,X_n)\stackrel{w}{⟶}\tilde{P}(·)\mathrm{a}.\mathrm{s}.\mathrm{and}\frac{1}{n}\sum _{i=1}^n{\delta }_{X_i}(·)\stackrel{w}{⟶}\tilde{P}(·)\mathrm{a}.\mathrm{s}.$$

(2)

The model (1) is completed by choosing a prior distribution for $\tilde{P}$. Inference consists in computing the conditional (posterior) distribution of $\tilde{P}$ given an observed sample $(X_1,\dots ,X_n)$, with most inferential conclusions depending on some average with respect to the posterior distribution; for example, under squared loss, for any measurable set $B\subseteq \mathbb{X}$, the best estimate of $\tilde{P}\left(B\right)$ is the posterior mean, $\mathbb{E}\left[\tilde{P}\left(B\right)\right|X_1,\cdots ,X_n]$. In addition, the posterior mean can be utilized for predictive inference since

$$\mathbb{P}(X_{n+1}\in B|X_1,\dots ,X_n)=\mathbb{E}\left[\tilde{P}\left(B\right)\right|X_1,\dots ,X_n].$$

(3)

A different modeling strategy uses the Ionescu–Tulcea theorem to define the law of the process from the sequence of predictive distributions, ${\left(\mathbb{P}(X_{n+1}\in ·|X_1,\dots ,X_n)\right)}_{n\ge 1}$. In that case, one can refer to Theorem 3.1 in [2] for necessary and sufficient conditions on ${\left(\mathbb{P}(X_{n+1}\in ·|X_1,\dots ,X_n)\right)}_{n\ge 1}$ to be consistent with exchangeability. The predictive approach to model building is deeply rooted in Bayesian statistics, where the parameter $\tilde{P}$ is assigned an auxiliary role and the focus is on observable “facts”, see [2,3,4,5,6]. Moreover, using the predictive distributions as primary objects allows one to make predictions instantly or helps ease computations. See [7] for a review on some well-known predictive constructions of priors for Bayesian inference.

In this work, we consider a class of predictive constructions based on measure-valued Pólya urn processes (MVPP). MVPPs have been introduced in the probabilistic literature [8,9] as an extension of k-color urn models, but their implications for (Bayesian) statistics have yet to be explored. A first aim of the paper is thus to show the potential use of MVPPs as predictive constructions in Bayesian inference. In fact, some popular models in Bayesian nonparametric inference can be framed in such a way, see Equation (8). A second aim of the paper is to suggest novel extensions of MVPPs that we believe can offer more flexibility in statistical applications.

MVPPs are essentially measure-valued Markov processes that have an additive structure, with the formal definition being postponed to Section 2.1 (Definition 1). Given an MVPP ${\left({\mu }_n\right)}_{n\ge 0}$, we consider a sequence of random observations that are characterized by $\mathbb{P}(X_1\in ·)={\mu }_0(·)/{\mu }_0\left(\mathbb{X}\right)$ and, for $n\ge 1$,

$$\mathbb{P}(X_{n+1}\in ·\mid X_1,{\mu }_1,\dots ,X_n,{\mu }_n)=\frac{{\mu }_n(·)}{{\mu }_n\left(\mathbb{X}\right)}.$$

(4)

The random measure ${\mu }_n$ is not necessarily measurable with respect to $(X_1,\dots ,X_n)$, so the predictive construction (4) is more flexible than models based solely on the predictive distributions of ${\left(X_n\right)}_{n\ge 1}$; for example, ${\left({\mu }_n\right)}_{n\ge 0}$ allows for the presence of latent variables or other sources of observable data (see also [10] for a covariate-based predictive construction). However, (4) can lead to an imbalanced design, which may break the symmetry imposed by exchangeability. Nevertheless, it is still possible that the sequence ${\left(X_n\right)}_{n\ge 1}$ satisfies (2) for some $\tilde{P}$, in which case Lemma 8.2 in [1] implies that ${\left(X_n\right)}_{n\ge 1}$ is asymptotically exchangeable with directing random measure $\tilde{P}$.

In Theorem 1, we show that, taking ${\left({\mu }_n\right)}_{n\ge 0}$ as primary, the sequence ${\left(X_n\right)}_{n\ge 1}$ in (4) can be chosen such that

$${\mu }_n={\mu }_{n-1}+R_{X_n},$$

(5)

where $x\mapsto R_x$ is a measurable map from $\mathbb{X}$ to the space of finite measures on $\mathbb{X}$. Models of the kind (4)–(5) are computationally efficient. Indeed, as new observations become available, predictions can be updated at a constant computational cost and with limited storage of information. If, in addition, ${\left(X_n\right)}_{n\ge 1}$ is asymptotically exchangeable, then (4)–(5) can provide a computationally simple approximation of an exchangeable scheme for Bayesian inference, along the lines in [11].

The recursive formula (5) allows us to interpret the dynamics of MVPPs in terms of an urn sampling scheme, as the name suggests. Let ${\mu }_0$ be a non-random finite measure on $\mathbb{X}$. Suppose we have an urn whose contents are described by ${\mu }_0$ in the sense that ${\mu }_0\left(B\right)$ denotes the total mass of balls with colors in $B\subseteq \mathbb{X}$. At time $n=1$, a ball is extracted at random from the urn, and we denote its color by $X_1$. The urn is then reinforced according to a replacement rule ${\left(R_x\right)}_{x\in \mathbb{X}}$, so that the updated composition becomes ${\mu }_1\equiv {\mu }_0+R_{X_1}$. At any time $n>1$, a ball of color $X_n$ is picked with probability distribution ${\mu }_{n-1}/{\mu }_{n-1}\left(\mathbb{X}\right)$, and the contents of the urn are subsequently reinforced by $R_{X_n}$. In the case the space of colors is finite, $\left|\mathbb{X}\right|=k$, the above procedure is better known as a generalized k-color Pólya urn [12].

We focus our analysis on MVPPs for which $R_x$ is concentrated on x; thus, after each draw, we reinforce only the color of the observed ball. More formally, we consider MVPPs that have a reinforcement measure of the kind $R_{X_n}=W_n{\delta }_{X_n}$, $n\ge 1$, where $W_n$ is some non-negative random variable. In that case, Equations (4) and (5) become

$$\mathbb{P}(X_{n+1}\in ·\mid X_1,W_1,\dots ,X_n,W_n)=\sum _{i=1}^n\frac{W_i}{{\mu }_0\left(\mathbb{X}\right)+{\sum }_{j=1}^n{W}_j}{\delta }_{X_i}(·)+\frac{{\mu }_0\left(\mathbb{X}\right)}{{\mu }_0\left(\mathbb{X}\right)+{\sum }_{j=1}^n{W}_i}{\mu }_0^{\prime }(·),$$

(6)

and

$${\mu }_n={\mu }_{n-1}+W_n{\delta }_{X_n}.$$

(7)

A notable example is Blackwell and MacQueen’s em Pólya sequence [13], which is a random process ${\left(X_n\right)}_{n\ge 1}$ characterized by $\mathbb{P}(X_1\in ·)=\nu (·)$ and, for $n\ge 1$,

$$\mathbb{P}(X_{n+1}\in ·\mid X_1,\cdots ,X_n)=\sum _{i=1}^n\frac{1}{\theta +n}{\delta }_{X_i}(·)+\frac{\theta }{\theta +n}\nu (·),$$

(8)

for some probability measure $\nu$ on $\mathbb{X}$ and a constant $\theta >0$. By [13], ${\left(X_n\right)}_{n\ge 1}$ is exchangeable and corresponds to the model (1) with Dirichlet process prior with parameters $(\theta ,\nu )$. It is easily seen that (8) is related to the MVPP ${\left({\mu }_n\right)}_{n\ge 0}$ given by ${\mu }_0=\theta \nu$ and, for $n\ge 1$,

$${\mu }_n={\mu }_{n-1}+{\delta }_{X_n}.$$

Therefore, we will call any MVPP a randomly reinforced Pólya process (RRPP) if it admits representation (6)–(7).

Existing studies on MVPPs look at models that have mostly a balanced design, i.e., $R_x\left(\mathbb{X}\right)=r$, $x\in \mathbb{X}$, and assume irreducibility-like conditions for ${\left(R_x\right)}_{x\in \mathbb{X}}$, see [8,9,14,15] and Remark 4 in [16]. In contrast, RRPPs require that $R_x\left({\left\{x\right\}}^c\right)=0$, and so are excluded from the analysis in those papers. In fact, this difference in reinforcement mechanisms mirrors the dichotomy within k-color urn models, where the replacement R is best described in terms of a matrix with random elements. There, the class of randomly reinforced urns [17] assumes an R with zero off-diagonal elements (i.e., we reinforce only the color of the observed ball), whereas the generalized Pólya urn models require the mean replacement matrix to be irreducible. Similarly to the k-color case, RRPPs need the use of different techniques, which yield completely different results than those in [8,9,14,15,16]. As an example, Theorem 1 in [16] and our Theorem 2 prove convergence of the kind (2), yet the limit probability measure in [16] is non-random.

The RRPP has been implicitly studied by [17,18,19,20,21,22,23], among others, with the focus being on the process ${\left(X_n\right)}_{n\ge 1}$. Those papers deal primarily with the k-color case (with the exception of [18,19,23]) and can be categorized on the basis of their assumptions on ${\left(W_n\right)}_{n\ge 1}$. For example, [18,19,21,22] assume that $W_n$ and $(X_1,W_1,\dots ,X_{n-1},W_{n-1},X_n)$ are independent, in which case the process ${\left(X_n\right)}_{n\ge 1}$ is conditionally identically distributed (c.i.d.) [21], that is, conditionally on current information, all future observations are identically distributed. It follows from [21] that c.i.d. processes preserve many of the properties of exchangeable sequences and, in particular, satisfy (2)–(3). In contrast, [17,20,23] assume that the reinforcement $W_n$ depends on the particular color $X_n$, and prove a version of (2) where $\tilde{P}$ is concentrated on the set of dominant colors for which the expected reinforcement is maximum. In this work, we reconsider the above models in the framework of RRPPs. For the c.i.d. case, we prove results whose analogues have already been established by [23] for the model with dominant colors. In particular, we extend the convergence in (2) to be in total variation and give a unified central limit theorem. We also examine the number of distinct values that are generated by the sequence ${\left(X_n\right)}_{n\ge 1}$.

In some applications, the definition of an MVPP can be too restrictive as it assumes that the probability law of the reinforcement R is known. However, we can envisage situations where the law is itself random, so we extend the definition of an MVPP by introducing a random parameter V. The resulting generalized measure-valued Pólya urn process (GMVPP) turns out to be a mixture of Markov processes and admits representation (4)–(5), conditional on the parameter V. When the reinforcement measure $R_x$ is concentrated on x, we call ${\left({\mu }_n\right)}_{n\ge 0}$ a generalized randomly reinforced Pólya process (GRRPP). We give a characterization of GRRPPs with exchangeable weights ${\left(W_n\right)}_{n\ge 1}$ and show that the process ${\left((X_n,W_n)\right)}_{n\ge 1}$ is partially conditionally identically distributed (partially c.i.d) [24], that is, conditionally on the past observations and the concurrent observation from the other sequence, the future observations are marginally identically distributed. We also extend some of the results for RRPPs to the generalized setting.

The paper is structured as follows. In Section 2.1, we recall the definition of a measure-valued Pólya urn process and prove representation (4)–(5) for a suitably selected sequence ${\left(X_n\right)}_{n\ge 1}$. Section 2.2 defines a particular subclass of MVPPs, called randomly reinforced Pólya processes (RRPP), which share with exchangeable Pólya sequences the property of reinforcing only the observed color. Section 3 is devoted to the study of the asymptotic properties of RRPPs. In Section 4, we give the definition of GMVPPs and GRRPPs, and obtain basic results.

2. Definitions and a Representation Result

Let $(\mathbb{X},d)$ be a complete separable metric space, endowed with its Borel $\sigma$-field $\mathcal{X}$. Denote by

$${\mathbb{M}}_F\left(\mathbb{X}\right),{\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right),{\mathbb{M}}_P\left(\mathbb{X}\right),$$

the collections of measures $\mu$ on $\mathbb{X}$ that are finite, finite and non-null, and probability measures, respectively. We regard ${\mathbb{M}}_F\left(\mathbb{X}\right)$, ${\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)$ and ${\mathbb{M}}_P\left(\mathbb{X}\right)$ as measurable spaces equipped with the $\sigma$-fields generated by $\mu \mapsto \mu \left(B\right)$, $B\in \mathcal{X}$. We further let

$${\mathbb{K}}_F(\mathbb{X},\mathbb{Y}),{\mathbb{K}}_P(\mathbb{X},\mathbb{Y}),$$

be the collections of transition kernels K from $\mathbb{X}$ to $\mathbb{Y}$ that are finite and probability kernels, respectively. Any non-null measure $\mu \in {\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)$ has a normalized version ${\mu }^{\prime }=\mu /\mu \left(\mathbb{X}\right)$. If $f:\mathbb{X}\to \mathbb{Y}$ is measurable, then $f^{♯}:{\mathbb{M}}_F\left(\mathbb{X}\right)\to {\mathbb{M}}_F\left(\mathbb{Y}\right)$ denotes the induced mapping of measures, $f^{♯}\left(\mu \right)(·)=\mu \left(f^{-1}(·)\right)$, $\mu \in {\mathbb{M}}_F\left(\mathbb{X}\right)$.

All random quantities are defined on a common probability space $(\mathsf{\Omega },\mathcal{H},\mathbb{P})$, which is assumed to be rich enough to support any required randomization. The symbol “⊥” will be used to denote independence between random objects, and “$\stackrel{d}{=}$” equality in distribution.

2.1. Measure-Valued Pólya urn Processes

Let $\mu \in {\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)$ describe the contents of an urn, as in Section 1. Once a ball is picked at random from $\mu$, the urn is reinforced according to a replacement rule, which is formally a kernel $R\in {\mathbb{K}}_F(\mathbb{X},\mathbb{X})$ that maps colors $x\mapsto R_x(·)$ to finite measures; thus,

$$\mu +R_x,$$

(9)

represents the updated urn composition if a ball of color x has been observed. In general, R is random and there exists a probability kernel $\mathcal{R}\in {\mathbb{K}}_P(\mathbb{X},{\mathbb{M}}_F\left(\mathbb{X}\right))$ such that $R_x\sim {\mathcal{R}}_x$, $x\in \mathbb{X}$. Then, the distribution of (9) prior to the sampling of the urn is given by

$${\widehat{R}}_{\mu }(·)={\int }_{\mathbb{X}}{\psi }_{\mu }^{♯}\left({\mathcal{R}}_x\right)(·){\mu }^{\prime }\left(dx\right),$$

(10)

where ${\psi }_{\mu }$ is the measurable map $\nu \mapsto \nu +\mu$ from ${\mathbb{M}}_F\left(\mathbb{X}\right)$ to ${\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)$. By Lemma 3.3 in [9], $\mu \mapsto {\widehat{R}}_{\mu }$ is a measurable map from ${\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)$ to ${\mathbb{M}}_P\left({\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)\right)$.

Definition 1

(Measure-Valued Pólya Urn Process [9]). A sequence ${\left({\mu }_n\right)}_{n\ge 0}$ of random finite measures on $\mathbb{X}$ is called a measure-valued Pólya urn process (MVPP) with parameters ${\mu }_0\in {\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)$ and $\mathcal{R}\in {\mathbb{K}}_P(\mathbb{X},{\mathbb{M}}_F\left(\mathbb{X}\right))$ if it is a Markov process with transition kernel $\widehat{R}$ given by (10). If, in particular, ${\mathcal{R}}_x={\delta }_{R_x}$ for some $R\in {\mathbb{K}}_F(\mathbb{X},\mathbb{X})$, then ${\left({\mu }_n\right)}_{n\ge 0}$ is said to be a deterministic MVPP.

The representation theorem below formalizes the idea of MVPP as an urn scheme.

Theorem 1.

A sequence ${\left({\mu }_n\right)}_{n\ge 0}$ of random finite measures is an MVPP with parameters $({\mu }_0,\mathcal{R})$ if and only if, for every $n\ge 1$,

$${\mu }_n={\mu }_{n-1}+R_{X_n}\mathrm{a}.\mathrm{s}.,$$

(11)

where ${\left(X_n\right)}_{n\ge 1}$ is a sequence of $\mathbb{X}$-valued random variables such that $X_1\sim {\mu }_0^{\prime }$ and, for $n\ge 2$,

$$\mathbb{P}(X_n\in ·\mid X_1,{\mu }_1,\dots ,X_{n-1},{\mu }_{n-1})={\mu }_{n-1}^{\prime }(·),$$

(12)

and R is a random finite transition kernel on $\mathbb{X}$ such that

$$\mathbb{P}(R_{X_n}\in ·\mid X_1,{\mu }_1,\dots ,X_{n-1},{\mu }_{n-1},X_n)={\mathcal{R}}_{X_n}(·).$$

(13)

Proof. 

If ${\left({\mu }_n\right)}_{n\ge 0}$ satisfies (11)–(13) for every $n\ge 1$, then it holds a.s. that

$$\mathbb{P}({\mu }_n\in ·\mid {\mu }_1,\dots ,{\mu }_{n-1})=\mathbb{E}[{\psi }_{{\mu }_{n-1}}^{♯}\left({\mathcal{R}}_{X_n}\right)(·)\mid {\mu }_1,\dots ,{\mu }_{n-1}]={\widehat{R}}_{{\mu }_{n-1}}(·).$$

Conversely, suppose ${\left({\mu }_n\right)}_{n\ge 0}$ is a MVPP with parameters $({\mu }_0,\mathcal{R})$. As $\mathcal{R}$ is a probability kernel from $\mathbb{X}$ to ${\mathbb{M}}_F\left(\mathbb{X}\right)$ and ${\mathbb{M}}_F\left(\mathbb{X}\right)$ is Polish, then there exists by Lemma 4.22 in [25] a measurable function $f(x,u)$ such that, for every $x\in \mathbb{X}$,

$$f(x,U)\sim {\mathcal{R}}_x,$$

whenever U is a uniform random variable on $[0,1]$, denoted $U\sim \mathrm{Unif}[0,1]$.

Let us prove by induction that there exists a sequence ${\left((X_n,U_n)\right)}_{n\ge 1}$ such that $X_1\sim {\mu }_0^{\prime }$, $U_1\perp X_1$, $U_1\sim \mathrm{Unif}[0,1]$, ${\mu }_1={\mu }_0+f(X_1,U_1)$ a.s., $({\mu }_2,{\mu }_3,\dots )\perp (X_1,U_1)\mid {\mu }_1$, and, for every $n\ge 2$,

(i)

$\mathbb{P}(X_n\in ·\mid X_1,U_1,{\mu }_1,\dots ,X_{n-1},U_{n-1},{\mu }_{n-1})={\mu }_{n-1}^{\prime }(·)$;

(ii)

$U_n\sim \mathrm{Unif}[0,1]$ and $U_n\perp (X_1,U_1,{\mu }_1,\dots ,X_{n-1},U_{n-1},{\mu }_{n-1},X_n)$;

(iii)

${\mu }_n={\mu }_{n-1}+f(X_n,U_n)$ a.s.;

(iv)

$({\mu }_{n+1},{\mu }_{n+2},\dots )\perp (X_n,U_n)\mid (X_1,U_1,{\mu }_1,\dots ,X_{n-1},U_{n-1},{\mu }_{n-1},{\mu }_n)$;

(v)

${\mu }_{n+1}\perp (X_1,U_1,\dots ,X_n,U_n)\mid ({\mu }_1,\dots ,{\mu }_n)$.

Then, Equations (11)–(13) follow from $\left(i\right)$–$\left(iii\right)$ with $R_{X_n}=f(X_n,U_n)$.

Regarding the base case, let ${\tilde{X}}_1$ and ${\tilde{U}}_1$ be independent random variables such that ${\tilde{U}}_1\sim \mathrm{Unif}[0,1]$ and ${\tilde{X}}_1\sim {\mu }_0^{\prime }$. It follows that, for any measurable set $B\subseteq {\mathbb{M}}_F\left(\mathbb{X}\right)$,

$$\mathbb{P}({\mu }_1\in B)={\widehat{R}}_{{\mu }_0}\left(B\right)=\mathbb{E}\left[{\psi }_{{\mu }_0}^{♯}\left({\mathcal{R}}_{{\tilde{X}}_1}\right)\left(B\right)\right]=\mathbb{P}\left(({\mu }_0+f({\tilde{X}}_1,{\tilde{U}}_1))\in B\right);$$

thus, ${\mu }_1\stackrel{d}{=}{\mu }_0+f({\tilde{X}}_1,{\tilde{U}}_1)$. By Theorem 8.17 in [25], there exist random variables $X_1$ and $U_1$ such that

$$({\mu }_1,X_1,U_1)\stackrel{d}{=}\left({\mu }_0+f({\tilde{X}}_1,{\tilde{U}}_1),{\tilde{X}}_1,{\tilde{U}}_1\right),$$

and $({\mu }_2,{\mu }_3,\dots )\perp (X_1,U_1)\mid {\mu }_1$. Then, in particular, $(X_1,U_1)\stackrel{d}{=}({\tilde{X}}_1,{\tilde{U}}_1)$ and $({\mu }_1,{\mu }_0+f(X_1,U_1))\stackrel{d}{=}({\mu }_0+f({\tilde{X}}_1,{\tilde{U}}_1),{\mu }_0+f({\tilde{X}}_1,{\tilde{U}}_1))$, so

$${\mu }_1={\mu }_0+f(X_1,U_1)\mathrm{a}.\mathrm{s}.$$

Regarding the induction step, assume that $\left(i\right)$–$\left(v\right)$ hold true until some $n>1$. Let ${\tilde{X}}_{n+1}$ and ${\tilde{U}}_{n+1}$ be such that ${\tilde{U}}_{n+1}\sim \mathrm{Unif}[0,1]$, ${\tilde{U}}_{n+1}\perp (X_1,U_1,{\mu }_1,\dots ,X_n,U_n,{\mu }_n,{\tilde{X}}_{n+1})$, and

$$\mathbb{P}({\tilde{X}}_{n+1}\in ·\mid X_1,U_1,{\mu }_1,\dots ,X_n,U_n,{\mu }_n)={\mu }_n^{\prime }(·).$$

It follows from $\left(v\right)$ that, for any measurable set $B\subseteq {\mathbb{M}}_F\left(\mathbb{X}\right)$,

$$\begin{array}{cc}\hfill \mathbb{P}({\mu }_{n+1}\in B|X_1,U_1,{\mu }_1,\dots ,& X_n,U_n,{\mu }_n)=\mathbb{E}\left[{\psi }_{{\mu }_n}^{♯}\left({\mathcal{R}}_{{\tilde{X}}_{n+1}}\right)\left(B\right)\right|X_1,U_1,{\mu }_1,\dots ,X_n,U_n,{\mu }_n]\hfill \\ \hfill & =\mathbb{P}\left(({\mu }_n+f({\tilde{X}}_{n+1},{\tilde{U}}_{n+1}))\in B|X_1,U_1,{\mu }_1,\dots ,X_n,U_n,{\mu }_n\right);\hfill \end{array}$$

thus, ${\mu }_{n+1}\stackrel{d}{=}{\mu }_n+f({\tilde{X}}_{n+1},{\tilde{U}}_{n+1})\mid X_1,U_1,{\mu }_1,\dots ,X_n,U_n,{\mu }_n$. By Theorem 8.17 in [25], there exist random variables $X_{n+1}$ and $U_{n+1}$ such that

$$\begin{array}{cc}\hfill ({\mu }_{n+1},X_1,U_1,{\mu }_1,& \dots ,X_n,U_n,{\mu }_n,X_{n+1},U_{n+1})\hfill \\ \hfill & \stackrel{d}{=}\left({\mu }_n+f({\tilde{X}}_{n+1},{\tilde{U}}_{n+1}),X_1,U_1,{\mu }_1,\dots ,X_n,U_n,{\mu }_n,{\tilde{X}}_{n+1},{\tilde{U}}_{n+1}\right),\hfill \end{array}$$

and $({\mu }_{n+2},{\mu }_{n+3},\dots )\perp (X_{n+1},U_{n+1})\mid (X_1,U_1,{\mu }_1\dots ,\dots ,X_n,U_n,{\mu }_n,{\mu }_{n+1})$. Then, in particular, $U_{n+1}\sim \mathrm{Unif}[0,1]$, $U_{n+1}\perp (X_1,U_1,{\mu }_1,\dots ,X_n,U_n,{\mu }_n,X_{n+1})$, and

$$\mathbb{P}(X_{n+1}\in ·\mid X_1,U_1,{\mu }_1,\dots ,X_n,U_n,{\mu }_n)={\mu }_n^{\prime }(·).$$

Moreover,

$$\left({\mu }_{n+1},{\mu }_n+f(X_{n+1},U_{n+1})\right)\stackrel{d}{=}\left({\mu }_n+f({\tilde{X}}_{n+1},{\tilde{U}}_{n+1}),{\mu }_n+f({\tilde{X}}_{n+1},{\tilde{U}}_{n+1})\right);$$

therefore,

$$\mathbb{P}\left({\mu }_{n+1}={\mu }_n+f(X_{n+1},U_{n+1})\right)=\mathbb{P}\left({\mu }_n+f({\tilde{X}}_{n+1},{\tilde{U}}_{n+1})={\mu }_n+f({\tilde{X}}_{n+1},{\tilde{U}}_{n+1})\right)=1.$$

By Theorem 8.12 in [25], statement $\left(v\right)$ with $n+1$ is equivalent to ${\mu }_{n+2}\perp (X_1,U_1)\mid ({\mu }_1,\dots ,{\mu }_{n+1})$ and ${\mu }_{n+2}\perp (X_{k+1},U_{k+1})\mid (X_1,U_1,\dots ,X_k,U_k,{\mu }_1,\dots ,{\mu }_{n+1})$, $k=1,\dots ,n$. The latter follows from the induction hypothesis since, by $\left(iv\right)$, we have $({\mu }_{k+2},\dots ,{\mu }_{n+2})\perp (X_{k+1},U_{k+1})\mid (X_1,U_1,\dots ,X_k,U_k,{\mu }_1,\dots ,{\mu }_{k+1})$ for every $k=1,\dots ,n$. □

The process ${\left(X_n\right)}_{n\ge 1}$ in Theorem 1 corresponds to the sequence of observed colors from the implied urn sampling scheme. Furthermore, the replacement rule takes the form $R_{X_n}=f(X_n,U_n)$, where f is some measurable function, $U_n\sim \mathrm{Unif}[0,1]$, and $U_n\perp (X_1,U_1,\dots ,X_{n-1},U_{n-1},X_n)$, from which it follows that

$${\mu }_n={\mu }_{n-1}+f(X_n,U_n),$$

(14)

and

$$\mathbb{P}(X_{n+1}\in ·\mid X_1,\dots ,X_n,{\left(U_m\right)}_{m\ge 1})=\frac{{\mu }_0(·)+{\sum }_{i=1}^{n}f(X_i,U_i)(·)}{{\mu }_0\left(\mathbb{X}\right)+{\sum }_{i=1}^{n}f(X_i,U_i)\left(\mathbb{X}\right)}.$$

(15)

Thus, the sequence ${\left(U_n\right)}_{n\ge 1}$ models the additional randomness in the reinforcement measure R. Janson [9] obtains a rather similar result; Theorem 1.3 in [9] states that any MVPP ${\left({\mu }_n\right)}_{n\ge 0}$ can be coupled with a deterministic MVPP ${\left({\overline{\mu }}_n\right)}_{n\ge 0}$ on $\mathbb{X}\times [0,1]$ in the sense that

$${\overline{\mu }}_n={\mu }_n\times \lambda ,$$

(16)

where $\lambda$ is the Lebesgue measure on $[0,1]$, and ${\mu }_n\times \lambda$ is the product measure on $\mathbb{X}\times [0,1]$. In our case, the MVPP defined by ${\overline{\mu }}_0={\mu }_0\times \lambda$ and, for $n\ge 1$,

$${\overline{\mu }}_n={\overline{\mu }}_{n-1}+f(X_n,U_n)\times \lambda ,$$

has a non-random replacement rule $R_{x,u}=f(x,u)\times \lambda$ and satisfies (16) on a set of probability one.

2.2. Randomly Reinforced Pólya Processes

It follows from (8) that any Pólya sequence generates a deterministic MVPP through

$${\mu }_n={\mu }_{n-1}+{\delta }_{X_n}.$$

Here, we consider a randomly reinforced extension of Pólya sequences in the form of an MVPP with replacement rule $R_x=W\left(x\right)·{\delta }_x$, $x\in \mathbb{X}$, where $W\left(x\right)$ is a non-negative random variable.

Definition 2

(Randomly Reinforced Pólya Process). We call an MVPP with parameters $({\mu }_0,\mathcal{R})$ a randomly reinforced Pólya process (RRPP) if there exists $\eta \in {\mathbb{K}}_P(\mathbb{X},{\mathbb{R}}_{+})$ such that ${\mathcal{R}}_x={\xi }_x^{♯}\left({\eta }_x\right)$, $x\in \mathbb{X}$, where ${\xi }_x:{\mathbb{R}}_{+}\to {\mathbb{M}}_F\left(\mathbb{X}\right)$ is the map $w\mapsto w{\delta }_x$.

Observe that, for RRPPs, the reinforcement measure $f(x,u)$ in (14)–(15) concentrates its mass on x; thus, we obtain the following variant of the representation result in Theorem 1.

Proposition 1.

Let ${\left({\mu }_n\right)}_{n\ge 0}$ be an RRPP with parameters $({\mu }_0,\eta )$. Then, there exist a measurable function $h:\mathbb{X}\times [0,1]\to {\mathbb{R}}_{+}$ and a sequence ${\left((X_n,U_n)\right)}_{n\ge 1}$ such that, using $W_n=h(X_n,U_n)$, we have for every $n\ge 1$ that

$${\mu }_n={\mu }_{n-1}+W_n{\delta }_{X_n}\mathrm{a}.\mathrm{s}.,$$

(17)

where $X_1\sim {\mu }_0^{\prime }$ and, for $n\ge 1$, $U_n\sim \mathrm{Unif}[0,1]$, $U_n\perp (X_1,U_1,\dots ,X_{n-1},U_{n-1},X_n)$, and

$$\mathbb{P}(X_{n+1}\in ·\mid X_1,W_1,\dots ,X_n,W_n)=\sum _{i=1}^n\frac{W_i}{{\mu }_0\left(\mathbb{X}\right)+{\sum }_{j=1}^n{W}_j}{\delta }_{X_i}(·)+\frac{{\mu }_0\left(\mathbb{X}\right)}{{\mu }_0\left(\mathbb{X}\right)+{\sum }_{j=1}^n{W}_j}{\mu }_0^{\prime }(·).$$

(18)

Moreover,

$$\mathbb{P}(W_n\in ·\mid X_1,W_1,\dots ,X_{n-1},W_{n-1},X_n)={\eta }_{X_n}(·).$$

(19)

It follows from (19) that $W\left(x\right)\equiv h(x,U)\sim {\eta }_x$, $x\in \mathbb{X}$, whenever $U\sim \mathrm{Unif}[0,1]$. Then, the random measure

$$R_x=W\left(x\right)·{\delta }_x$$

(20)

is such that $R_x\sim {\mathcal{R}}_x$, where ${\mathcal{R}}_x$ appears in Definition 2.

3. Asymptotic Properties of RRPP

In this section, we study the asymptotic properties of RRPPs through the sequence ${\left(X_n\right)}_{n\ge 1}$ in the representation (17). We show that the limit behavior of ${\left({\mu }_n\right)}_{n\ge 0}$ depends on the relationship between weights and observations. In particular, when $W\left(x\right)\equiv W$ in (20) is constant with respect to the color x, the process ${\left(X_n\right)}_{n\ge 1}$ is conditionally identically distributed (c.i.d.) and, for every $A\in \mathcal{X}$, the normalized sequence ${\left({\mu }_n^{\prime }\left(A\right)\right)}_{n\ge 0}$ is a bounded martingale. We consider the c.i.d. case in Section 3.3. In contrast, if some colors x have a higher expected reinforcement, then they tend to dominate the observation process and, as n grows to infinity, the probability measure ${\mu }_n^{\prime }$ concentrates its mass on the subset of dominant colors, see Theorem 2.

3.1. Preliminaries

Our focus is on the convergence of the normalized sequence ${\left({\mu }_n^{\prime }\right)}_{n\ge 0}$, which by Theorem 1 is a.s. equal to the predictive distributions (18). We also consider the sequence of empirical frequencies of ${\left(X_n\right)}_{n\ge 1}$, defined for $n\ge 1$ by

$${\widehat{\mu }}_n^{\prime }=\frac{1}{n}\sum _{i=1}^n{\delta }_{X_i}.$$

We obtain conditions under which the convergence in (2) extends to convergence in total variation, where the total variation distance between any two probability measures $\alpha ,\beta \in {\mathbb{M}}_P\left(\mathbb{X}\right)$ is given by

$$d_{TV}(\alpha ,\beta )=\underset{B\in \mathcal{X}}{sup}|\alpha \left(B\right)-\beta \left(B\right)|.$$

To state some of the results, we recall the definition of support of a probability measure $\gamma \in {\mathbb{M}}_P\left({\mathbb{R}}_{+}\right)$,

$$\mathrm{supp}\left(\gamma \right)=\{u\ge 0:\gamma ((u-ϵ,u+ϵ))>0,\forall ϵ>0\}.$$

Of particular interest is the conditional probability of observing a new color, given by

$${\theta }_n\equiv \mathbb{P}(X_{n+1}\notin \{X_1,\dots ,X_n\}\mid X_1,W_1,\dots ,X_n,W_n)=\frac{\theta }{\theta +{\sum }_{j=1}^n{W}_j}{\mu }_0^{\prime }\left({\{X_1,\dots ,X_n\}}^c\right),$$

for $n\ge 1$, where $\theta ={\mu }_0\left(\mathbb{X}\right)$. This would inform us on the number of distinct values in a sample $(X_1,\dots ,X_n)$ of size n,

$$L_n=max\{k\in \{1,\dots ,n\}:X_k\notin \{X_1,\dots ,X_{k-1}\}\},$$

since ${\theta }_n=\mathbb{P}(L_{n+1}=L_n+1|X_1,W_1,\dots ,X_n,W_n)$.

The following modes of convergence are used when we investigate the rate of convergence of the distance between ${\mu }_n^{\prime }$ and ${\widehat{\mu }}_n$.

Almost sure (a.s.) conditional convergence. Let $\mathcal{G}={\left({\mathcal{G}}_n\right)}_{n\ge 0}$ be a filtration and $\tilde{Q}\in {\mathbb{K}}_P(\mathsf{\Omega },\mathbb{X})$. A sequence ${\left(Y_n\right)}_{n\ge 1}$ is said to converge to $\tilde{Q}$ in the sense of a.s. conditional convergence w.r.t. $\mathcal{G}$ if the conditional distribution of $Y_n$, given ${\mathcal{G}}_n$, converges weakly on a set of probability one to $\tilde{Q}$, that is, as $n\to \infty$,

$$\mathbb{P}(Y_n\in ·\mid {\mathcal{G}}_n)\stackrel{w}{⟶}\tilde{Q}(·)\mathrm{a}.\mathrm{s}.$$

We refer to [22] for more details.

Stable convergence. Stable convergence is a strong form of convergence in distribution, albeit weaker than a.s. conditional convergence. A sequence ${\left(Y_n\right)}_{n\ge 1}$ is said to converge stably to $\tilde{Q}$ if

$$\mathbb{E}\left[Vf\left(Y_n\right)\right]⟶\mathbb{E}\left[V{\int }_{\mathbb{X}}f\left(x\right)\tilde{Q}\left(dx\right)\right],$$

for all continuous bounded functions f and any integrable random variable V. The main application of stable convergence is in central limit theorems that allow for mixing variables in the limit. See [26] for a complete reference on stable convergence.

In the sequel, the stable and a.s. conditional limits will be some Gaussian law, which we denote by $\mathcal{N}(\mu ,{\sigma }^2)$ for parameters $(\mu ,{\sigma }^2)$, where $\mathcal{N}(\mu ,0)={\delta }_{\mu }$.

3.2. RRPP with Dominant Colors

Using (20), let us define, for $x\in \mathbb{X}$,

$$w\left(x\right)=\mathbb{E}\left[W\left(x\right)\right]\mathrm{and}\overline{w}=\underset{x\in \mathbb{X}}{sup}w\left(x\right).$$

We further let

$$\mathcal{D}=\{x\in \mathbb{X}:w\left(x\right)=\overline{w}\},$$

be the set of dominant colors. The model (18) with $\mathcal{D}\subset \mathbb{X}$ has been studied by [23] under the assumption that $\overline{w}$ is strictly greater than the next largest value of $w(·)$ in the support of $w^{♯}\left({\mu }_0^{\prime }\right)$. Then, the probability of observing a non-dominant color, $x\in {\mathcal{D}}^c$, vanishes, and the predictive and the empirical distributions converge in total variation to a common random probability measure, which is concentrated on $\mathcal{D}$. For completeness reasons, we report here the main results from [23].

Theorem 2

([23], Theorem 3.3). For any RRPP ${\left({\mu }_n\right)}_{n\ge 0}$ that satisfies

$$\begin{array}{cc}\hfill & W\left(x\right)\le \beta <\infty ;\hfill \\ \hfill & \overline{w}\in \mathrm{supp}\left(w^{♯}\left({\mu }_0^{\prime }\right)\right);\hfill \\ \hfill & \overline{w}>{\overline{w}}^c\equiv sup\{u\ge 0:u\in \mathrm{supp}\left(w^{♯}\left({\mu }_0^{\prime }(·|{\mathcal{D}}^c)\right)\right\},\hfill \end{array}$$

(21)

there exists a random probability measure $\tilde{P}$ on $\mathbb{X}$ with $\tilde{P}\left(\mathcal{D}\right)=1$ a.s. such that

$$d_{TV}({\mu }_n^{\prime },\tilde{P})\stackrel{a.s.}{⟶}0and{d}_{TV}({\widehat{\mu }}_n^{\prime },\tilde{P})\stackrel{a.s.}{⟶}0.$$

Under conditions (21), Theorem 3.3 in [23] implies ${\sum }_{i=1}^n{W}_i/n\stackrel{a.s.}{⟶}\overline{w}$. If ${\mu }_0$ is further diffuse, then ${\sum }_{n=1}^{\infty }{\theta }_n=\infty$ a.s., and so $L_n\stackrel{a.s.}{⟶}\infty$ by Theorem 1 in [27]; thus, by Theorem 1 in [27], Proposition 3.4 in [23] shows that the actual growth rate is that of a Pólya sequence,

$$\frac{L_n}{logn}\stackrel{a.s.}{⟶}\frac{\theta }{\overline{w}}.$$

(22)

In addition to the uniform convergence in Theorem 2, the authors in [23] obtain set-wise rates of convergence. To state their result, we introduce, for any $A\in \mathcal{X}$,

$$q_A=\underset{n\to \infty }{lim}\mathbb{E}\left[W_{n+1}^2{\delta }_{X_{n+1}}\left(A\right)\right|X_1,W_1,\dots ,X_n,W_n],$$

which exists a.s. under the assumptions of Theorem 2.

Theorem 3

([23], Theorem 4.2). Let ${\left({\mu }_n\right)}_{n\ge 0}$ be an RRPP satisfying (21). Suppose $\overline{w}>2{\overline{w}}^c$. Define

$$V\left(A\right)=\frac{1}{{\overline{w}}^2}\left\{{\left(\tilde{P}\left(A^c\right)\right)}^2{q}_A+{\left(\tilde{P}\left(A\right)\right)}^2{q}_{A^c}\right\}andU\left(A\right)=V\left(A\right)-\tilde{P}\left(A\right)\tilde{P}\left(A^c\right).$$

Then,

$$\sqrt{n}\left({\mu }_n^{\prime }\left(A\right)-{\widehat{\mu }}_n^{\prime }\left(A\right)\right)\stackrel{stably}{⟶}\mathcal{N}(0,U\left(A\right)),$$

and

$$\sqrt{n}\left({\mu }_n^{\prime }\left(A\right)-\tilde{P}\left(A\right)\right)\stackrel{a.s.cond.}{⟶}\mathcal{N}(0,V\left(A\right))w.r.t.{\left({\mathcal{F}}_n^{X,W}\right)}_{n\ge 1},$$

where ${\mathcal{F}}_n^{X,W}=\sigma (X_1,W_1,\dots ,X_n,W_n)$, $n\ge 1$ is the filtration generated by ${\left((X_n,W_n)\right)}_{n\ge 1}$.

3.3. RRPP with Independent Weights

Let ${\left({\mu }_n\right)}_{n\ge 0}$ be an RRPP with reinforcement distribution ${\eta }_x\equiv \eta$ that does not depend on x. Using the notation of Section 3.2, we have

$$w\left(x\right)\equiv \overline{w},$$

(23)

and, thus, $\mathcal{D}=\mathbb{X}$. An equivalent formulation can be given in terms of the sequence of weights ${\left(W_n\right)}_{n\ge 1}$ in Proposition 1, whereby

$$W_n=h\left(U_n\right),$$

(24)

for some measurable function h, with $U_n\perp (X_1,U_1,\dots ,X_{n-1},U_{n-1},X_n)$ and $U_n\sim \mathrm{Unif}[0,1]$. Then, $W_n\stackrel{i.i.d.}{\sim }\eta$ and $W_n\perp (X_1,\dots ,X_n)$, which implies that $\mathbb{E}\left[W_1\right]=\overline{w}$.

The model (18) with weights (24) has been studied by [18,19,22], among others, where the authors obtain central limit theorems and study the growth rate of $L_n$ when $\overline{w}<\infty$. Their results rely on the fact that ${\left(X_n\right)}_{n\ge 1}$ is conditionally identically distributed (c.i.d.) with respect to the filtration generated by ${\left((X_n,W_n)\right)}_{n\ge 1}$. By [21], an $\mathbb{X}$-valued random sequence ${\left(Y_n\right)}_{n\ge 1}$ that is adapted to a filtration ${\left({\mathcal{F}}_n\right)}_{n\ge 1}$ is said to be c.i.d. with respect to ${\left({\mathcal{F}}_n\right)}_{n\ge 1}$ if and only if ${\left(Y_n\right)}_{n\ge 1}$ is identically distributed and, for every $n,k\ge 1$,

$$\mathbb{P}(Y_{n+k}\in ·\mid {\mathcal{F}}_n)=\mathbb{P}(Y_{n+1}\in ·\mid {\mathcal{F}}_n).$$

(25)

Proposition 2

([19], Lemma 6). For any RRPP ${\left({\mu }_n\right)}_{n\ge 0}$ with ${\eta }_x\equiv \eta$, the observation process ${\left(X_n\right)}_{n\ge 1}$ is c.i.d. with respect to the filtration generated by ${\left((X_n,W_n)\right)}_{n\ge 1}$.

C.i.d. processes preserve many of the properties of exchangeable sequences, see [21]. For example, if ${\left(Y_n\right)}_{n\ge 1}$ is c.i.d., then there exists a random probability measure such that (2)–(3) hold true with respect to the filtration used in the definition (25). It follows for the model in Proposition 2 that there exists $\tilde{P}\in {\mathbb{K}}_P(\mathsf{\Omega },\mathbb{X})$ such that, for every $A\in \mathcal{X}$,

$${\mu }_n^{\prime }\left(A\right)\stackrel{a.s.}{⟶}\tilde{P}\left(A\right).$$

In fact, by (25), the sequence ${\left({\mu }_n^{\prime }\left(A\right)\right)}_{n\ge 0}$ is a bounded martingale. On the other hand, (23) implies that $\mathcal{D}=\mathbb{X}$; therefore, any RRPP with ${\eta }_x\equiv \eta$ whose weights are bounded, $W_1\le \beta <\infty$, satisfies the assumptions of Theorem 2. In that case,

$$d_{TV}({\mu }_n^{\prime },\tilde{P})\stackrel{a.s.}{⟶}\tilde{P}.$$

It follows from Theorem 4.2 in [23] that the boundedness condition in (21) is needed to show that $\left(i\right)$ ${\sum }_{i=1}^n{W}_i/n\stackrel{a.s.}{⟶}\overline{w}$; and $\left(ii\right)$ ${\mu }_n^{\prime }$ converge set-wise to $\tilde{P}$, which is non-trivial in that setting. Here, $\left(i\right)$ is granted as ${\left(W_n\right)}_{n\ge 1}$ is i.i.d., and $\left(ii\right)$ has already been established; thus, we obtain the following result for RRPPs with independent weights.

Theorem 4.

For any RRPP ${\left({\mu }_n\right)}_{n\ge 0}$ with ${\eta }_x\equiv \eta$, there exists a random probability measure $\tilde{P}$ on $\mathbb{X}$ such that

$$d_{TV}({\mu }_n^{\prime },\tilde{P})\stackrel{a.s.}{⟶}0and{d}_{TV}({\widehat{\mu }}_n^{\prime },\tilde{P})\stackrel{a.s.}{⟶}0.$$

Proof. 

Let ${\left((X_n,W_n)\right)}_{n\ge 1}$ be the joint observation process associated to ${\left({\mu }_n\right)}_{n\ge 0}$ by Proposition 1. As ${\eta }_x\equiv \eta$, Equation (19) implies that $W_n\stackrel{i.i.d.}{\sim }\eta$; thus, by the strong law of large numbers,

$$\frac{1}{n}\sum _{i=1}^n{W}_i\stackrel{a.s.}{⟶}\overline{w}\le \infty .$$

(26)

Let us define, for $n\ge 1$,

$$P_n(·)=\mathbb{P}(X_{n+1}\in ·\mid {\mathcal{F}}_n^{X,W}),\mathrm{where}{\mathcal{F}}_n^{X,W}=\sigma (X_1,W_1,\dots ,X_n,W_n).$$

By Proposition 2, ${\left(X_n\right)}_{n\ge 1}$ is c.i.d. with respect to ${\left({\mathcal{F}}_n^{X,W}\right)}_{n\ge 1}$, so there exists by Lemmas 2.1 and 2.4 in [21] a random probability measure $\tilde{P}$ on $\mathbb{X}$ such that, for every $A\in \mathcal{X}$,

$$P_n\left(A\right)\stackrel{a.s.}{⟶}\tilde{P}\left(A\right).$$

(27)

Moreover, ${\int }_{\mathbb{X}}f\left(x\right)P_n\left(dx\right)=\mathbb{E}\left[{\int }_{\mathbb{X}}f\left(x\right)\tilde{P}\left(dx\right)\right|{\mathcal{F}}_n^{X,W}]$ a.s. for every bounded measurable $f:\mathbb{X}\to \mathbb{R}$. Fix $m\ge 1$. By a monotone class argument, we can show that, for every bounded measurable $f:{\mathbb{X}}^2\to \mathbb{R}$,

$${\int }_{\mathbb{X}}f(X_m,x)P_n\left(dx\right)=\mathbb{E}[{\int }_{\mathbb{X}}f(X_m,x)\tilde{P}\left(dx\right)\mid {\mathcal{F}}_n^{X,W}]\mathrm{a}.\mathrm{s}.,\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l} n>m;$$

thus, $P_n\left(\left\{X_m\right\}\right)=\mathbb{E}\left[\tilde{P}\left(\left\{X_m\right\}\right)\right|{\mathcal{F}}_n^{X,W}]$ a.s., and so ${\left(P_n\left(\left\{X_m\right\}\right)\right)}_{n>m}$ is a uniformly integrable martingale. It follows from martingale convergence that, as $n\to \infty$,

$$P_n\left(\left\{X_m\right\}\right)\stackrel{a.s.}{⟶}\tilde{P}\left(\left\{X_m\right\}\right).$$

(28)

Using (26)–(28), we can repeat the argument in the proof of Proposition 3.1 in [23] to show that $\left(i\right)$ $d_{TV}(P_n,\tilde{P})\stackrel{a.s.}{⟶}0$, and so $d_{TV}({\mu }_n^{\prime },\tilde{P})\stackrel{a.s.}{⟶}0$ by Proposition 1; and $\left(ii\right)$ $d_{TV}({\widehat{\mu }}_n^{\prime },\tilde{P})\stackrel{a.s.}{⟶}0$. □

Equation (26) implies that ${\theta }_n\stackrel{a.s.}{⟶}0$. If, in addition, $\overline{w}<\infty$, then ${\sum }_{n=1}^{\infty }{\theta }_n=\infty$ a.s. and $L_n\stackrel{a.s.}{⟶}\infty$. In fact, as long as $\overline{w}<\infty$, the sequence ${\left(L_n\right)}_{n\ge 1}$ grows at the same rate as (22).

Proposition 3

([18], Lemma 6). Let $\eta \in {\mathbb{M}}_P\left(\mathbb{X}\right)$ and ${\mu }_0$ be diffuse. If $\overline{w}<\infty$, then

$$\frac{L_n}{logn}\stackrel{a.s.}{⟶}\frac{\theta }{\overline{w}}.$$

If $\overline{w}=\infty$, then ${\theta }_n$ may approach zero fast enough that we stop seeing new observations as $n\to \infty$. For example, let us consider random reinforcement with a totally skewed stable distribution $S_{\alpha }(1,\sigma ,0)$ for $\alpha \in (0,2]$ and $\sigma >0$. If $\alpha <1$, then $\overline{w}=\infty$, and we show that $n^{1/\alpha }{\theta }_n$ is stochastically bounded, which implies that $L_n$ converges to a finite limit.

Proposition 4.

Let η be a $S_{\alpha }(1,\sigma ,0)$ distribution with stability parameter $\alpha <1$, and ${\mu }_0$ be diffuse. Then, ${\theta }_n=O_p\left(n^{-1/\alpha }\right)$ and

$$\underset{n\to \infty }{lim}{L}_n<\infty a.s.$$

Proof. 

From the properties of stable distributions, we obtain $n^{-1/\alpha }{\sum }_{i=1}^n{W}_i\stackrel{d}{=}{W}_1$ for every $n\ge 1$ and, as a consequence,

$${\theta }_n=n^{-1/\alpha }\frac{\theta }{n^{-1/\alpha }\theta +n^{-1/\alpha }{\sum }_{i=1}^n{W}_i}\stackrel{d}{=}{n}^{-1/\alpha }\frac{\theta }{n^{-1/\alpha }\theta +W_1}\le n^{-1/\alpha }\frac{\theta }{W_1}.$$

By Theorem 5.4.1 in [28], $\mathbb{E}[1/W_1]<\infty$, and so $1/W_1<\infty$ a.s. It follows for every $M>0$ that $\mathbb{P}(n^{1/\alpha }{\theta }_n>M)\le \mathbb{P}(\theta /W_1>M)$, which can be made arbitrarily small by taking M large enough. Regarding the second assertion, as $1/\alpha >1$, we have

$$\mathbb{E}\left[\underset{n\to \infty }{lim}{L}_n\right]=\underset{n\to \infty }{lim}\sum _{i=1}^n\mathbb{E}\left[11_{\{L_i=L_{i-1}+1\}}\right]=\sum _{n=1}^{\infty }\mathbb{E}\left[{\theta }_n\right]\le \sum _{n=1}^{\infty }\frac{\theta }{n^{1/\alpha }}\mathbb{E}[1/W_1]<\infty .$$

□

Proposition 4 can be extended for any fat tailed reinforcement distribution $\eta$ by means of a generalized central limit theorem (see, e.g., [28] (p. 62)).

The rate of convergence of (18) and ${\widehat{\mu }}_n^{\prime }$ has already been studied for the model with independent weights under different assumptions, see, e.g., [19] (p. 1363), Examples 4.2 and 4.5 in the technical report to [18], Corollary 4.1 in [22] for $\mathbb{X}=\{0,1\}$. In the next theorem, we combine ideas from [18,20] to give a fairly general result.

Theorem 5.

Let $\eta \in {\mathbb{M}}_P\left({\mathbb{R}}_{+}\right)$. If $\mathbb{E}\left[W_1^2\right]<\infty$, then

$$\sqrt{n}({\mu }_n^{\prime }\left(A\right)-{\widehat{\mu }}_n\left(A\right))\stackrel{stably}{⟶}\mathcal{N}(0,U\left(A\right)),whereU\left(A\right)=\frac{\mathrm{Var}\left(W_1\right)}{\mathbb{E}\left[W_1^2\right]}\tilde{P}\left(A\right)\tilde{P}\left(A^c\right).$$

(29)

If, in addition, $\mathbb{E}\left[W_1^4\right]<\infty$, then, with respect to the filtration generated by ${\left((X_n,W_n)\right)}_{n\ge 1}$,

$$\sqrt{n}({\mu }_n^{\prime }\left(A\right)-\tilde{P}\left(A\right))\stackrel{a.s.cond.}{⟶}\mathcal{N}(0,V\left(A\right)),whereV\left(A\right)=\frac{\mathbb{E}\left[W_1^2\right]}{{\overline{w}}^2}\tilde{P}\left(A\right)\tilde{P}\left(A^c\right).$$

(30)

Proof. 

Let us define, for $n\ge 1$,

$$P_n(·)=\mathbb{P}(X_{n+1}\in ·\mid X_1,W_1,\dots ,X_n,W_n).$$

The assertions in Theorem 5 have already been established by [18] when $W_1\ge \gamma >0$. In that case, Examples 4.2 and 4.5 in the technical report to [18] show that (29) is a consequence of the fact that

$$\mathbb{E}\left[\underset{1\le k\le n}{max}\left|Y_{n,k}\right|\right]⟶0\mathrm{and}\sum _{k=1}^n{Y}_{n,k}^2\stackrel{p}{⟶}U\left(A\right),$$

(31)

where $Y_{n,k}=\frac{1}{\sqrt{n}}\left\{{\delta }_{X_k}\left(A\right)-k{P}_k\left(A\right)+(k-1)P_{k-1}\left(A\right)\right\}$, and (30) follows from

$$\mathbb{E}\left[\underset{n\ge 1}{sup}\sqrt{n}|P_{n-1}\left(A\right)-P_n\left(A\right)|\right]<\infty \mathrm{and}n\sum _{k\ge n}{\left(P_{k-1}\left(A\right)-P_k\left(A\right)\right)}^2\stackrel{a.s.}{⟶}V\left(A\right).$$

Replicating the approach of Proposition 9 in [20], we avoid using the assumption $W_1\ge \gamma >0$ by conditioning on the sets $H_n=\{2{\sum }_{i=1}^n{W}_i\ge n\overline{w}\}$, $n\ge 1$. By (26), $11_{H_n}\stackrel{a.s.}{⟶}1$, so (29) follows from (31) with

$$Y_{n,k}=\frac{1}{\sqrt{n}}11_{H_{k-1}}\left\{{\delta }_{X_k}\left(A\right)-k{P}_k\left(A\right)+(k-1)P_{k-1}\left(A\right)\right\},$$

whereas (30) is, ultimately, a result of

$$\mathbb{E}\left[\underset{n\ge 1}{sup}\sqrt{n}·11_{H_n}|P_{n-1}\left(A\right)-P_n\left(A\right)|\right]<\infty \mathrm{and}n\sum _{k\ge n}{\left(P_{k-1}\left(A\right)-P_k\left(A\right)\right)}^2\stackrel{a.s.}{⟶}V\left(A\right).$$

□

4. Generalized Measure-Valued Pólya Urn Processes

The definition of an MVPP assumes that the law of the reinforcement $\mathcal{R}$ is fixed, yet, in some situations, $\mathcal{R}$ can itself be random (e.g., RRPP with exchangeable weights, see Section 4.1). To avoid measurability issues, we assume a parametric model for $\mathcal{R}$, with the parameter taking values in a Polish space $\mathbb{V}$.

Definition 3

(Generalized Measure-Valued Pólya Urn Process). Let V be a $\mathbb{V}$-valued random variable. A sequence ${\left({\mu }_n\right)}_{n\ge 0}$ of random finite measure on $\mathbb{X}$ is called a generalized measure-valued Pólya urn process (GMVPP) with uncertainty parameter V, initial state ${\mu }_0\in {\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)$ and replacement rule $\mathcal{R}\in {\mathbb{K}}_P(\mathbb{V}\times \mathbb{X},{\mathbb{M}}_F\left(\mathbb{X}\right))$ if ${\mu }_1\mid V\sim {\widehat{R}}_{{\mu }_0}^V$, and, for every $n\ge 2$,

$$\mathbb{P}({\mu }_n\in ·\mid V,{\mu }_1,\dots ,{\mu }_{n-1})={\widehat{R}}_{{\mu }_{n-1}}^V(·),$$

where $\widehat{R}$ is the transition probability kernel from $\mathbb{V}\times {\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)$ to ${\mathbb{M}}_F^{\ast }\left(\mathbb{X}\right)$ given by

$$(v,\mu )\mapsto {\widehat{R}}_{\mu }^v(·)={\int }_{\mathbb{X}}{\psi }_{\mu }^{♯}\left(\mathcal{R}(v,x)\right)(·){\mu }^{\prime }\left(dx\right),$$

and ${\psi }_{\mu }$ is the map $\nu \mapsto \nu +\mu$.

It follows from Definition 3 that any GMVPP is a mixture of Markov chains with initial state ${\mu }_0$ and transition kernel ${\widehat{R}}^V$. A separate modeling approach, which we do not examine here, defines a measure-valued Markov chain with transition kernel

$$\mu \mapsto {\int }_{\mathbb{X}}{\psi }_{\mu }^{♯}\left(\mathcal{R}(\mu ,x)\right)(·){\mu }^{\prime }\left(dx\right).$$

In fact, some of the predictive constructions in [11,29] can be framed in such a way.

Theorem 1 extends to GMVPPs, provided that we condition all quantities on the parameter V. As a consequence, there exists a measurable function f from $\mathbb{V}\times \mathbb{X}\times [0,1]$ to ${\mathbb{M}}_F\left(\mathbb{X}\right)$ and a random sequence ${\left((X_n,U_n)\right)}_{n\ge 1}$ such that

$${\mu }_n={\mu }_{n-1}+f(V,X_n,U_n)\mathrm{a}.\mathrm{s}.,$$

(32)

where $U_n\sim \mathrm{Unif}[0,1]$, $U_n\perp (V,X_1,U_1,\dots ,X_{n-1},U_{n-1},X_n)$, $X_1\mid (V,{\left(U_m\right)}_{m\ge 1})\sim {\mu }_0^{\prime }$, and, for $n\ge 1$,

$$\mathbb{P}(X_{n+1}\in ·\mid V,X_1,\dots ,X_n,{\left(U_m\right)}_{m\ge 1})=\frac{{\mu }_0(·)+{\sum }_{i=1}^{n}f(V,X_i,U_i)(·)}{{\mu }_0\left(\mathbb{X}\right)+{\sum }_{i=1}^{n}f(V,X_i,U_i)\left(\mathbb{X}\right)},$$

(33)

and

$$\mathbb{P}(f(V,X_n,U_n)\in ·\mid V,X_1,U_1,\dots ,X_{n-1},U_{n-1},X_n)=\mathcal{R}(V,X_n)(·).$$

(34)

The definition of a randomly reinforced Pólya process is similarly generalized to cover the case of a random reinforcement distribution $\eta$.

Definition 4

(Generalized Randomly Reinforced Pólya Process). We call a GMVPP with parameters $(V,{\mu }_0,\mathcal{R})$ a generalized randomly reinforced Pólya process (GRRPP) if there exists $\eta \in {\mathbb{K}}_P(\mathbb{V}\times \mathbb{X},{\mathbb{R}}_{+})$ such that $\mathcal{R}(v,x)={\xi }_x^{♯}\left(\eta (v,x)\right)$, where ${\xi }_x:{\mathbb{R}}_{+}\to {\mathbb{M}}_F\left(\mathbb{X}\right)$ is the map $w\mapsto w{\delta }_x$.

For GRRPPs, the function f in the representation (32)–(34) can be written as

$$f(v,x,u)=h(v,x,u)·{\delta }_x,$$

where h is a measurable function from $\mathbb{V}\times \mathbb{X}\times [0,1]$ to ${\mathbb{R}}_{+}$ such that $h(v,x,U)\sim \eta (v,x)$ for all $v\in \mathbb{V}$ and $x\in \mathbb{X}$, whenever $U\sim \mathrm{Unif}[0,1]$. Letting $W_n=h(V,X_n,U_n)$, we obtain

$${\mu }_n={\mu }_{n-1}+W_n{\delta }_{X_n}\mathrm{a}.\mathrm{s}.,$$

(35)

where

$$\mathbb{P}(X_{n+1}\in ·\mid V,X_1,\dots ,X_n,{\left(U_m\right)}_{m\ge 1})=\frac{{\mu }_0(·)+{\sum }_{i=1}^n{W}_i{\delta }_{X_i}(·)}{{\mu }_0\left(\mathbb{X}\right)+{\sum }_{i=1}^n{W}_i},$$

(36)

and

$$\mathbb{P}(W_n\in ·\mid V,X_1,U_1,\dots ,X_{n-1},U_{n-1},X_n)=\eta (V,X_n)(·).$$

(37)

The weights $W_n$ in (36) allow us to incorporate additional information about the observations ${\left(X_n\right)}_{n\ge 1}$. As an example, consider the problem of computer-based classification, where the output usually includes confidence scores, which reflect the software’s confidence that the classifications are correct. In analyzing the number and dimension of the types already discovered, or the probability of detecting a new type, a typical procedure would take into account only those classifications whose confidence scores are above a certain threshold. Alternatively, we could adopt a Bayesian perspective and weigh each classification according to its confidence score. Denoting by ${\left((X_n,W_n)\right)}_{n\ge 1}$ the sequence of classifications and confidence scores, we would model the distribution of the next classification by (36).

4.1. GRRPP with Exchangeable Weights

Let ${\left({\mu }_n\right)}_{n\ge 0}$ be a GRRPP with reinforcement distribution $\eta \left(v\right)$ that does not depend on x. Then,

$$W_n=h(V,U_n),$$

for some measurable function $h(v,u)$. The next result shows that the sequence ${\left(W_n\right)}_{n\ge 1}$ is exchangeable with directing random measure $\tilde{\eta }\equiv \eta \left(V\right)$. Moreover, ${\left({\mu }_n\right)}_{n\ge 0}$ is completely parameterized by $({\mu }_0,\tilde{\eta })$.

Theorem 6.

A sequence ${\left({\mu }_n\right)}_{n\ge 0}$ of random finite measures is a GRRPP with parameters $({\mu }_0,\tilde{\eta })$ for $\tilde{\eta }\in {\mathbb{K}}_P(\mathsf{\Omega },{\mathbb{R}}_{+})$ if and only if ${\mu }_0=\theta \nu$ and, for every $n\ge 1$,

$${\mu }_n={\mu }_{n-1}+W_n{\delta }_{X_n}a.s.,$$

where $\theta \in (0,\infty )$, $\nu \in {\mathbb{M}}_P\left(\mathbb{X}\right)$, ${\left(W_n\right)}_{n\ge 1}$ is an exchangeable process with directing random measure $\tilde{\eta }$, and ${\left(X_n\right)}_{n\ge 1}$ is a sequence of $\mathbb{X}$-valued random variables such that $X_1\mid {\left(W_k\right)}_{k\ge 1}\sim \nu$ and, for $n\ge 1$,

$$\mathbb{P}(X_{n+1}\in ·\mid X_1,\dots ,X_n,{\left(W_k\right)}_{k\ge 1})=\sum _{i=1}^n\frac{W_i}{\theta +{\sum }_{j=1}^n{W}_j}{\delta }_{X_i}(·)+\frac{\theta }{\theta +{\sum }_{j=1}^n{W}_j}\nu (·).$$

(38)

Proof. 

Let ${\left({\mu }_n\right)}_{n\ge 0}$ be a GRRPP with parameters $({\mu }_0,\tilde{\eta })$, and consider the representation (35)–(37). Put $\theta ={\mu }_0\left(\mathbb{X}\right)$ and $\nu ={\mu }_0^{\prime }$. It follows from (37) that

$$W_n\mid \tilde{\eta }\stackrel{i.i.d.}{\sim }\tilde{\eta };$$

thus, ${\left(W_n\right)}_{n\ge 1}$ is exchangeable. Moreover, $W_n=h(V,U_n)$, $n\ge 1$, so (38) follows from (36).

Conversely, suppose ${\mu }_n={\mu }_{n-1}+W_n{\delta }_{X_n}$, where the process ${\left((X_n,W_n)\right)}_{n\ge 1}$ is as described. It follows from (38) and Theorem 8.12 in [25] that

$${\left(W_k\right)}_{k\ge 1}\perp X_1\mathrm{and}{\left(W_{n+k}\right)}_{k\ge 1}\perp (X_1,\dots ,X_{n+1})\mid (W_1,\dots ,W_n),n\ge 1.$$

(39)

Since ${\left(W_n\right)}_{n\ge 1}$ is exchangeable with directing random measure $\tilde{\eta }$, we have

$$W_n\mid \tilde{\eta }\stackrel{i.i.d.}{\sim }\tilde{\eta }.$$

(40)

Furthermore, $\tilde{\eta }$ is measurable with respect to the tail $\sigma$-field of ${\left(W_n\right)}_{n\ge 1}$, so, by (39),

$$\tilde{\eta }\perp X_1\mathrm{and}\tilde{\eta }\perp (X_1,\dots ,X_{n+1})\mid (W_1,\dots ,W_n),n\ge 1.$$

(41)

Using (39)–(41), we can show that

$$W_1\perp X_1\mid \tilde{\eta }\mathrm{and}{W}_{n+1}\perp (X_1,W_1,\dots ,X_n,W_n,X_{n+1})\mid \tilde{\eta },n\ge 1.$$

Then, $\mathbb{P}({\mu }_1\in ·\mid \tilde{\eta })=\mathbb{P}({\mu }_0+W_1{\delta }_{X_1}\in ·\mid \tilde{\eta })={\int }_{\mathbb{X}}{\psi }_{{\mu }_0}^{♯}\left({\xi }_x^{♯}\left(\tilde{\eta }\right)\right)(·){\mu }_0^{\prime }\left(dx\right)$ and, for $n\ge 2$,

$$\begin{array}{cc}\hfill \mathbb{P}({\mu }_n\in ·\mid \tilde{\eta },{\mu }_1,\dots ,& {\mu }_{n-1})\hfill \\ \hfill & =\mathbb{E}\left[\mathbb{P}({\mu }_{n-1}+W_n{\delta }_{X_n}\in ·\mid \tilde{\eta },X_1,\dots ,W_{n-1},X_n)\right|\tilde{\eta },{\mu }_1,\dots ,{\mu }_{n-1}]\hfill \\ \hfill & =\mathbb{E}\left[\mathbb{E}[{\psi }_{{\mu }_{n-1}}^{♯}\left({\xi }_{X_n}^{♯}\left(\tilde{\eta }\right)\right)(·)\mid X_1,\dots ,X_{n-1},{\left(W_m\right)}_{m\ge 1}]\right|\tilde{\eta },{\mu }_1,\dots ,{\mu }_{n-1}]\hfill \\ \hfill & ={\int }_{\mathbb{X}}{\psi }_{{\mu }_{n-1}}^{♯}\left({\xi }_x^{♯}\left(\tilde{\eta }\right)\right)(·){\mu }_{n-1}^{\prime }\left(dx\right).\hfill \end{array}$$

□

It follows from the proof of Theorem 6 that $(X_1,W_1)\sim {\mu }_0^{\prime }\times \mathbb{E}\left[\tilde{\eta }\right]$ and, for $n\ge 1$,

$$\mathbb{P}\left((X_{n+1},W_{n+1})\in ·\mid X_1,W_1,\dots ,X_n,W_n\right) = \left({\mu }_n^{\prime }\times \mathbb{E}\left[\tilde{\eta }\right|W_1,\dots ,W_n]\right)(·).$$

(42)

As ${\mu }_n^{\prime }$ and $\mathbb{E}\left[\tilde{\eta }\right|W_1,\dots ,W_n]$ are both symmetric with respect to $((X_1,W_1),\dots ,(X_n,W_n))$, then (42) is a symmetric function of $((X_1,W_1),\dots ,(X_n,W_n))$. This is a necessary but not sufficient condition for ${\left((X_n,W_n)\right)}_{n\ge 1}$ to be exchangeable, see Proposition 3.2 and Example 3.1 in [2]. In Proposition 5, we show that ${\left((X_n,W_n)\right)}_{n\ge 1}$ is exchangeable if and only if either ${\mu }_0^{\prime }$ is degenerate or the weights are a.s. identical. On the other hand, for every $n,k\ge 1$, the sequence ${\left((X_n,W_n)\right)}_{n\ge 1}$ satisfies

$$\mathbb{P}(W_k\in ·\mid X_1)=\mathbb{P}(W_1\in ·\mid X_1),\mathbb{P}(X_k\in ·\mid W_1)=\mathbb{P}(X_1\in ·\mid W_1),$$

(43)

and

$$\begin{array}{cc}\hfill & \mathbb{P}(W_{n+k}\in ·|X_1,W_1,\cdots ,X_n,W_n,X_{n+1})=\mathbb{P}(W_{n+1}\in ·|X_1,W_1,\cdots ,X_n,W_n,X_{n+1}),\hfill \\ \hfill & \mathbb{P}(X_{n+k}\in ·|X_1,W_1,\cdots ,X_n,W_n,W_{n+1})=\mathbb{P}(X_{n+1}\in ·|X_1,W_1,\cdots ,X_n,W_n,W_{n+1}).\hfill \end{array}$$

(44)

By [24], Equations (43) and (44) are defining a process that is partially conditionally identity distributed (partially c.i.d.). Analogously to the c.i.d. case, partially c.i.d. processes preserve many of the properties of partially exchangeable sequences, see [24].

Proposition 5.

Under the conditions of Theorem 6, ${\left((X_n,W_n)\right)}_{n\ge 1}$ is partially c.i.d. Moreover, ${\left((X_n,W_n)\right)}_{n\ge 1}$ is exchangeable if and only if either ${\mu }_0^{\prime }$ is degenerate or $W_n=W_1$ a.s., $n\ge 1$. In that case, ${\left((X_n,W_n)\right)}_{n\ge 1}$ is partially exchangeable.

Proof. 

It follows that ${\left((X_n,W_n)\right)}_{n\ge 1}$ is partially c.i.d. if and only if $X_2\stackrel{d}{=}{X}_1\mid W_1$, $W_2\stackrel{d}{=}{W}_1\mid X_1$, and (44) is true for every $n\ge 1$ with $k=2$. By hypothesis, ${\left(W_n\right)}_{n\ge 1}$ is exchangeable and ${\left(W_n\right)}_{n\ge 1}\perp X_1$, so $W_2\stackrel{d}{=}{W}_1\mid X_1$. Moreover, applying (39) repeatedly, we obtain

$$\begin{array}{cc}\hfill \mathbb{P}(W_{n+2}\in ·\mid X_1& ,W_1,\dots ,X_n,W_n,X_{n+1})\hfill \\ \hfill & =\mathbb{E}\left[\mathbb{P}(W_{n+2}\in ·\mid X_1,\cdots ,W_{n+1},X_{n+2})|X_1,W_1,\cdots ,X_n,W_n,X_{n+1}\right]\hfill \\ \hfill & =\mathbb{E}\left[\mathbb{P}(W_{n+2}\in ·\mid W_1,\dots ,W_{n+1})|W_1,\dots ,W_n\right]\hfill \\ \hfill & =\mathbb{P}(W_{n+1}\in ·\mid W_1,\dots ,W_n)=\mathbb{P}(W_{n+1}\in ·\mid X_1,W_1,\dots ,X_n,W_n,X_{n+1}).\hfill \end{array}$$

On the other hand, by (38),

$$\begin{array}{cc}\hfill \mathbb{P}(X_{n+2}\in ·\mid X_1,W_1,\dots ,X_n,W_n,W_{n+1})& =\mathbb{E}\left[{\mu }_{n+1}^{\prime }(·)\mid X_1,W_1,\dots ,X_n,W_n,W_{n+1}\right]\hfill \\ \hfill & =\frac{{\mu }_n(·)+W_{n+1}·{\mu }_n^{\prime }(·)}{{\mu }_{n+1}\left(\mathbb{X}\right)}={\mu }_n^{\prime }(·)\hfill \\ \hfill & =\mathbb{P}(X_{n+1}\in ·\mid X_1,W_1,\dots ,X_n,W_n,W_{n+1}).\hfill \end{array}$$

Analogously, $\mathbb{P}(X_2\in ·\mid W_1)={\mu }_1(·)=\mathbb{P}(X_1\in ·\mid W_1)$, which completes the proof of the first part.

If ${\mu }_0^{\prime }$ is degenerate, then ${\left((X_n,W_n)\right)}_{n\ge 1}$ is trivially exchangeable. If $W_n=W_1$ a.s. instead, then one can show that ${\left((X_n,W_n)\right)}_{n\ge 1}$ satisfies condition $\left(b\right)$ of Proposition 3.2 in [2], which, together with the symmetry of (42), implies by Theorem 3.1 in [2] that ${\left((X_n,W_n)\right)}_{n\ge 1}$ is exchangeable.

Conversely, suppose that ${\left((X_n,W_n)\right)}_{n\ge 1}$ is exchangeable. As ${\left((X_n,W_n)\right)}_{n\ge 1}$ is partially c.i.d., the predictive distributions (42) converge to a product random measure [24]. It follows from de Finetti’s theorem that ${\left((X_n,W_n)\right)}_{n\ge 1}$ is partially exchangeable, so, in particular,

$$(X_1,W_1,X_2,W_2)\stackrel{d}{=}(X_1,W_2,X_2,W_1).$$

However, $W_2\perp X_2\mid (X_1,W_1)$ from (36), so $W_1\perp X_2\mid (X_1,W_2)$. Thus, for every bounded measurable function $\tilde{f}$, there exists a measurable function $g_{\tilde{f}}$ such that

$$\mathbb{E}\left[\tilde{f}\left(X_2\right)\right|X_1,W_1,W_2]=g_{\tilde{f}}(X_1,W_2)\mathrm{a}.\mathrm{s}.$$

Integrating $\tilde{f}\left(X_2\right)$ with respect to (38) and rearranging the terms, we obtain

$$W_1\left(\tilde{f}\left(X_1\right)-g_{\tilde{f}}(X_1,W_2)\right)=\theta \left(g_{\tilde{f}}(X_1,W_2)-\mathbb{E}\left[\tilde{f}\left(X_1\right)\right]\right)\mathrm{a}.\mathrm{s}.$$

Assume that ${\mu }_0^{\prime }$ is non-degenerate. Then, there is an $\tilde{f}$ such that $\mathbb{P}\left(\tilde{f}\left(X_1\right)=\mathbb{E}\left[\tilde{f}\left(X_1\right)\right]\right)=0$; e.g., take $\tilde{f}=11_B$ for some $B\in \mathcal{X}$ such that $0<\mathbb{P}(X_1\in B)<1$. It follows that

$$\begin{array}{cc}\hfill \mathbb{P}\left(\tilde{f}\left(X_1\right)=g_{\tilde{f}}(X_1,W_2)=0\right)& =\mathbb{P}\left(\tilde{f}\left(X_1\right)=\mathbb{E}\left[\tilde{f}\left(X_2\right)\right|X_1,W_1,W_2]\right)\hfill \\ \hfill & =\mathbb{P}\left(\tilde{f}\left(X_1\right)=\mathbb{E}\left[\tilde{f}\left(X_1\right)\right]\right)=0;\hfill \end{array}$$

therefore,

$$W_1=\frac{\theta \left(g_{\tilde{f}}(X_1,W_2)-\mathbb{E}\left[\tilde{f}\left(X_1\right)\right]\right)}{\tilde{f}\left(X_1\right)-g_{\tilde{f}}(X_1,W_2)}\mathrm{a}.\mathrm{s}.$$

In other words, there exists a measurable function $\tilde{h}$ such that $W_1=\tilde{h}(X_1,W_2)$ a.s., and so $W_2=\tilde{h}(X_1,W_1)$ a.s. by partial exchangeability. It follows from $X_1\perp (W_1,W_2)$ that, for every $A\in \mathcal{B}\left({\mathbb{R}}_{+}\right)$,

$$\mathbb{P}(W_2\in A|W_1)=\mathbb{P}(W_2\in A|X_1,W_1)=11_A\left(W_2\right)\mathrm{a}.\mathrm{s}.;$$

thus, $W_2=W_1$ a.s. and, from exchangeability, $W_n=W_1$ a.s., $n\ge 1$. □

4.2. Asymptotic Properties of GRRPP with Exchangeable Weights

It follows from (38) that the GRRPP with exchangeable weights is a mixture of RRPPs with independent weights, with the mixing distribution affecting only the sequence ${\left(W_n\right)}_{n\ge 1}$. Thus, we expect that the results in Section 3.3 carry over to this more general setting. In this section, we concentrate on the behavior of ${\theta }_n$ and the sequence ${\left(L_n\right)}_{n\ge 1}$.

Assume that $\mathbb{P}(W_1>0|\tilde{\eta })>0$. If $\mathbb{E}\left[W_1\right]<\infty$, then $0<\mathbb{E}\left[W_1\right|\tilde{\eta }]<\infty$ a.s., and, by the law of large numbers for exchangeable random variables (see [1], Section 2),

$$\frac{1}{n}\sum _{i=1}^n{W}_i\stackrel{a.s.}{⟶}\mathbb{E}\left[W_1\right|\tilde{\eta }]\in (0,+\infty ).$$

Then, if ${\mu }_0$ is diffuse, $n·{\theta }_n\stackrel{a.s.}{⟶}\theta /\mathbb{E}\left[W_1\right|\tilde{\eta }]$ and ${\sum }_{i=1}^n{\theta }_n=\infty$ a.s., so Theorem 1 in [27] implies

$$\frac{L_n}{logn}=\frac{L_n}{{\sum }_{k=1}^n{\theta }_k}\left(\frac{1}{logn}\sum _{k=1}^n\frac{1}{k}(k·{\theta }_k)\right)\stackrel{a.s.}{⟶}\frac{\theta }{\mathbb{E}\left[W_1\right|\tilde{\eta }]}.$$

If $\mathbb{E}\left[W_1\right]=\infty$, then $L_n$ may converge to a finite limit, as $n\to \infty$. For example, let us consider a strictly stable reinforcement distribution as in Proposition 4.

Proposition 6.

Let ${\left({\mu }_n\right)}_{n\ge 0}$ be a GRRPP with parameters $(V,{\mu }_0,\eta )$ such that V is a strictly positive random variable with $\mathbb{E}\left[V^{-1}\right]<\infty$, ${\mu }_0$ is diffuse, and $\eta \left(v\right)$, $v>0$ is a $S_{\alpha }(1,v,0)$ distribution with stability parameter $\alpha <1$. Then, ${\theta }_n=O_P\left(n^{-1/\alpha }\right)$ and

$$\underset{n\to \infty }{lim}{L}_n<\infty a.s.$$

Proof. 

It follows from how the weights in the representation (35) are chosen that we can take

$$W_n=V{F}^{-1}\left(U_n\right),$$

where $U_n\sim \mathrm{Unif}[0,1]$, $U_n\perp (V,X_1,U_1,\dots ,X_{n-1},U_{n-1},X_n)$, and $F^{-1}$ is the inverse of the $S_{\alpha }(1,1,0)$ distribution function. Then,

$${\theta }_n=\frac{\theta }{\theta +{\sum }_{i=1}^n{W}_i}\le n^{-1/\alpha }\frac{\theta }{V{n}^{-1/\alpha }{\sum }_{i=1}^n{F}^{-1}\left(U_i\right)}\stackrel{d}{=}{n}^{-1/\alpha }\frac{\theta }{VY},$$

for some $Y\sim S_{\alpha }(1,1,0)$ such that $Y\perp V$. It follows for every $M>0$ that $\mathbb{P}(n^{1/\alpha }{\theta }_n>M)\le \mathbb{P}(\theta /VY>M)$, which can be made arbitrarily small by taking M large enough. Regarding the second assertion, as $1/\alpha >1$ and $\mathbb{E}[\theta /(VY\left)\right]<\infty$ by Theorem 5.4.1 in [28], we have

$$\mathbb{E}\left[\underset{n\to \infty }{lim}{L}_n\right]=\underset{n\to \infty }{lim}\sum _{i=1}^n\mathbb{E}\left[11_{\{L_i=L_{i-1}+1\}}\right]=\sum _{n=1}^{\infty }\mathbb{E}\left[{\theta }_n\right]\le \sum _{n=1}^{\infty }\frac{\theta }{n^{1/\alpha }}\mathbb{E}[1/VY]<\infty .$$

□

Extensions of Proposition 6 can be obtained by exploiting the central limit theorems for exchangeable random variables, which are found in [30,31].

5. Discussion

In this paper, we study the extension of randomly reinforced urns [17] to an unbounded set of possible colors. The resulting measure-valued urn process provides a predictive characterization of the law of an asymptotically exchangeable sequence of random variables, which corresponds to the observation process of an implied urn sampling scheme. In fact, the model (6)–(7) fits into a line of recent research, which explores efficient predictive constructions for fast online prediction or approximately-Bayesian solutions, see [11,29,32] and references therein. To that end, one direction for future work is to generalize the functional relationship in (7) and/or, as one referee suggested, to consider finitely-additive measures, along the lines discussed in [33].

We investigate the asymptotic properties of the sequences of predictive distributions and empirical frequencies of the observation process, and prove their convergence in total variation distance to a common random limit. The rate of convergence of their difference is given set-wise; so, another possible direction for future research is to consider a stronger distance. As far as we know, the topic of merging of the predictive and empirical distributions is largely unexplored. Within the relevant literature, we mention the works of [4,34], where the authors study the rate of convergence of the Wasserstein or Prokhorov distances under exchangeability, and the papers by Berti et al. [21], Berti et al. [35], who consider the c.i.d. case and regard the difference between the predictive and empirical measures as a map in the space of real bounded functions.