Exchangeably Weighted Bootstraps of General Markov U-Process

Abstract

We explore an exchangeably weighted bootstrap of the general function-indexed empirical U-processes in the Markov setting, which is a natural higher-order generalization of the weighted bootstrap empirical processes. As a result of our findings, a considerable variety of bootstrap resampling strategies arise. This paper aims to provide theoretical justifications for the exchangeably weighted bootstrap consistency in the Markov setup. General structural conditions on the classes of functions (possibly unbounded) and the underlying distributions are required to establish our results. This paper provides the first general theoretical study of the bootstrap of the empirical U-processes in the Markov setting. Potential applications include the symmetry test, Kendall’s tau and the test of independence.

1. Introduction

U-statistics are a class of estimators, initially explored in association with unbiased estimators by [1] and officially introduced by [2], and are defined as follows: let ${\left\{X_i\right\}}_{i=1}^{\infty }$ be a sequence of random variables defined on a measurable space $(E,\mathcal{E})$, and let $h:E^m\to \mathbb{R}$ be a measurable function, the U-statistics of order m and kernel h based on the sequence $\left\{X_i\right\}$ are

$$U_n\left(h\right)={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{\left(i_1,\dots ,i_m\right)\in I_n^m}h\left(X_{i_1},\dots ,X_{i_m}\right),n\ge m,$$

where

$$I_n^m=\left\{\left(i_1,\dots ,i_m\right):i_j\in \mathbb{N},1\le i_j\le n,i_j\ne i_k\mathrm{if}j\ne k\right\}.$$

The empirical variance, Gini’s mean difference or Kendall’s rank correlation coefficient are common examples of U-estimators, while a classical test based on a U-statistic is Wilcoxon’s signed rank test for the hypothesis of the location at zero (see, e.g., [3], Example 12.4).The authors in [1,2,4] provided, amongst others, the first asymptotic results for the case in which the underlying random variables have independent and identical distributions. Extensive literature works have treated the theory of U-statistics, for instance, see [5,6,7,8], etc. Complex statistical issues are also amenable to being solved using U-processes. Examples include tests for goodness-of-fit, nonparametric regression and density estimation. U-processes are a set of U-statistics that are indexed by a family of kernels. U-processes might be viewed as infinite-dimensional variants of U-statistics with a single kernel function or as nonlinear stochastic extensions of empirical processes. Both thoughts have the following advantages: first, considering a large group of statistics rather than a single statistic is more statistically interesting. Second, we may use ideas from the theory of empirical processes to construct limit or approximation theorems for U-processes. Nevertheless, achieving results in U-processes is not easy. Extending U-statistics to U-processes necessitates a significant effort and distinct methodologies; generalizing empirical processes to U-processes is quite challenging, especially when U-processes are presented in the stationary setting. We highlight that the U-processes are used often in statistics, such as when higher order terms are a part of von Mises expansions. Particularly, the study of estimators (including function estimators) with various smoothness degrees involves U-statistics. For instance, Ref. [9] applied almost-sure uniform bounds for $\mathbb{P}$-canonical U-processes to analyze the product limit estimator for truncated data. Two new tests for normality based on U-processes were also presented in [10]. Inspired by [11,12,13], they developed other tests for normality that employed weighted $L_1$-distances between the standard normal density and local U-statistics based on standardized observations as test statistics. Estimating the mean of multivariate functions in the case of possibly heavy-tailed distributions was explored by [14]; they presented the median-of-means too, and both explorations were based on U-statistics. Moreover, other researchers emphasized the importance of U-processes; refs. [15,16,17] used them for testing qualitative features of functions in nonparametric statistics, ref. [18] represented the cross-validation for density estimation using U-statistics, in addition to [6,7,19], where the authors established limiting distributions of M-estimators. Since then, this discipline has made significant advancements, and the results have been broadly interpreted. Asymptotic behaviors were demonstrated under weak dependence assumptions, for example, in the works of [20,21,22] or more recently in [23] as well as more generally in [24,25]. However, in practice, explicit computation is not always possible due to the complexity of the U-processes’ limiting distributions or their functionals. We suggest a general bootstrap of the U-processes in the Markov setting to solve this issue, which is a challenging problem. The concept of the bootstrap, given by [26], in the case of independent and identically distributed (iid) random variables, is to resample from an original sample of observations of an unknown marginal distribution function $F\left(x\right)$, $X_1,\dots ,X_n$, a new i.i.d sample $X_1^{*},\dots ,X_n^{*}$ with the marginal distribution function $F_n\left(x\right)$, which represents the empirical distribution function constructed from the original sample. Moreover, it is commonly known that the bootstrap approach gives a better approximation to the statistic’s distribution, mainly when the sample size is small [27]. Bootstraps for U-statistics of independent observations were studied by [28,29,30,31]. However, the bootstrap technique is not the same for dependent variables because the dependence structure cannot be conserved in the new sample. For this reason, other blockwise bootstrap methods were introduced, aiming to keep the structure of dependence. Among those methods, we can cite the circular block bootstrap introduced by [32] and the nonoverlapping block bootstrap introduced by [33]. In [34], the authors proposed a bootstrap method related to the weakly dependent stationary observation, the stationary bootstrap. This latter can be seen as an expansion of the circular block bootstrap, where a random variable, such as a geometric random variable, can be used for the block length. It is important to note that Efron’s initial bootstrap formulation (see [26]) had a few flaws. To be more precise, certain observations might be sampled several times while others might not be at all. A more generalized version of the bootstrap, the weighted bootstrap, was developed to get around this issue and was also demonstrated to be computationally more appealing in some applications. This resampling strategy was initially described in [35] and thoroughly investigated by [28], who coined the name “weighted bootstrap”. For example, Bayesian bootstrap when the weighted vector

$$({\xi }_{n1},\dots ,{\xi }_{nn})=(M_{n1},\dots ,M_{nn}),$$

is equal to the vector of n spacings of $n-1$ ordered uniform $(0,1)$ random variables in distributions, that is, $(M_{n1},\dots ,M_{nn})$ follows a Dirichlet distribution of parameters $(n;1,\dots ,1).$ For more details, see [36]. This diversity of resampling approaches necessitates the use of a uniform approach, commonly known as general weighted resampling, which was first described by [37] and has since been developed by [38,39]. In [40], the authors investigated the almost-sure rate of convergence of strong approximation for the weighted bootstrap process by a sequence of Brownian bridge processes; refer to [41] for the multivariate setting and [42] for recent references. The concept of the generalized bootstrap, introduced by [37], was extended to the class of nondegenerate U-statistics of degree two and the corresponding Studentized U-statistics by [43]; refer to [44,45]. In [46], the author generalized this theory for a higher order. In his work, he developed a multiplier inequality of a U-process for i.i.d. random variables. We mention that the multiplier processes’ theory is directly and strongly related to the symmetrization inequalities investigated by [6,7].

This paper aims to investigate the exchangeable bootstrap for U-processes in the same way that [46] did but without the restriction of the independence setting. The previous reference focused on U-processes in an independent framework, whereas this paper considers U-processes in the dependent setting of Markov chains. We believe we are the first to present a successful consideration in this general context. We combine the techniques of the renewal bootstrap with the randomly weighted bootstrap in a nontrivial way. We mention a connection between moving-blocks bootstrap and its modification, matched-block bootstrap, at this point. Instead of artificially splitting a sample into fixed-size blocks and then resampling them, the latter seeks to match the blocks to create a smoother transition; for more information, see [47]. The main difficulties in proving Theorem 3 are due to the random size of the resampled blocks. This randomness generates a problem with the random stopping times, which cannot be removed by replacing a random stopping time with its expectation. In the present setting, the bootstrap random variables are generated by resampling from a random number of blocks. One can think that using the conditioning arguments can overcome the problem, but the answer is negative. Our proof uses some arguments from [46,47] by verifying bootstrap stochastic equicontinuity by comparing it to the original process in a similar way as in [48]. However, as we shall see later, integrating concepts from these papers is not enough to solve the problem. To deal with U-processes in the Markov framework, sophisticated mathematical derivations are necessary. We present the first complete theoretical justification of the bootstrap consistency. This justification requires the efficient use of large sample theoretical approaches established for U-empirical processes.

The rest of this paper is organized as follows. Section 2 is devoted to the introduction of the Markov framework, the U-process, the bootstrap weights and the definitions needed in our work. In Section 3, we recall the necessary ingredient for U-statistics and U-processes in the Markov setting. Furthermore, we provide some asymptotic results including the weak convergence of U-processes in Theorem 1. In Section 4, we derive the main results concerning the bootstrap of the U-processes. In Section 5, we collect some examples of weighted U-statistics. Some concluding remarks and possible future developments are relegated to Section 6. To prevent the interruption of the flow of the presentation, all proofs are gathered in Section 7. Appendix A contains a few pertinent technical findings and proofs.

2. Notation and Definitions

In what follows, we aim to properly define our settings. For this reason, we have collected the definitions and notation needed.

2.1. Markov Chain

Let $X={\left(X_n\right)}_{n\in \mathbb{N}}$ be an homogeneous $\psi$-irreducible Markov chain, that means that the chain has stationary transition probabilities, defined on a measurable space $(E,\mathcal{E})$, where $\mathcal{E}$ is a separable $\sigma$-algebra. Let $\pi (x,dy)$ be the transition probability and $\nu =\nu {\left(i\right)}_{i>0}$ the initial probability. Therefore, we denote by ${\mathbb{P}}_{\nu }$ or just $\mathbb{P}$ the probability measure for $\mathbb{P}=(\pi ,\nu )$. Likewise, ${\mathbb{E}}_{\nu }$ denotes the integration with respect to ${\mathbb{P}}_{\nu }$. In our framework, let ${\mathbb{P}}_x$ be a probability measure such that $X_0=x$, $X_0\in E$ and ${\mathbb{E}}_x(·)$ is the ${\mathbb{P}}_x$-expectation. We further assume that the Markov chain is Harris positive recurrent with an atom $\mathit{A}$.

Definition 1

(Harris recurrent). A Markov chain $X={\left(X_n\right)}_{n\in \mathbb{N}}$ is said to be Harris recurrent if there exists a σ-finite measure such that, for ψ a positive measure on a countable generated measurable space $(E,\mathcal{E})$, $\psi \left(E\right)>0$ and if for all $B\in E$ with $\psi \left(B\right)>0$

$$P_x\left({\cup }_{i=1}^{\infty }(X_i\in B)\right)=1foranyx\in E.$$

Recall that a chain is positive Harris recurrent and aperiodic if and only if it is ergodic ([49] Proposition 6.3), i.e., there exists a probability measure $\pi$, called the stationary distribution, such that, in total variation distance,

$$\underset{n\to +\infty }{lim}{∥{\mathbb{P}}^n(x,·)-\pi ∥}_{\mathrm{tv}}=0.$$

Definition 2

(Small sets). A set $S\in \mathcal{E}$ is said to be Ψ-small if there exists $\delta >0,$ a positive probability measure Ψ supported by S and an integer $m\in {\mathbb{N}}^{*}$, such that

$$\forall x\in S,B\in \mathcal{E},{\mathbb{P}}^m(x,B)\ge \delta \Psi \left(B\right).$$

(1)

Definition 3.

Let ${\left(X_n\right)}_{n\ge 1}$ be a Markov chain taking value in $(E,\mathcal{E})$. We say that ${\left(X_n\right)}_{n\ge 1}$ is positive recurrent if

1. 

${\left(X_n\right)}_{n\ge 1}$ is $(A,p,\nu ,m)$ recurrent (or Harris recurrent if $\mathcal{E}$ is countably generated), where $A\in \mathcal{E}$ is a set, $0<p<1$, m is an integer and ν is a probability measure.

2. 

$\underset{x\in A}{sup}{\mathbb{E}}_x\left({\mathcal{T}}_0\right)<\infty$, where ${\mathcal{T}}_0$ is the hitting time of A by the m-step chain, roughly speaking, ${\mathcal{T}}_0=min\{i\ge 1:X_{i,m}\in A\}$.

Definition 4.

A ψ-irreducible aperiodic chain X is called regenerative or atomic if there exists a measurable set $\mathbf{A}$ called an atom, in such a way that $\psi \left(\mathbf{A}\right)>0$ and for all $(x,y)\in {\mathbf{A}}^2$ we have $\mathbb{P}(x,·)=\mathbb{P}(y,·)$. Roughly speaking, an atom is a set on which the transition probabilities are the same. If a finite number of states or subsets are visited from the chain, then any state or any subset of the states is actually an atom.

Definition 5

(Aperiodicity). Assuming ψ-irreducibility, there exists $d^{\prime }\in {\mathbb{N}}^{*}$ and disjoints sets $D_1,\dots ,$$D_{d^{\prime }}$ (set $D_{d^{\prime }+1}=D_1$) positively weighted by ψ such that

$$\psi (E\setminus {\cup }_{1⩽i⩽d^{\prime }}{D}_i)=0$$

and

$$\forall x\in D_i,\mathbb{P}(x,D_{i+1})=1.$$

The period of the chain is the greatest common divisor d of such integers, it is said to be aperiodic if $d=1$.

Definition 6

(Irreducibility). The chain is ψ-irreducible if there exists a σ-finite measure ψ such that, for all set $B\in \mathcal{E}$, when $\psi \left(B\right)>0$, for any $x\in E$, there exists $n>0$ such that ${\mathbb{P}}^n(x,B)>0$.

One of the most important properties of Harris recurrent Markov chains is the existence of an invariant distribution which we is called $\mathit{\mu }$ (a limiting probability distribution, also called occupation measure). Furthermore, Harris recurrent Markov chains can always be embedded in a certain Markov chain on an extended sample space with a recurrent atom. The existence of a recurrent atom $\mathit{A}$ gives an immediate consequence for the construction of a regenerative extension of this chain. The time that the chain hits a given atom (recurrent state) is seen as the regenerative time. In [50,51], the authors give the construction of such a regenerative extension. The development of a regenerative extension makes the use of regenerative techniques possible in order to study this type of Markov chain. As we mentioned above, we assume in this work that the Harris recurrent chain is atomic, i.e., the set which is infinitely almost sure is well-defined and accessible, this set $\mathit{A}$ is called an atom. By definition, an atom $\mathit{A}$ is a set in $\mathcal{E}$, where $\mathit{\mu }\left(\mathit{A}\right)>0$, and for all $x,y\in \mathit{A}$, $\mathit{\pi }(x,·)=\mathit{\pi }(y,·)$. Let ${\mathbb{P}}_{\mathbf{A}}$ (respectively, ${\mathbb{E}}_{\mathbf{A}}$) be the probability measure on the underlying space such that $x\in \mathbf{A}$ (respectively, the ${\mathbb{P}}_{\mathbf{A}}$-expectation).

The conditions imposed on the Markov chain ensure that the defined atom $\mathit{A}$ (or the constructed one in the case of a nonatomic chain) is one recurrent class, and let us define the following terms.

2.1.1. Hitting Times

Define ${\mathcal{T}}_j:E\to \mathbb{N}\cup \{\infty \}$ by

$$\begin{array}{ccc}\hfill {\mathcal{T}}_0& :=& inf\{n\ge 0:X_n\in \mathit{A}\},\hfill \\ \hfill {\mathcal{T}}_j& :=& inf\{n\ge {\mathcal{T}}_{j-1}:X_n\in \mathit{A}\}.\hfill \end{array}$$

(2)

A well-known property of the hitting time is that for all $j\in \mathbb{N}$, ${\mathcal{T}}_j<\infty$, ${\mathbb{P}}_{\nu }-a.s$ ([52], chap. I14).

2.1.2. Renewal Times

Using the hitting times, we can define the renewal times as

$$\begin{array}{ccc}\hfill {\tau }_0& :=& {\mathcal{T}}_0+1,\hfill \\ \hfill \tau \left(j\right)& :=& {\mathcal{T}}_j-{\mathcal{T}}_{j-1}.\hfill \end{array}$$

(3)

Similar to the regenerative process, the sequence of renewal times ${\left\{\tau \left(j\right)\right\}}_{j=1}^{\infty }$ is i.i.d. and it is independent of the choice of the initial probability. All over this work, we set $\tau =\tau \left(1\right)$ and $\alpha ={\mathbb{E}}_{\mathit{A}}\left(\tau \right)$.

Definition 7

(Strong Markov property). Let ${\left(X_n\right)}_{n\ge 0}$ be a Markov chain, with T the stopping time of ${\left(X_n\right)}_{n\ge 0}$. Then, conditionally on $T<\infty$ and $X_T=i$, ${\left(X_{T+n}\right)}_{n\ge 0}$ is a sequence of a Markov chain and is independent of $X_0,\dots ,X_T$.

2.1.3. Regenerative Blocks

Let $l_n:=max\{j:{\sum }_{i=0}^j\tau \left(j\right)\le n\}$ be the number of visits to the atom $\mathit{A}$. Using the strong property of a Markov chain, it is possible to divide the given sample $(X_1,\dots ,X_n)$ into a sequence of blocks ${\left\{{\mathbf{B}}_j\right\}}_{j=0}^{l_n}$ such that:

$$\begin{array}{ccc}\hfill {\mathbf{B}}_0& =& \left\{X_1,\cdots ,X_{{\mathcal{T}}_0}\right\},\hfill \\ \hfill {\mathbf{B}}_j& =& \left\{X_{{\mathcal{T}}_{j-1}+1},\cdots ,X_{{\mathcal{T}}_j}\right\}in\mathbb{T}=\bigcup _{n=1}^{\infty }{E}^n,\mathrm{for}\mathrm{all}j=1,\cdots ,l_n,\hfill \\ \hfill {\mathbf{B}}_{l_n}^{\left(n\right)}& =& \left\{X_{{\mathcal{T}}_{l_n-1}+1},\cdots ,X_n\right\},\hfill \end{array}$$

(4)

where $l_n$ is the total number of blocks. The length of each block is denoted by

$$l\left({\mathbf{B}}_j\right):={\mathcal{T}}_j-{\mathcal{T}}_{j-1}.$$

2.2. Exchangeable Weights

In what follows, $\mathbf{\xi }$ represents a real-valued random variable, ${\mathbf{\xi }}_i$ are independent from $\left(X_i\right)$. For $1\le p<\infty$, we denote the p-norm by

$${\parallel \mathbf{\xi }\parallel }_p={\left(\mathbb{E}\left(\right|\mathbf{\xi }{|}^p)\right)}^{1/p}.$$

Definition 8

(Exchangeability). Let ${\mathbf{\xi }}_{n1},\dots ,{\mathbf{\xi }}_{nn}$ be a sequence of random variables with joint distribution ${\mathbb{P}}_{\mathbf{\xi }}$ and let $\mathsf{\Sigma }\left(n\right)$ be the group of all permutations acting on $\{1,\dots ,n\}$. We say that ${\mathbf{\xi }}_{n1},\dots ,{\mathbf{\xi }}_{nn}$ is exchangeable if, for all $\sigma \left(i\right)\in \mathsf{\Sigma }\left(n\right)$,

$${\mathbb{P}}_{\xi }({\mathbf{\xi }}_{n1},\dots ,{\mathbf{\xi }}_{nn})={\mathbb{P}}_{\xi }({\mathbf{\xi }}_{n\sigma \left(1\right)},\dots ,{\mathbf{\xi }}_{n\sigma \left(n\right)}).$$

Assuming the following:

(A1) 

$({\mathbf{\xi }}_1,\dots ,{\mathbf{\xi }}_n)$ are exchangeable non-negative, symmetric and for all n

$$\sum _{i=1}^n{\mathbf{\xi }}_i=n.$$

(A2) 

$\frac{1}{n}{\displaystyle \underset{1\le i\le n}{max}}{({\mathbf{\xi }}_i-1)}^2\to 0$ in ${\mathbb{P}}_{\xi }$-probability which is satisfied by the assumption of the moment

$$\underset{n}{sup}{\parallel {\xi }_1\parallel }_{2m,1}<\infty .$$

(A3) 

There exists $c>0$ such that, in ${\mathbb{P}}_{\xi }$-probability,

$$\frac{1}{n}\sum _{i=1}^n{({\mathbf{\xi }}_i-1)}^2\to c^2>0.$$

(A4) 

Assume

$$\underset{\lambda \to \infty }{lim}\underset{t\ge \lambda }{\overline{lim}}{t}^2{\mathbb{P}}_{\xi }({\mathbf{\xi }}_1\ge t)=0.$$

2.3. The U-Process Framework

Let ${\left(X_n\right)}_{n\in \mathbb{N}}$ be a sequence of random variables with values in a measurable space $(E,\mathcal{E})$. Let $h:E^m\to \mathbb{R}$ be a measurable function symmetric in its arguments. The U-statistic of order (or degree) m and kernel $h(·)$ is defined as:

$$U_n\left(h\right)={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}h(X_{i_1},\dots ,X_{i_m}),\mathrm{for}n\ge m.$$

(5)

Accordingly, a U-process is the collection $\{U_n\left(h\right):h\in \mathcal{F}\}$, where $\mathcal{F}$ is the class of kernels $h(·)$ of m variables. The decoupling inequality of U-statistics and U-processes plays a central role in the latest developments in the asymptotic theory. As a result, the decoupling inequality can give a relation between the quantities

$$\mathbb{E}\Phi \left(\left|\sum _{I_n^m}h\left(X_{i_1},\dots ,X_{i_m}\right)\right|\right)\mathrm{and}\mathbb{E}\Phi \left(\left|\sum _{I_n^m}h\left(X_{i_1}^1,\dots ,X_{i_m}^m\right)\right|\right),$$

where $\Phi (·)$ is a non-negative function and $\left\{X_i^k\right\},k=1,\dots ,m$ are independent copies of the original sequence $\left\{X_i\right\}$. One of the useful reasons for decoupling is randomization, which is frequently used in the study of the asymptotic theory of U-statistics, and was studied by [6,7]. The main idea of randomization is to compare the tail probabilities or moments of the original U-statistic or process, ${\sum }_{I_n^m}h(X_{i_1},\dots ,X_{i_m})$, with the tail probabilities or moments of the statistic

$$\sum _{I_n^m}{\epsilon }_{i_1}\dots {\epsilon }_{i_r}h(X_{i_1},\dots ,X_{i_m}),$$

where ${\epsilon }_i$ are independent Rademacher variables, independent from $X_i$, $1\le r\le m$ and the variables depend on the degree of degeneracy (centering) of the kernel $h(·)$.

Definition 9

([6]). A symmetric ${\mathbb{P}}^m$-integrable kernel $h:E^m\to \mathbb{R}$ is $\mathbb{P}$-degenerate of order $r-1$ if and only if

$$\int h(x_1,\dots ,x_m)d{\mathbb{P}}^{m-r+1}(x_r,\dots ,x_m)=\int hd{\mathbb{P}}^m$$

holds for any $x_1,\dots ,x_{r-1}\in E$, whereas

$$\int h(x_1,\dots ,x_m)d{\mathbb{P}}^{m-r}(x_{r+1},\dots ,x_m),$$

is not a constant function. If h is furthermore ${\mathbb{P}}^m$-centered, that is, ${\mathbb{P}}^{m}f=0$, we write $h\in L_2^{c,r}\left({\mathbb{P}}^m\right)$. For notational simplicity, we usually write $L_2^{c,m}\left({\mathbb{P}}^m\right)=L_2^{c,m}\left(\mathbb{P}\right)$.

Moreover, $h(·)$ is said to be canonical or completely degenerated if the integral with respect to one variable is equal to zero, i.e.,

$$\int h(x_1,\dots ,x_m)d\mathbb{P}\left(x_1\right)=0\mathrm{for}\mathrm{all}{x}_2,\dots ,x_m\in E.$$

The fact that the kernel is completely degenerate with the condition ${\mathbb{P}}^m{h}^2<\infty$ is used for the orthogonality of the different elements of the Hoeffding decomposition of the U-statistics.

Definition 10

(Covering number). The covering number $N_p(\epsilon ,Q,\mathcal{F})$ is defined as the minimal number of balls with radius ε that are needed to cover a class of functions $\mathcal{F}$ in the norm $L_p\left(Q\right)$, where Q is the measure on E with finite support.

We can associate some distances $e_{n,p}$ to the covering numbers, where

$$e_{n,p}=(U_n{\left(\right|f-g|}^p{\left)\right)}^{1/p}.$$

In this work, we use the two distances defined afterward

$$e_{n,2}(f,g)={\left(\frac{(n-m)!}{n!}\sum _{0\le i_1<\dots <i_m\le n}(f-g){(X_{i_1},\dots ,X_{i_m})}^2\right)}^{1/2}.$$

For decoupled statistics, we also associate covering numbers, well-known as $\tilde{N}(ϵ,\mathcal{F},{\tilde{e}}_{n,p})$ and a distance, which can be defined for $p=2$ as follows:

$${\tilde{e}}_{n,2}(f,g)=n^{1/2}\frac{(n-m)!}{n!}{\left[{\mathbb{E}}_{\epsilon }{\left(\sum _{0\le i_1<\dots <i_m\le n}{\epsilon }_{i_1}(f-g)(X_{i_1},\dots ,X_{i_m})\right)}^2\right]}^{1/2}.$$

Definition 11.

A class $\mathcal{F}$ of measurable functions $E\to \mathbb{R}$ is said to be of VC type (or Vapnik–Chervonenkis type) for an envelope F and admissible characteristic $(C,v)$ (positive constants) such that $C\ge {\left(3\sqrt{e}\right)}^v$ and $v\ge 1$, if for all probability measure $\mathbb{Q}$ on $(E,\mathcal{E})$ with ${0<\parallel F\parallel }_{L_2\left(Q\right)}<\infty$ and every $0<ϵ<1$,

$$N\left({ϵ\parallel F\parallel }_{L_2\left(\mathbb{Q}\right)},\mathcal{F},{\parallel ·\parallel }_{L_2\left(\mathbb{Q}\right)}\right)\le C{ϵ}^{-v}.$$

We assume that the class is countable to avoid measurability issues (the noncountable case may be handled similarly by using an outer probability and additional measurability assumptions, see [53]).

Definition 12

(Stochastic equicontinuity, ([54])). Let $\left\{Z_n\right\}$ be a sequence of stochastic processes. Call $\left\{Z_n\right\}$ stochastically equicontinuous at $t_0$ if for each $\delta >0$, there exists a neighborhood D of $t_0$ such that

$$\underset{n}{lim\; sup}\mathbb{P}\left\{\underset{D}{sup}|Z_n\left(t\right)-Z_n\left(t_0\right)|\right\}<\epsilon .$$

(6)

In the context of the U-process $\left\{U_n\right\}$, the stochastic equicontinuity at a function $g\in \mathcal{F}$ implies generally that $|U_n\left(h\right)-U_n\left(g\right)|$ should be uniformly small for all $h(·)$ close enough to $g(·)$, with high probability and for all n large enough.

2.4. Gaussian Chaos Process

Definition 13.

Let H denote a real separable Hilbert space with scalar product ${\langle ·,·\rangle }_H$. We say that a stochastic process $\mathbb{G}=\{{\mathbb{G}}_{\mathbb{P}}\left(h\right),h\in H\}$ defined in a complete probability space $(E,\mathcal{E},\mathbb{P})$ is an isonormal Gaussian process (or a Gaussian process on H) if ${\mathbb{G}}_{\mathbb{P}}$ is a centered Gaussian family of random variables such that $\mathbb{E}\left({\mathbb{G}}_{\mathbb{P}}\left(h\right){\mathbb{G}}_{\mathbb{P}}\left(g\right)\right)={\langle h,g\rangle }_H$ for all $h,g\in H$.

Define the mapping $h\to {\mathbb{G}}_{\mathbb{P}}\left(h\right)$. Under the assumption mentioned above, this map is linear and it provides a linear isometry of H onto a closed subspace $L_2(E,\mathcal{E},\mathbb{P})$ which contains a zero mean Gaussian random variables as its elements. Let ${\mathbb{K}}_{\mathbb{P}}$ be the isonormal Gaussian chaos process associated with ${\mathbb{G}}_{\mathbb{P}}$ determined by:

$${\mathbb{K}}_P\left(h_m^{\psi }\right)={(m!)}^{\frac{1}{2}}{\mathfrak{R}}_m\left({\mathbb{G}}_{\mathbb{P}}\left(\psi \right),\mathbb{E}{\psi }^2,0,\dots ,0\right),$$

where $h_m^{\psi }\left(x_1,\dots ,x_m\right)=\psi \left(x_1\right)\cdots \psi \left(x_m\right),\psi \in L_2\left(\mathbb{P}\right)$ and ${\mathfrak{R}}_m$ is a polynomial defined as a sum of monomials of degree m; ref. [6] give us a simple expression of this polynomial, extracted from Newton’s identity given by

$$\sum _{1\le i_1<·<i_m\le n}{t}_{i_1}\cdots t_{i_m}={\mathfrak{R}}_m\left(\sum _{i=1}^n{t}_i,\sum _{i=1}^n{t}_i^2,\dots ,\sum _{i=1}^n{t}_i^m\right).$$

Therefore,

$$\sum _{1\le i_1<·<i_m\le n}\psi \left(x_{i_1}\right)\cdots \psi \left(x_{i_m}\right)={\mathfrak{R}}_m\left(\sum _{i=1}^n\psi \left(x_i\right),\sum _{i=1}^n\psi {\left(x_i\right)}^2,\dots ,\sum _{i=1}^n\psi {\left(x_i\right)}^m\right).$$

Hence, by the continuous mapping theorem, we can see that CLT and LLN give:

$$\begin{array}{c}{\displaystyle \left({\left(\genfrac{}{}{0pt}{}{n}{m_1}\right)}^{\frac{1}{2}}{U}_n\left(h_{m_1}^{{\psi }_1}\right),\right.\left.\dots ,{\left(\genfrac{}{}{0pt}{}{n}{m_r}\right)}^{\frac{1}{2}}{U}_n\left(h_{m_r}^{{\psi }_r}\right)\right)}\hfill \\ \to \left({\left(m_1!\right)}^{\frac{1}{2}}{R}_{k_1}\left({\mathbb{G}}_{\mathbb{P}}\left({\psi }_1\right),\mathbb{E}{\psi }_1^2,0,\dots ,0\right)\dots ,{\left(m_r!\right)}^{\frac{1}{2}}{\mathfrak{R}}_{k_r}\left({\mathbb{G}}_{\mathbb{P}}\left({\psi }_r\right),\mathbb{E}{\psi }_r^2,0,\dots ,0\right)\right).\hfill \end{array}$$

Under the linearity of the kernel, we only need to show that:

$$\begin{array}{c}{\displaystyle \left\{{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{\frac{1}{2}}{U}_n\left(f\right):f\in \mathcal{F}\right\}}\hfill \\ \underset{d}{\to }\left\{{\mathbb{K}}_P\left(f_k\right)=m!{\mathfrak{R}}_m\left({\mathbb{G}}_{\mathbb{P}}\left(\psi \right),\mathbb{E}{\psi }^2,0,\dots ,0\right):f_k\in \mathcal{F}\right\}\mathrm{in}{\ell }^{\infty }\left(\mathcal{F}\right),\hfill \end{array}$$

to hold the weak convergence. The limit ${\mathbb{K}}_{\mathbb{P}}$ is useful in the case of degenerate U-statistics and it provides a convergence of all moments, which in turn plays a crucial role because it is due to the hypercontractivity, which makes the uniform integrability better. For a good explanation of ${\mathbb{K}}_{\mathbb{P}}$, readers are invited to see ([6] Chapter 4, Section 4.2).

2.5. Technical Assumptions

For our results, we need the following assumptions.

(C.1)

(Block-length assumption) For all $q\ge 1$ and $l\ge 1$,

$${\mathbb{E}}_{\nu }\left[{\mathbf{\tau }}^l\right]<\infty ,{\mathbb{E}}_A\left[{\mathbf{\tau }}^q\right]<\infty ;$$

(C.2)

(Nonregenerative blocks) For $l\ge 1$, we have

$${\mathbb{E}}_{\nu }\left[{\left(\sum _{i_1=1}^{{\mathcal{T}}_0}\sum _{i_2={\mathcal{T}}_0+1}^{{\mathcal{T}}_1}\sum _{i_3={\mathcal{T}}_1+1}^{{\mathcal{T}}_2}\dots \sum _{i_m={\mathcal{T}}_{m-1}+1}^{{\mathcal{T}}_m}\left|h(X_{i_1},\dots ,X_{i_m})\right|\right)}^l\right]<\infty$$

and

$${\mathbb{E}}_{\nu }\left[{\left(\sum _{i_1={\mathcal{T}}_0+1}^{{\mathcal{T}}_1}\sum _{i_2={\mathcal{T}}_1+1}^{{\mathcal{T}}_2}\dots \sum _{i_{m-1}={\mathcal{T}}_{m-1}+1}^{{\mathcal{T}}_m}\sum _{i_m=\mathcal{T}\left(l_n\right)+1}^n\left|h(X_{i_1},\dots ,X_{i_{m-1}},X_{i_m})\right|\right)}^l\right]<\infty$$

(C.3)

(Block-sum: moment assumptions) For $l\ge 1$, we have

$${\mathbb{E}}_{\nu }\left[{\left(\sum _{i_1={\mathcal{T}}_0+1}^{{\mathcal{T}}_1}\sum _{i_2={\mathcal{T}}_1+1}^{{\mathcal{T}}_2}\dots \sum _{i_m={\mathcal{T}}_{m-1}+1}^{{\mathcal{T}}_m}\left|h(X_{i_1},\dots ,X_{i_m})\right|\right)}^l\right]<\infty ,$$

and

$${\mathbb{E}}_A\left[{\left(\sum _{{\mathcal{T}}_0+1\le i_1\le \dots \le i_m\le {\mathcal{T}}_1}h(X_{i_1},\dots ,X_{i_m})\right)}^l\right]<\infty ;$$

(C.4)

For $l\ge 1$, we have

$$\begin{array}{ccc}& & {\mathbb{E}}_{\nu }\left[(\sum _{i_1={\mathcal{T}}_0+1}^{{\mathcal{T}}_1}\sum _{i_2={\mathcal{T}}_1+1}^{{\mathcal{T}}_2}\underset{\mathrm{u}\mathrm{times}}{\underbrace{\sum _{i_k={\mathcal{T}}_k+1}^{{\mathcal{T}}_{k+1}}\dots \sum _{i_k={\mathcal{T}}_k+1}^{{\mathcal{T}}_{k+1}}}}\sum _{i_{k+u}={\mathcal{T}}_{k+u}+1}^{{\mathcal{T}}_{k+u+1}}\dots \sum _{i_m={\mathcal{T}}_{m-1}+1}^{{\mathcal{T}}_m}\right.\hfill \\ & & \left.|h(X_{i_1},\underset{\mathrm{u}\mathrm{times}}{\underbrace{X_{i_k},\dots ,X_{i_k},}}{X}_{i_{k+u}},\dots ,X_{i_m})|{)}^l\right]<\infty ;\hfill \end{array}$$

(C.5)

(Nondegeneracy.) We suppose also that

$${\mathbb{E}}_A\left[{\left(\sum _{i={\mathcal{T}}_0+1}^{{\mathcal{T}}_1}{h}_1\left(X_i\right)\right)}^2\right]>0.$$

Remark 1

(Moment assumptions). In practice, we recall that block-moment assumptions for the split Markov chain can be generally checked by establishing drift conditions of Lyapunov’s type for the original chain; see Chapter 11 in [55,56], as well as All these moment conditions are discussed in detail in ([57], Chapters 11 and 17). There is a key condition in the proof of ergodic theorems in the Markovian context, which is the fact that ${\mathbb{E}}_{\mathbf{A}}\left({\tau }_0\right)<\infty$, for any $\mathbf{A}$ that is a set in $\mathcal{E}$, such that $\psi \left(\mathbf{A}\right)>0$. In fact, when there is a finite invariant measure and an atom $\mathbf{A}$, then this condition is readily found. We also refer to [58] for an explicit check of such conditions on several important examples and to §4.1.2 of [59] for sufficient conditions expressed in terms of a uniform return rate to small sets. Finally, as discussed in Chapter 8 of [60], similar conditions can be expressed in potential kernels. Observe that, in the positive recurrent case, the assumptions of (C.1) are not independent when $\nu =\mu$: from the basic renewal theory, one has ${\mathbb{P}}_{\mu }\left(\mathbf{\tau }=k\right)=$${\left({\mathbb{E}}_A\left[\mathbf{\tau }\right]\right)}^{-1}{\mathbb{P}}_A\left(\mathbf{\tau }\ge k\right)$ for all $k\ge 1$. Hence, conditions ${\mathbb{E}}_{\mu }\left[{\mathbf{\tau }}^l\right]<\infty$ and ${\mathbb{E}}_A\left[{\mathbf{\tau }}^{l+1}\right]<\infty$ are equivalent.

3. Preliminary Results

A significant issue was detected in recovering the estimation of our parameter of interest using the U-process. The given shape of this parameter is as follows:

$$\mathbf{\mu }\left(h\right)={\int }_{x_1\in E}\dots {\int }_{x_k\in E}h(x_1,\dots ,x_k)\mathbf{\mu }\left(d{x}_1\right)\dots \mathbf{\mu }\left(d{x}_k\right),$$

where $h:E^m\to \mathbb{R}$ is a kernel function. The estimation of this parameter should be possible using the U-statistics of the form:

$$U_n\left(h\right)={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}h(X_{i_1},\dots ,X_{i_m}),\mathrm{for}n\ge m,$$

(7)

As the parameter of interest is defined and based on Kac’s theorem for the occupation measure, $\mathbf{\mu }\left(h\right)$ in the regeneration setup can be written as follows:

$$\begin{array}{ccc}\hfill \mathbf{\mu }\left(h\right)& =& \frac{1}{{\left({\mathbb{E}}_A\left(\mathbf{\tau }\right)\right)}^m}{\mathbb{E}}_A\left(\sum _{i_1={\mathcal{T}}_0+1}^{{\mathcal{T}}_{\mathbf{1}}}\sum _{i_2={\mathcal{T}}_{\mathbf{1}}+1}^{{\mathcal{T}}_{\mathbf{2}}}\dots \sum _{i_m={\mathcal{T}}_{(\mathbf{m}-\mathbf{1})}+1}^{{\mathcal{T}}_m}h(X_{i_1},\dots ,X_{i_m})\right).\hfill \end{array}$$

(8)

In the Markovian context and since the variables are not independent, the approximation related to the i.i.d. blocks and the regenerative case is introduced below:

Definition 14

(Regenerative kernel). Let $h:E^m\to \mathbb{R}$ a kernel. We define the regenerative kernel ${\mathbf{\omega }}_h:{\mathbb{T}}^m\to \mathbb{R}$ as follows:

$${\mathbf{\omega }}_h((x_{11},\dots ,x_{1n_1}),\dots ,(x_{k1},\dots ,x_{k{n}_k}))=\sum _{i_1=1}^{n_1}\dots \sum _{i_k=1}^{n_k}h(x_{1i_1},\dots ,x_{k{i}_k}).$$

(9)

It is not necessary that the kernel ${\mathbf{\omega }}_h(·)$ be symmetric, as soon as $h(·)$. In fact, we can use the symmetrization of $S_m{\mathbf{\omega }}_h$ in the following way

$$\left(S_m{\mathbf{\omega }}_h\right)={(m!)}^{-1}\sum \sum _{\sigma \left(1\right)=1}^{n_1}\dots \sum _{\sigma \left(m\right)=1}^{n_k}h\left(x_{\sigma \left(1\right)},\dots ,x_{\sigma \left(m\right)}\right),$$

(10)

where the first sum is over all permutations $\sigma =\{i_1,\dots ,i_m\}$ of $\{1,\dots ,m\}$. Next, we consider the U-statistic formed by the regenerative data.

Definition 15

(Regenerative U-statistic). Let $h:E^m\to \mathbb{R}$ a kernel such that $\mathbf{\mu }\left(\right|h\left|\right)<\infty$ and set $\tilde{h}(·)=h(·)-\mathbf{\mu }\left(h\right).$ The regenerative U-statistic associated with the sequence of regenerative blocks ${\left\{{\mathbf{B}}_j\right\}}_{j=1}^L$, generated by the Markov chain is given by

$$R_{l_n}\left(h\right)={\left(\genfrac{}{}{0pt}{}{l_n-1}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n-1}^m}{\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}).$$

(11)

Hence, $R_{l_n}\left(h\right)$ is a standard U-statistic with mean zero.

Proposition 1.

Let us define

$$W_n\left(h\right)=U_n\left(h\right)-\mu \left(h\right)-\left(\genfrac{}{}{0pt}{}{l_n-1}{m}\right){\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}{R}_{l_n}\left(h\right).$$

(12)

Then, under conditions (C.1), (C.2), (C.3) and (C.4), we have the following stochastic convergences:

$$\begin{array}{ccc}\hfill W_n\left(h\right)& \to & 0,{\mathbb{P}}_{\nu }-a.s.\hfill \end{array}$$

Before stating the weak convergence in the next theorem, we define the corresponding U-processes related to the U-statistic $U_n$ and the regenerative U-statistic $R_L$, respectively:

$$\begin{array}{ccc}\hfill Z_n& :=& {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}\left[U_n-\mu \left(h\right)\right],\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill T_{l_n}& :=& {\left(\genfrac{}{}{0pt}{}{l_n}{m}\right)}^{1/2}\left[R_{l_n}-\mathbb{E}\left(R_{l_n}\right)\right].\hfill \end{array}$$

(14)

Theorem 1.

Let ${\left(X_n\right)}_n$ be a positive recurrent Harris Markov chain, with an accessible atom $\mathbf{A}$, $X_n$ satisfies the conditions (C.1) and (C.2) (moments assumptions), (C.3), (C.4), (C.5) and, for a fixed $\gamma >0$, $\mathbb{E}{\left(\mathbf{\tau }\right)}^{2+\gamma }<\infty$. Let $\mathcal{F}$ be a uniform bounded class of functions with an envelope H square-integrable such that:

$${\int }_0^{\infty }{(logN(\epsilon ,\mathcal{F},e_{n,2}))}^{m/2}d\epsilon <\infty .$$

Then, the process $Z_n$ converges weakly in probability under ${\mathbb{P}}_{\nu }$ to a Gaussian process ${\mathbb{G}}_{\mathbb{P}}$ indexed by $\mathcal{F}$ whose sample paths are bounded and uniformly continuous with respect to the metric $L_2\left(\mathbf{\mu }\right)$.

The Bootstrapped U-Processes

Trying to facilitate the bootstrap technique, we write the detailed steps of the regenerative block construction and the weighted bootstrap method in Algorithm 1:

Algorithm 1 Regenerative block and weighted bootstrap construction.

Identify the number of visits $l_n={\sum }_{i=0}^n{𝟙}_{X_i\in A}$ to the atom $\mathit{A}$. Divide the sample $X^{\left(n\right)}=(X_1,\dots ,X_n^{\left(n\right)})$ into $(l_n+1)$ regenerative blocks ${\mathbf{B}}_0,\dots ,{\mathbf{B}}_{l_n-1},{\mathbf{B}}_{l_n}^{\left(n\right)}\in \mathbb{T}$, each block $B_i$ with a length $l\left({\mathbf{B}}_i\right)\equiv {\tau }_i$. Drop the first and the last blocks if ${\tau }_{l_n}<n$ to avoid bias. Let $\mathbf{\xi }=({\xi }_{i,l_n},i=1,\dots ,n)$ be a triangular array of random variables. Define the weighted bootstrap empirical measure from the data: $${\mathbb{P}}_n^{*}=\frac{1}{l_n}\sum _{i=1}^n{\xi }_{i,l_n}{\delta }_{{\mathbf{B}}_i}.$$

In what follows, we denote by ${\mathbb{P}}^{*}$ and ${\mathbb{E}}^{*}$, respectively, the conditional probability and the conditional expectation given the sample $\{X_1,\dots ,X_n\}$. The same notation is used for the sample $\{{\mathbf{B}}_1,\dots ,{\mathbf{B}}_{L_n}\}$. Define the bootstrapped U-statistic as

$$\begin{array}{c}\hfill U_n^{*}\left(h\right)={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}{\xi }_{i_1,n}\dots {\xi }_{i_m,n}h(X_{i_1},\dots ,X_{i_m})\end{array}$$

(15)

and the regenerative bootstrapping

$$R_{l_n}^{*}\left(h\right)={\left(\genfrac{}{}{0pt}{}{l_n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}{\xi }_{i_1,l_n}\dots {\xi }_{i_m,l_n}{\omega }_h({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}).$$

(16)

and the U-processes are:

$$\begin{array}{ccc}\hfill Z_n^{*}& :=& {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}{\left[U_n^{*}\left(h\right)-U_n\left(h\right)\right]}_{h\in \mathcal{F}}\hfill \\ & =& {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_n^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)h(X_{i_1},\dots ,X_{i_m}).\hfill \end{array}$$

(17)

and

$$\begin{array}{ccc}\hfill T_{l_n}^{*}& :=& {\left(\genfrac{}{}{0pt}{}{l_n}{m}\right)}^{1/2}{\left[R_{l_n}^{*}\left(h\right)-\mathbb{E}\left(R_{l_n}^{*}\left(h\right)\right)\right]}_{h\in \mathcal{F}}\hfill \\ & =& {\left(\genfrac{}{}{0pt}{}{l_n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_h({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}).\hfill \end{array}$$

(18)

Given ${\Delta }_n$, a real-valued function, defined on the product probability space, we say that ${\Delta }_n$ is of an order $o_{{\mathbb{P}}_{\xi }}^o\left(1\right)$ in ${\mathbb{P}}_{\nu }^o$-probability if for any $\epsilon ,\delta >0$

$${\mathbb{P}}_{\nu }^o\left\{{\mathbb{P}}_{\xi \mid X}^o\left(\left|{\Delta }_n\right|>\epsilon \right)>\delta \right\}⟶0\mathrm{as}n\to \infty$$

and that ${\Delta }_n$ is of an order $O_{{\mathbb{P}}_{\xi }}^o\left(1\right)$ in ${\mathbb{P}}_{\nu }^o$-probability if for any $\delta >0$, there exists a $0<M<\infty$ such that

$${\mathbb{P}}_{\nu }^o\left\{{\mathbb{P}}_{\xi \mid X}^o\left(\left|{\Delta }_n\right|\ge M\right)>\delta \right\}⟶0\mathrm{as}n\to \infty$$

We must comment here that the bootstrap works in probability if

$$d_{BL}(T_{l_n}^{*},T_{l_n})\to 0\mathrm{in}\mathrm{probability},$$

where

$$d_{BL}(T_{l_n}^{*},T_{l_n})=\underset{g\in BL\left(l^{\infty }\left(\mathcal{F}\right)\right)}{sup}\left|\mathbb{E}g{\left(T_{l_n}^{*}\right)}^{\circ }-\mathbb{E}g\left(T_{l_n}\right)\right|,$$

and

$$BL\left(l^{\infty }\left(\mathcal{F}\right)\right):=\left\{g:l^{\infty }\left(\mathcal{F}\right)\to \mathbb{R},|g\left(x\right)-g\left(y\right)|\le {\parallel x-y\parallel }_{\mathcal{F}},{\parallel g\parallel }_{\infty }\le 1\right\},$$

and $g{\left(T_{l_n}^{*}\right)}^{\circ }$ is the measurable envelope of $g\left(T_{l_n}^{*}\right)$. In addition, for any measurable random elements, $Y_n$ and Y, the convergence in law of $Y_n$ to Y is in the sense of Hoffman–Jorgensen, which is defined as

$$\mathbb{E}g{\left(Y_n\right)}^{\circ }\to \mathbb{E}g\left(Y\right),$$

for g bounded and continuous. This weak convergence is metrizable by Theorem A1 in Appendix A.

Proposition 2.

Suppose that the bootstrap weights $\left({\xi }_1,\dots ,{\xi }_n\right)$ satisfy Assumptions (A1)–(A4). Let

$$W_n^{*}\left(h\right):=U_n^{*}\left(h\right)-\left(\genfrac{}{}{0pt}{}{l_n-1}{m}\right){\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}{R}_{l_n}^{*}\left(h\right).$$

(19)

Then, we have

$$\begin{array}{ccc}\hfill W_n^{*}\left(h\right)& \to & 0,{\mathbb{P}}_{\nu }\times {\mathbb{P}}_{\xi }-a.s.\hfill \end{array}$$

The proof of Proposition 2 is postponed until Section 7.

Now, in the following lemma, there are some instrumental results needed later.

Lemma 1.

Let ${\left(X_n\right)}_n$ be a Markov chain defined in Section 2.1. Define $p:=\mathbb{P}(X_0\in \mathit{A})={\alpha }^{-1}$. Then, for any initial probability ν, we have:

(i) 

For some $\eta >0$ and $C>0$:

$$\left|\frac{{\mathbb{E}}_{\nu }\left(l_n\right)}{np}-1\right|\le \frac{C}{n}and\sqrt{n}\left(\frac{l_n}{np}-1\right)\to N(0,{\eta }^2).$$

(20)

(ii) 

$\frac{n*}{n}\to 1$ in ${\mathbb{P}}_{\nu }\times {\mathbb{P}}_{\xi }$-probability.

(iii) 

Let $X_i$ be a sequence of random variables. If

$$T_n=\frac{1}{n}\sum _{i=1}^n{X}_i\to Ca.s.,$$

then for any integer $t_n$ valued sequence of random variables,

$$\frac{1}{t_n}\sum _{i=1}^{t_n}{X}_i\to Cin{\mathbb{P}}_{\nu }-probability.$$

The proof of Lemma 1 is postponed until Section 7.

4. Weighted Bootstrap Weak Convergence

In this section, we extend some existing results concerning the multiplier U-process to prove the bootstrap uniform weak convergence. Most of these results can be found in [46], generalizing the empirical process work of [38] in the i.i.d. setting. The weak convergence is proved for degenerate U-processes, as we mentioned before, and under the weighted regenerative bootstrap schemes described in Algorithm 1. Before stating the weak convergence theorem, we recall the following important results. The next theorem, proved in [46], is a sharp multiplier inequality, which is essential in the study of the multiplier U-process. These results are based on the decoupling symmetrized U-process, a basic framework of U-statistics. In [47], the author solved these problems for the empirical process settings in the Markov setting (multinomial bootstrap), which we generalize to the U-process by considering more general weights, i.e., the exchangeable weighted bootstrap.

Theorem 2

([46]). Let $({\xi }_1,\dots ,{\xi }_n)$ be a random vector independent of $(Y_1,\dots ,Y_n)$. Then, there exists some measurable function ${\psi }_n:{\mathbb{R}}_{\ge 0}^m\to {\mathbb{R}}_{\ge 0}$ such that the expected supremum of the decoupled (Here “decoupled” refers to the fact that $\left\{Y_i^{\left(k\right)}\right\},k\in \mathbb{N}$ are independent copies of $\left\{Y_i\right\}$, and $\left\{{ϵ}_i^{\left(k\right)}\right\},k\in \mathbb{N}$ are independent copies of the Rademacher sequence $\left\{{ϵ}_i\right\}$.) U-processes

$$\begin{array}{c}\hfill \mathbb{E}{∥\sum _{\begin{array}{c}1\le i_k\le {\ell }_k,1\le k\le m\end{array}}{ϵ}_{i_1}^{\left(1\right)}\cdots {ϵ}_{i_m}^{\left(m\right)}f\left(Y_{i_1}^{\left(1\right)},\dots ,Y_{i_m}^{\left(m\right)}\right)∥}_{\mathcal{F}}\le {\psi }_n({\ell }_1,\dots ,{\ell }_m),\end{array}$$

for all $1\le {\ell }_1,\dots ,{\ell }_m\le n$, consequently,

$$\begin{array}{c}{\displaystyle \mathbb{E}{∥\sum _{1\le i_1,\dots ,i_m\le l_{n-1}}{\xi }_{i_1}\cdots {\xi }_{i_m}f\left(Y_{i_1}^{\left(1\right)},\dots ,Y_{i_m}^{\left(m\right)}\right)∥}_{\mathcal{F}}}\hfill \\ \le K{\int }_{{\mathbb{R}}_{\ge 0}^m}\mathbb{E}{\psi }_n\left(\sum _{i=1}^{l_{n-1}}{\mathbb{1}}_{|{\xi }_i|>t_1},\dots ,\sum _{i=1}^{l_{n-1}}{\mathbb{1}}_{|{\xi }_i|>t_m}\right)d{t}_1\cdots d{t}_m.\hfill \end{array}$$

Furthermore, if there exists a concave and nondecreasing function ${\overline{\psi }}_n:\mathbb{R}\to \mathbb{R}$ such that ${\psi }_n({\ell }_1,\dots ,{\ell }_m)={\overline{\psi }}_n\left({\prod }_{k=1}^m{\ell }_k\right)$, then

$$\begin{array}{c}{\displaystyle \mathbb{E}{∥\sum _{1\le i_1,\dots ,i_m\le n}{\xi }_{i_1}\cdots {\xi }_{i_m}f\left(Y_{i_1}^{\left(1\right)},\dots ,Y_{i_m}^{\left(m\right)}\right)∥}_{\mathcal{F}}}\hfill \\ \le K{\int }_{{\mathbb{R}}_{\ge 0}^m}{\overline{\psi }}_n\left(\sum _{1\le i_1,\dots ,i_m\le n}\prod _{k=1}^{m}P{\left(|{\xi }_{i_k}|>t_k\right)}^{1/m}\right)d{t}_1\cdots d{t}_m.\hfill \end{array}$$

Here, $K>0$ is a constant depending on m only and can be taken as $K=2^{2m}{\prod }_{k=2}^m(k^k-1)$ for $m\ge 2$.

Lemma 2

([46]). Let $\{{\mathcal{F}}_{({\ell }_1,\dots ,{\ell }_m),n}:1\le {\ell }_1,\dots ,{\ell }_m\le n,n\in \mathbb{N}\}$ be function classes such that ${\mathcal{F}}_{({\ell }_1,\dots ,{\ell }_m),n}\supset {\mathcal{F}}_{(n,\dots ,n),n}$ for all $1\le {\ell }_1,\dots ,{\ell }_m\le n$. Suppose that the ${\xi }_i$’s have the same marginal distributions with ${\parallel {\xi }_1\parallel }_{2m,1}<\infty$. Suppose that there exists some bounded measurable function $a:{\mathbb{R}}_{\ge 0}^{m\left(n\right)}\to {\mathbb{R}}_{\ge 0}$ with $a({\ell }_1,\dots ,{\ell }_m)\to 0$ as ${\ell }_1\wedge \dots \wedge {\ell }_m\to \infty$, such that the expected supremum of the decoupled U-processes satisfies

$$\begin{array}{cc}\hfill & \mathbb{E}{∥\sum _{\begin{array}{c}1\le i_k\le {\ell }_k,1\le k\le m\end{array}}{ϵ}_{i_1}^{\left(1\right)}\cdots {ϵ}_{i_m}^{\left(m\right)}f\left(Y_{i_1}^{\left(1\right)},\dots ,Y_{i_m}^m\right)∥}_{{\mathcal{F}}_{({\ell }_1,\dots ,{\ell }_m),n}}\hfill \\ \hfill & \le a({\ell }_1,\dots ,{\ell }_m){\left(\prod _{k=1}^m{\ell }_k\right)}^{1/2},\hfill \end{array}$$

for all $1\le {\ell }_1,\dots ,{\ell }_m\le n$. Then,

$$\begin{array}{c}\hfill n^{-m/2}\mathbb{E}{∥\sum _{1\le i_1,\dots ,i_m\le n}{\xi }_{i_1}\cdots {\xi }_{i_{m\left(n\right)}}f\left(Y_{i_1}^{\left(1\right)},\dots ,Y_{i_m}^{\left(m\right)}\right)∥}_{{\mathcal{F}}_{(n,\dots ,n),n}}\to 0,n\to \infty .\end{array}$$

The main result of this paper is represented in the following theorem. It is worth noting here that it is not easy to prove the stochastic equicontinuity in the present setting as explained in the introduction.

Definition 16

(Permissible classes of function). Let $(E,\mathcal{E},\mathbb{P})$ be a measurable space ($\mathcal{E}$ a Borel σ-field on E). Let $\mathcal{F}$ be a class of functions indexed by a parameter x that belongs to a set E. $\mathcal{F}$ is called permissible if it can be indexed by a E such that:

• There exists a function $g(x,f)=f\left(x\right)$ defined from $S\times \mathcal{F}$ to $\mathbb{R}$ in such a way that this function is $\mathcal{L}\otimes \mathcal{B}\left(\mathcal{F}\right)$ measurable function, where $\mathcal{B}\left(\mathcal{F}\right)$ is the Borel σ-algebra generated by the metric on $\mathcal{F}$.
• E is a Suslin measurable space whose mean E is an analytic subset of a compact metric space E from which it inherits its metric and Borel σ-field.

Theorem 3.

Suppose Assumptions (A1) to (A4), and Conditions (C.1)–(C.5) hold. Let $\mathcal{F}\subset L_2^{c,m}\left(P\right)$ be permissible and admit a $P^m$-square integrable envelope F such that

$$\begin{array}{c}\hfill {\int }_0^1{\left(\underset{Q}{sup}log\mathcal{N}\left(ϵ{\parallel F\parallel }_{L_2\left(Q\right)},\mathcal{F},L_2\left(Q\right)\right)\right)}^{m/2}dϵ<\infty ,\end{array}$$

where the supremum is taken over all discrete probability measures. Then,

$$\begin{array}{c}\hfill \underset{\psi \in \mathrm{BL}}{sup}\left|{\mathbb{E}}_{\xi }\psi \left(Z_n^{*}\left(h\right)\right)-\mathbb{E}\psi (c·{\mathbb{K}}_{\mathbb{P}})\right|{\to }_{{\mathbb{P}}_{\nu }}0,\end{array}$$

where c is the constant in (A3), and the convergence in probability ${\to }_{{\mathbb{P}}_{\nu }}$ is with respect to the outer probability of ${\mathbb{P}}^{\infty }$ defined on $(E^{\infty },{\mathcal{E}}^{\infty })$.

The proof of Theorem 3 is postponed until Section 7.

4.1. Bootstrap Weights Examples

Let $({\mathbf{\xi }}_1,\cdots ,{\mathbf{\xi }}_n)$ be a class of real random variables satisfying Assumptions (A1)–(A4). We give some examples of bootstrap weights; for instance, refer to [38,61] for more explanations.

4.1.1. Bayesian Resampling Scheme

In this case, $({\mathbf{\xi }}_1,\dots ,{\mathbf{\xi }}_n)$ are positive i.i.d. random variables with mean $\mu$ and finite variance ${\sigma }^2$. The weights satisfy $\parallel {\mathbf{\xi }}_1{\parallel }_{2,1}<\infty$, and we define

$${\overline{\mathbf{\xi }}}_n=\sum _{i=1}^n{\mathbf{\xi }}_i.$$

The Bayesian bootstrapped weight can be defined as:

$${\mathbf{\xi }}_{ni}={\mathbf{\xi }}_i/{\overline{\mathbf{\xi }}}_n,$$

satisfying

$$\parallel {\mathbf{\xi }}_{n1}{\parallel }_{2,1}={\int }_0^{\infty }\sqrt{{\mathbb{P}}_{\xi }({\mathbf{\xi }}_{n1}\ge u)}du.$$

For ${\mathbf{\xi }}_{ni}\sim Exponential\left(1\right)$ or ${\mathbf{\xi }}_{ni}\sim Gamma(4,1)$, the Bayesian weights are distributionally equivalent with Dirichlet weights. For the value of $c^2$, we have:

$$\frac{1}{n}\sum _{i=1}^n{\left({\mathbf{\xi }}_{ni}-1\right)}^2\to \frac{{\sigma }^2}{{\mu }^2}:=c^2,n\to \infty .$$

4.1.2. Efron’s Resampling Scheme

For Efron’s bootstrap, we have

$$({\mathbf{\xi }}_1,\dots ,{\mathbf{\xi }}_n)\sim \mathrm{Multinomial}(n;n^{-1},\dots ,n^{-1}).$$

Condition (A1) follows directly. Condition (A3) follows from ([37] Lemma 4.1), and Condition (A2) is detailed in [43].

4.1.3. The Delete h-Jackknife

In [62], the authors permute deterministic weights $w_n$, where

$$w_n=\left\{\frac{n}{n-h},\dots ,\frac{n}{n-h},0,\dots ,0\right\}\mathrm{with}\sum _{i=1}^n{w}_{ni}=n$$

in order to build new bootstrap weights, and they defined the new weights ${\mathbf{\xi }}_{nj}:=w_{n{R}_n\left(j\right)}$ where $R_n(·)$ is a random permutation uniformly distributed over $\{1,\dots ,n\}$. These weights are called the delete h-Jackknife. In order to achieve Assumption (A3), we must assume that $h/n\to \alpha \in (0,1)$, as $c^2=h/(n-h)$ and $c>0$.

4.1.4. The Multivariate Hypergeometric Resampling Scheme

As its name indicates, the bootstrap weights of this scheme follow the multivariate hypergeometric distribution with density:

$$P\left({\mathbf{\xi }}_{n1}={\epsilon }_1,\dots ,{\xi }_{nn}={\epsilon }_n\right)=\frac{\left(\genfrac{}{}{0pt}{}{K}{{\epsilon }_1}\right)\cdots \left(\genfrac{}{}{0pt}{}{K}{{\epsilon }_n}\right)}{\left(\genfrac{}{}{0pt}{}{nK}{n}\right)},$$

where K is a positive integer. Assumption (A3) is satisfied with $c^2=(K-1)/K$.

Remark 2.

As was pointed out in [38], the preceding mentioned bootstraps are “smoother” in some way than the multinomial bootstrap because they place some (random) weight on all elements in the sample, whereas the multinomial bootstrap applies the positive weight at a proportion of about $1-{(1-n^{-1})}^n\to 1-e^{-1}=0.6322$ of each element of the sample, on average. Notice that when ${\omega }_i\sim Gamma(4,1)$, the ${\xi }_{ni}/n$ are equivalent to four spacings from a sample of $4n-1$ Uniform $(0,1)$ random variables. In [63,64], it was noticed that in addition to being four times more expensive to implement, the choice of four spacings depends on the functional of interest and is not universal.

Remark 3.

It is noteworthy that choosing the bootstrap weights ${\xi }_{ni}$ properly implies a smaller limit variance, that is, $c^2$ is smaller than 1. A typical example is the multivariate hypergeometric bootstrap ([38] Example 3.4) and the subsample bootstrap, ([65] Remark 2.2-(3)). A thorough treatment of the weight selection is undoubtedly outside the scope of the current work; for review, we refer the readers to [66].

Remark 4.

In the present paper, we considered a renewal type of bootstrap for atomic Markov chains under minimal moment conditions on renewal times. The atomic Markov chains assumption can be dropped by mimicking the ideas of [50,51] by introducing an artificial atom and deriving the bootstrap procedure that applies to nonatomic Markov chains. Precisely, in the case of a general irreducible chain X with a transition kernel $\Pi (x,dy)$ satisfying a minorization condition:

$$\forall x\in S,\Pi (x,dy)⩾\delta \psi \left(dy\right),$$

for an accessible measurable set S, a probability measure ψ and $\delta \in ]0,1[$ (note that such a minorization condition always holds for Π or an iterate when the chain is irreducible), an atomic extension $(X,Y)$ of the chain may be explicitly constructed by the Nummelin splitting technique (see [49]) from the parameters $(S,\delta ,\psi )$ and the transition probability Π, see for instance [47,67]. From a practical viewpoint, the size of the first block may be large compared to the size n of the whole trajectory, for instance, in the case where the expected return time to the (pseudo-)atom when starting with the initial probability distribution is large. The effective sample size for constructing the data blocks and the corresponding statistic is then dramatically reduced. However, in [68], some simulations were given together with examples including content-dependent storage systems and general AR models supporting the method discussed in this work.

5. Applications

Example 1

(Symmetry test). This example gives an application for the bootstrap U-statistics, inspired by the goodness-of-fit tests in [69], where they considered the symmetry test for the distribution of $X_t$. Let ${\left(X_t\right)}_{t\in \mathbb{N}}$ be a stationary mixing process with $f_X(·)$ the Lebesgue density. We test the hypothesis:

$$\left\{\begin{array}{cccc}{\displaystyle H_0:}& f_X\left(u\right)& =f_X(-u),& almost every were,\\ \\ {\displaystyle H_1:}& f_X\left(u\right)& \ne f_X(-u)& on a set of positive measure.\end{array}\right.$$

(21)

The estimator of $f_X\left(u\right)$ is:

$${\widehat{f}}_X\left(u\right)=\frac{1}{n{h}_n}\sum _{i=1}^{n}K\left(\frac{u-X_i}{h_n}\right),$$

where $K(·)$ is a kernel function and $h_n>0$ is a smoothing parameter or the bandwidth. An appropriate estimator of the integrated squared difference represent the symmetry test:

$$I={\int }_{\mathbb{R}}{\left(f_X\left(u\right)-f_X(-u)\right)}^{2}du.$$

According to [69], I can be estimated by

$${\widehat{I}}_n:=\frac{4}{n^2{h}_n}\sum _{1\le i<j\le n}{\Phi }_n\left(X_i,X_j\right),$$

where ${\Phi }_n\left(X_i,X_j\right)=K_{X_i,X_j}-K_{X_i,-X_j}$ with $K_{X_i,Y_j}=K\left(\frac{X_i-Y_j}{h_n}\right)$, for $Y_j\in \left\{X_j,-X_j\right\}$. Clearly, ${\widehat{I}}_n$ is a degenerate U-statistic with kernel varying with the sample size n. Thus, the stationary bootstrap test,

$${\widehat{I}}_n^{*}:=\frac{4}{n^2{h}_n}\sum _{1\le i<j\le n}{\Phi }_n\left(X_i^{*},X_j^{*}\right),$$

can be shown to have the same limit as ${\widehat{I}}_n$.

Example 2

(Kendall’s tau). The covariance matrix quantifies the linear dependency in a random vector. The rank correlation is another measure of the nonlinear dependency in a random vector. Two generic vectors $y=\left(y_1,y_2\right)$ and $z=\left(z_1,z_2\right)$ in ${\mathbb{R}}^2$ are said to be concordant if $\left(y_1-z_1\right)\left(y_2-z_2\right)>0$. For $m,k=1,\dots ,p$, define

$${\tau }_{mk}=\frac{1}{n(n-1)}\sum _{1\le i\ne j\le n}\mathbf{1}\left\{\left(X_{im}-X_{jm}\right)\left(X_{ik}-X_{jk}\right)>0\right\}.$$

Then, Kendall’s tau rank correlation coefficient matrix $T={\left\{{\tau }_{mk}\right\}}_{m,k=1}^p$ is a matrix-valued U-statistic with a bounded kernel. It is clear that ${\tau }_{mk}$ quantifies the monotonic dependency between $\left(X_{1m},X_{1k}\right)$ and $\left(X_{2m},X_{2k}\right)$ and it is an unbiased estimator of

$$\mathbb{P}\left(\left(X_{1m}-X_{2m}\right)\left(X_{1k}-X_{2k}\right)>0\right),$$

that is, the probability that $\left(X_{1m},X_{1k}\right)$ and $\left(X_{2m},X_{2k}\right)$ are concordant.

Example 3

(Test of independence). In [2] the author introduced the parameter

$$▵={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{D}^2(y_1,y_2)dF(y_1,y_2),$$

where $D(y_1,y_2)=F(y_1,y_2)-F(y_1,\infty )F(\infty ,y_2)$ and $F(·,·)$ is the distribution function of $Y_1$ and $Y_2$. The parameter Δ has the property that $▵=0$ if and only if $Y_1$ and $Y_2$ are independent. From [8], an alternative expression for Δ can be developed by introducing the functions

$$\psi \left(y_1,y_2,y_3\right)=\left\{\begin{array}{cc}1\hfill & if{y}_2\le y_1<y_3\hfill \\ 0\hfill & if{y}_1<y_2,y_{3}or{y}_1\ge y_2,y_3\hfill \\ -1\hfill & if{y}_3\le y_1<y_2\hfill \end{array}\right.$$

and

$$\phi \left(y_{1,1},y_{1,2},\dots ,y_{5,1},y_{5,2}\right)=\frac{1}{4}\psi \left(y_{1,1},y_{1,2},y_{1,3}\right)\psi \left(y_{1,1},y_{1,4},y_{1,5}\right)\psi \left(y_{1,2},y_{2,2},y_{3,2}\right)\psi \left(y_{1,2},y_{4,2},y_{5,2}\right).$$

We have

$$▵=\int \dots \int \phi \left(y_{1,1},y_{1,2},\dots ,y_{5,1},y_{5,2}\right)dF\left(y_{1,1},y_{1,2}\right)\dots dF\left(y_{1,5},y_{2,5}\right).$$

The corresponding U-statistics may be used to test the independence.

6. Conclusions

The present paper was concerned with the randomly weighted bootstrap of the U-process in a Markov framework. A large number of bootstrap resampling schemes emerged as special cases of our setting, in particular, the multinomial bootstrap, which is the best-known bootstrap scheme introduced by [26]. One of the main tools was the approximation of the Markov U-process by the corresponding regenerative one. We looked to mimic this result in Proposition 2, in order to approximate the weighted-bootstrap U-process $U_n^{*}$ to the regenerative weighted-bootstrap U-process $R_{l_n}^{*}$. Other technical arguments were given in Lemma 1 extended from the work of [47]. These intricate tools were used to reach the full independence of regenerative block variables by proving that a deterministic one could substitute the random size of blocks, which was the main problem for the extension of the bootstrap results to the Markov framework. After a lengthy proof to arrive at independence, we used the results of [46]. All the above steps led us to prove the weak convergence of the regenerative-block weighted-bootstrap U-process, which implied the weak convergence of the weighted-bootstrap U-process. It will be of interest to consider the extension of the paper to the semi-Markov setting. A more delicate problem is to consider the setting of incomplete data such as censored cases or missing data. To the best of our knowledge, this problem has not been considered, even for the original sample (without bootstrap) in the Markov framework. It would be interesting to extend our work to the case of the local stationary process, which requires nontrivial mathematics; this would go well beyond the scope of the present paper.

7. Mathematical Development

This section is devoted to the proof of our results. The previously defined notations continue to be used in what follows.

Proof of Proposition 2.

We have

$$\begin{array}{c}{\displaystyle U_n^{*}\left(h\right)-\left(\genfrac{}{}{0pt}{}{l_n-1}{m}\right){\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}{R}_{l_n}^{*}\left(h\right)}\hfill \\ ={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}{\xi }_{i_1,n^{*}}\dots {\xi }_{i_m,n^{*}}h(X_{i_1},\dots ,X_{i_m})\hfill \\ -{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}{\xi }_{i_1,l_n}\dots {\xi }_{i_m,l_n}{\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})\hfill \\ ={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}{\xi }_{i_1,n^{*}}\dots {\xi }_{i_m,n^{*}}h(X_{i_1},\dots ,X_{i_m})\hfill \\ -{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}{\xi }_{i_1,l_n}\dots {\xi }_{i_m,l_n}{\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})\hfill \\ +{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}h(X_{i_1},\dots ,X_{i_m})-{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}h(X_{i_1},\dots ,X_{i_m})\hfill \\ +{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}\left\{{\xi }_{i_1,l_n}\dots {\xi }_{i_m,l_n}-1\right\}{\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})\hfill \\ -{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}\left\{{\xi }_{i_1,l_n}\dots {\xi }_{i_m,l_n}-1\right\}{\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})\hfill \\ ={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}\left\{{\xi }_{i_1,n^{*}}\dots {\xi }_{i_m,n^{*}}-1\right\}h(X_{i_1},\dots ,X_{i_m})\hfill \\ -{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}\left\{{\xi }_{i_1,l_n}\dots {\xi }_{i_m,l_n}-1\right\}{\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})\hfill \\ +{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}{\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})\hfill \\ -{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}h(X_{i_1},\dots ,X_{i_m}).\hfill \end{array}$$

Given $J\subseteq \{1,\dots ,m\}(J=\varnothing$ is not excluded), and $\mathbf{i}=\left(i_1,\dots ,i_m\right)\in$${\{1,\dots ,n\}}^m$, we set ${\mathbf{i}}_J$ to be the point of ${\{1,\dots ,n\}}^{\left|J\right|}$ obtained from i by deleting the coordinates in the places not in J (e.g., if $\mathbf{i}=(3,4,2,1)$, then $\left.{\mathbf{i}}_{\{1,3\}}=(3,2)\right)$. Furthermore, ${\sum }_{i_J}$ indicates the sum over $1\le i_j\le n,j\in J$; for instance, if $m=4$ and $J=\{1,3\}$, then

$$\sum _{{\mathbf{i}}_J}{h}_{\mathbf{i}}=\sum _{{\mathbf{i}}_{\{1,3\}}}{h}_{i_1,i_2,i_3,i_4}=\sum _{1\le i_1,i_3\le n}{h}_{i_1,i_2,i_3,i_4}\left(X_{i_1}^{\left(1\right)},\dots ,X_{i_4}^{\left(4\right)}\right).$$

By convention, ${\sum }_{\mathbf{i}}a=a$. Notice that

$$\begin{array}{c}{\displaystyle \mathbb{E}{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}\left\{{\xi }_{i_1,n}\dots {\xi }_{i_m,n}-1\right\}h(X_{i_1},\dots ,X_{i_m})}\hfill \\ ={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_n^m}\mathbb{E}\left\{{\xi }_{i_1,n}\dots {\xi }_{i_m,n}-1\right\}\mathbb{E}h(X_{i_1},\dots ,X_{i_m})\hfill \\ ={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{{\mathbf{i}}_{\{1,\dots ,m-1\}}}\sum _{j=1}^m\mathbb{E}\left\{\prod _{k=1,k\ne j}^m{\xi }_{i_k,n}\mathbb{E}\left\{\sum _{i_j=1}^n({\xi }_{i_j,n}-1)\mid \prod _{k=1,k\ne j}^m{\xi }_{i_k,n}\right\}\right\}\hfill \\ \times \mathbb{E}h(X_{i_1},\dots ,X_{i_m})\hfill \\ =0.\hfill \end{array}$$

In a similar way, we have

$$\begin{array}{c}\hfill \mathbb{E}{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}\left\{{\xi }_{i_1,l_n}\dots {\xi }_{i_m,l_n}-1\right\}{\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})=0.\end{array}$$

Making use of Proposition 1 and the law of large numbers, we infer that

$$\begin{array}{c}\hfill U_n^{*}\left(h\right)-\left(\genfrac{}{}{0pt}{}{l_n-1}{m}\right){\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}{R}_{l_n}^{*}\left(h\right)=0,a.s.\end{array}$$

Hence, the proof is completed. □

Proof of Lemma 1.

The proof of part $i)$ and part $iii)$ follows from ([47] Lemma 3.1 and Lemma 3.2). In order to prove $ii)$, we need to show that, for every $ϵ>0$,

$${\mathbb{P}}_{\nu }\times {\mathbb{P}}_{\xi |X}\left(\left|\frac{n*}{n}-1\right|>ϵ\right)\to 0,$$

(22)

which follows if, conditioned on the sample,

$${\mathbb{P}}_{\xi |X}\left(\left|\frac{n*}{n}-1\right|>ϵ\right)\to 0.$$

(23)

We have:

$$\begin{array}{ccc}\hfill \frac{n*}{n}-1& =& \frac{{\sum }_{i=1}^{l_n}{\xi }_i{\tau }_i}{n}-1=\frac{l_n}{n}\left[\frac{{\sum }_{i=1}^{l_n}\left({\xi }_i{\tau }_i+{\xi }_i{\mathbb{E}}^{*}\left(\tau \right)-{\xi }_i{\mathbb{E}}^{*}\left(\tau \right)\right)-n}{l_n}\right]\hfill \\ & =& \frac{l_n}{n}\left[\frac{{\sum }_{i=1}^{l_n}{\xi }_i\left({\tau }_i-{\mathbb{E}}^{*}\left(\tau \right)\right)}{l_n}\right]+\frac{l_n}{n}\left[\frac{{\sum }_{i=1}^{l_n}{\xi }_i{\mathbb{E}}^{*}\left(\tau \right)}{l_n}-\frac{n}{l_n}\right]\hfill \\ & =& I+II.\hfill \end{array}$$

(24)

We denote by ${\mathbb{E}}^{*}$ the expectation conditionally on $X_1,\dots ,X_n$. By the fact that ${\tau }_i$ are i.i.d. and using Chebyshev’s inequality, we have:

$$\begin{array}{ccc}\hfill {\mathbb{P}}_{\xi |X}\left(\left|I\right|>ϵ\right)& \le & {ϵ}^{-2}{\left(\frac{l_n}{n}\right)}^2\frac{1}{l_n}{\mathbb{E}}_{\xi |X}{\left({\xi }_{1,l_n}\left({\tau }_1-{\mathbb{E}}^{*}\left(\tau \right)\right)\right)}^2\hfill \\ & \le & 2{ϵ}^{-2}\left(\frac{l_n}{n}\right)\frac{1}{n}\mathbb{E}\left({\xi }_{1,l_n}^2\right)\frac{1}{l_n}\sum _{i=1}^{l_n}{\tau }_i^2\to 0\mathrm{in}\mathrm{probability}.\hfill \end{array}$$

The last inequality follows using $i)$, which implies that $\frac{l_n}{n}\to p$ and $iii)$ where

$$\frac{1}{l_n}\sum _{i=1}^{l_n}{\tau }_i^2\to \mathbb{E}\left({\tau }^2\right),$$

for $\mathbb{E}\left({\xi }_{1,l_n}^2\right)<\infty$. For $II$ we have:

$$\begin{array}{ccc}\hfill II& =& \frac{l_n}{n}\left[\frac{{\sum }_{i=1}^{l_n}{\xi }_i{\mathbb{E}}^{*}\left(\tau \right)}{l_n}-\frac{n}{l_n}\right]=\frac{l_n}{n}\left[{\mathbb{E}}^{*}\left(\tau \right)-\frac{n}{l_n}\right]\mathrm{by}\left(\mathrm{A}1\right)\hfill \\ & =& \frac{l_n}{n}\left[\frac{1}{l_n}\sum _{i=1}^{l_n}\left({\tau }_i-\mathbb{E}\left(\tau \right)+\mathbb{E}\left(\tau \right)\right)-\frac{n}{l_n}\right]\hfill \\ & =& \frac{l_n}{n}\left[\frac{1}{l_n}\sum _{i=1}^{l_n}\left({\tau }_i-\mathbb{E}\left(\tau \right)\right)\right]+\frac{l_n}{n}\left[\mathbb{E}\left(\tau \right)-\frac{n}{l_n}\right].\hfill \end{array}$$

The last equality converges to zero by the fact that $n/l_n\to \alpha =\mathbb{E}\left(\tau \right)$ and by iii)

$$\frac{1}{l_n}\sum _{i=1}^{l_n}\left({\tau }_i-\mathbb{E}\left(\tau \right)\right)\to 0.$$

This proves Lemma 1. □

Proof of Theorem 3.

For the weak convergence, we need to show the finite-dimensional convergence and the asymptotic equicontinuity. According to Proposition 2 and [6], the finite-dimensional convergence is considered if, for every fixed finite collection of functions $\{f_1,\dots ,f_k\}\subset \mathcal{F}$,

$$\begin{array}{c}\hfill \left({\left(\genfrac{}{}{0pt}{}{n}{m_1}\right)}^{-1/2}{R}_{l_n}^{*}\left(f_1\right),\dots {\left(\genfrac{}{}{0pt}{}{n}{m_k}\right)}^{-1/2}{R}_{l_n,}^{*}\left(f_k\right)\right)\to \left({\mathbb{K}}_{\mathbb{P}}\left(f_1\right),\dots ,{\mathbb{K}}_{\mathbb{P}}\left(f_k\right)\right),\end{array}$$

where ${\mathbb{K}}_P$ is the Gaussian chaos process. According to Cramér–Wold and the countability of $\mathcal{F}$, we only need to show that for any $f\in L_2^{c,m}\left(\mathbb{P}\right)$,

$$\begin{array}{c}\hfill \underset{\psi \in \mathrm{BL}}{sup}|\mathbb{E}\left[\psi \left({\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}{R}_{l_n}^{*}\left(f\right)\right)|\{{\mathbf{B}}_i\}]-\mathbb{E}\psi (c·{\mathbb{K}}_{\mathbb{P}}\left(f\right))\right|\to 0 \mathrm{a}.\mathrm{s}.\end{array}$$

(25)

By ([6] Section 4.2) and ([29] Section 2A), any $f\in L_2^{c,m}\left(\mathbb{P}\right)$ can be expanded in $L_2\left({\mathbb{P}}^m\right)$ by $f={\sum }_{q=1}^{\infty }{c}_q{h}_m^{{\psi }_q}$, where $\left\{c_q\right\}$ is a sequence of real numbers and

$$h_m^{{\psi }_q}(x_1,\dots ,x_m)\equiv {\psi }_q\left(x_1\right)\cdots {\psi }_q\left(x_m\right)$$

for some bounded ${\psi }_q\in L_2^{c,1}\left(\mathbb{P}\right)$. Fix $ϵ>0$. Then, there exists $Q_{ϵ}\in \mathbb{N}$ such that with $f_n^{ϵ}\equiv {\sum }_{q=1}^{Q_{ϵ}}{c}_q{h}_m^{{\psi }_q}$,

$${\parallel f-f_{ϵ}\parallel }_{L_2\left(P^m\right)}\le ϵ.$$

The left-hand side of (25) can be further bounded by

$$\begin{array}{cc}\hfill & \underset{\psi \in \mathrm{BL}}{sup}|\mathbb{E}\left[\psi \left({\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}{R}_{l_n}^{*}\left(f\right)\right)|\{{\mathbf{B}}_i\}]-\mathbb{E}\psi (c·{\mathbb{K}}_{\mathbb{P}}\left(f\right))\right|\hfill \\ \hfill & \le \underset{\psi \in \mathrm{BL}}{sup}\left|\mathbb{E}\right[\psi \left({\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}{R}_{l_n}^{*}\left(f\right)\right)|\{{\mathbf{B}}_i\}]-\mathbb{E}\left[\psi \left({\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}{R}_{l_n}^{*}\left(f^{ϵ}\right)\right)|\{{\mathbf{B}}_i\}]\right|\hfill \\ \hfill & +\underset{\psi \in \mathrm{BL}}{sup}|\mathbb{E}\left[\psi \left({\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}{R}_{l_n}^{*}\left(f^{ϵ}\right)\right)|\{{\mathbf{B}}_i\}]-\mathbb{E}\psi (c·{\mathbb{K}}_{\mathbb{P}}\left(f^{ϵ}\right))\right|\hfill \\ \hfill & +\underset{\psi \in \mathrm{BL}}{sup}\left|\mathbb{E}\psi (c·{\mathbb{K}}_{\mathbb{P}}\left(f^{ϵ}\right))-\mathbb{E}\psi (c·{\mathbb{K}}_{\mathbb{P}}\left(f\right))\right|\hfill \\ \hfill & \equiv \left(I\right)+\left(II\right)+\left(III\right).\hfill \end{array}$$

(26)

Let $\overline{f^{ϵ}}\equiv f-f^{ϵ}$; noting that $\psi$ is bounded by one and using Lemma 1, we can replace $l_n$ by $\phi \left(n\right)=⌊\frac{n}{{\mathbb{E}}_A\left(\tau \right)}⌋$ which is deterministic. In the following, we denote by $\pi$ a random permutation uniformly distributed over $\mathsf{\Sigma }\left(n\right)$, the set of all permutations over $1,\dots ,n.$ We have

$$\begin{array}{cc}\hfill {\left(I\right)}^2& \le {\mathbb{E}}^{*}{\left|2\wedge {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}{R}_{l_n}^{*}\left(\overline{f^{ϵ}}\right)\right|}^2\hfill \\ \hfill & \lesssim {\mathbb{E}}_{\xi |X}{\mathbb{E}}_R{\left(1\wedge n^{-m/2}\sum _{1\le i_1\ne \dots \ne i_m\le \phi \left(n\right)}\left({\xi }_{{\pi }_{i_1}}-1\right)\cdots \left({\xi }_{{\pi }_{i_m}}-1\right){\overline{f}}_{\epsilon }\left({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}\right)\right)}^2\hfill \\ \hfill & \lesssim \sum _{\begin{array}{c}{\alpha }_i\in \{1,2\}:{\sum }_{i=1}^l{\alpha }_i=2m,{\alpha }_1\ge \dots \ge {\alpha }_l,1\le l\le m\end{array}}{\mathbb{E}}_{\xi }^{*}[1\wedge n^{-m/2}{\mathbb{E}}_R\left[\prod _{i=1}^l{\left({\xi }_{{\pi }_i}-1\right)}^{{\alpha }_i}\right]\hfill \\ \hfill & \times \sum _{\begin{array}{c}{i}_1\ne \dots \ne i_m,\\ i_1^{\prime }\ne \dots \ne i_m^{\prime },\\ i_j=i_j^{\prime },1\le j\le max\{j:{\alpha }_j=2\}\end{array}}{\overline{f}}^{ϵ}\left({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m\left(n\right)}\right){\overline{f}}_n^{ϵ}\left({\mathbf{B}}_{i_1^{\prime }},\dots ,{\mathbf{B}}_{i_m^{\prime }\left(n\right)}\right)]\hfill \\ \hfill & \lesssim \sum _{\begin{array}{c}{\alpha }_i\in \{1,2\}:{\sum }_{i=1}^l{\alpha }_i=2m,\\ {\alpha }_1\ge \dots \ge {\alpha }_l,1\le \ell \le m\end{array}}\mathbb{E}{\left[1\wedge \frac{1}{n}\sum _{i=1}^n{\left({\xi }_i-1\right)}^2\right]}^m\hfill \\ \hfill & \times n^{-\ell }\sum _{\begin{array}{c}{i}_1\ne \dots \ne i_m,\\ i_1^{\prime }\ne \dots \ne i_m^{\prime },\\ i_j=i_j^{\prime },1\le j\le max\{j:{\alpha }_j=2\}\end{array}}{\overline{f}}^{ϵ}\left({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}\right){\overline{f}}^{ϵ}\left({\mathbf{B}}_{i_1^{\prime }},\dots ,{\mathbf{B}}_{i_m^{\prime }}\right).\hfill \end{array}$$

We have, according to [43], for $({\xi }_1,\dots ,{\xi }_n)$ a non-negative sequence of variables such that ${\sum }_{i=1}^n{\xi }_i=n$ and for $\pi =({\pi }_1,\dots ,{\pi }_n)$ a random permutation of $\{1,\dots ,n\}$, for any $\ell \in \mathbb{N}$ and $\alpha =({\alpha }_1,\dots ,{\alpha }_{\ell })\in {\mathbb{N}}^{\ell }$,

$$\begin{array}{c}\hfill \left|{\mathbb{E}}_{\pi }\left[\prod _{i=1}^{\ell }{\left({\xi }_{{\pi }_i}-1\right)}^{{\alpha }_i}\right]\right|\le C_{l,\alpha }{n}^{-\ell }{\left[\sum _{i=1}^{l_n}{\left({\xi }_i-1\right)}^2\right]}^{{\sum }_i{\alpha }_i/2}.\end{array}$$

Furthermore, according to [70,71], we have:

$$\begin{array}{cc}\hfill & n^{-\ell }\sum _{\begin{array}{c}{i}_1\ne \dots \ne i_m,\\ i_1^{\prime }\ne \dots \ne i_m^{\prime },\\ i_j=i_j^{\prime },1\le j\le max\{j:{\alpha }_j=2\}\end{array}}{\overline{f}}^{ϵ}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}){\overline{f}}^{ϵ}({\mathbf{B}}_{i_1^{\prime }},\dots ,{\mathbf{B}}_{i_m^{\prime }})\hfill \\ \hfill \\ \hfill & {\to }_{\mathrm{a}.\mathrm{s}.}{\mathbb{E}}_{\mathbf{A}}{\left(\tau \right)}^{-\ell }\mathbb{E}{\overline{f}}^{ϵ}({\mathbf{B}}_1,\dots ,{\mathbf{B}}_m){\overline{f}}^{ϵ}({\mathbf{B}}_1^{\prime },\dots ,{\mathbf{B}}_m^{\prime })\hfill \\ \hfill & (\mathrm{where}{\mathbf{B}}_j={\mathbf{B}}_j^{\prime }\mathrm{for}1\le j\le max\{j:{\alpha }_j=2\}\mathrm{and}\mathrm{for}{l}_n/n\to {\mathbb{E}}_{\mathbf{A}}{\left(\tau \right)}^{-1})\hfill \\ \hfill \\ \hfill & \le {\mathbb{E}}_{\mathbf{A}}{\left(\tau \right)}^{-\ell }{\mathbb{P}}^m{\overline{f}}^{{ϵ}^2}\le {ϵ}^2.(\mathrm{Under} \mathrm{Conditions} (\mathrm{C}.1) \mathrm{and} (\mathrm{C}.3).)\hfill \end{array}$$

Hence we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim\; sup}\left(I\right){\lesssim }_{m,\xi }ϵ,\mathrm{a}.\mathrm{s}.\end{array}$$

(27)

Now for the second term, we have:

$$\begin{array}{cc}\hfill & {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}{R}_{l_n}^{*}\left(f^{ϵ}\right)\hfill \\ \hfill & =\frac{1}{{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}}\sum _{q=1}^{Q_{ϵ}}{c}_q\sum _{1\le i_1<\dots <i_m\le \phi \left(n\right)}({\xi }_{{\pi }_{i_1}}-1)\cdots ({\xi }_{{\pi }_{i_m}}-1){\psi }_q\left({\mathbf{B}}_{i_1}\right)\cdots {\psi }_q\left({\mathbf{B}}_{i_m}\right)\hfill \\ \hfill & =\frac{\phi {\left(n\right)}^{m/2}}{{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}}\sum _{q=1}^{Q_{ϵ}}{c}_q{\mathfrak{R}}_m\left(\frac{1}{\phi {\left(n\right)}^{1/2}}\sum _{i=1}^{\phi \left(n\right)}({\xi }_{{\pi }_i}-1){\psi }_q\left({\mathbf{B}}_i\right),\dots ,\frac{1}{\phi {\left(n\right)}^{m/2}}\sum _{i=1}^{\phi \left(n\right)}{({\xi }_{{\pi }_i}-1)}^m{\psi }_q^m\left({\mathbf{B}}_i\right)\right)\hfill \\ \hfill & \equiv (1+o\left(1\right)){(m!)}^{1/2}{\mathbb{E}}_{\mathbf{A}}{\left(\tau \right)}^{-m/2}\sum _{q=1}^{Q_{ϵ}}{c}_q{\mathfrak{R}}_m(A_{\phi \left(n\right),q}^{\left(1\right)},\dots ,A_{\phi \left(n\right),q}^{\left(m\right)}),\hfill \end{array}$$

where ${\mathfrak{R}}_m$ is the polynomial of degree m (see [6], p. 175):

$$\begin{array}{c}\hfill \sum _{1\le i_1<\dots <i_m\le \phi \left(n\right)}{t}_{i_1}\cdots t_{i_m}={\mathfrak{R}}_m\left(\sum _{i=1}^{\phi \left(n\right)}{t}_i,\sum _{i=1}^{\phi \left(n\right)}{t}_i^2,\dots ,\sum _{i=1}^{\phi \left(n\right)}{t}_i^m\right).\end{array}$$

(28)

As we mentioned before, this polynomial follows from Newton’s inequality and allows us to show a polynomial function as a sum of monomials. All we need now is to check each argument of this polynomial function. □

For $\ell =1$: We first recall the following lemma from [53].

Lemma 3

([53]). Let $(a_1,\dots ,a_n)$ be a vector and $({\xi }_1,\dots ,{\xi }_n)$ be a vector of exchangeable random variables. Suppose that

$$\begin{array}{c}\hfill {\overline{a}}_n=\frac{1}{n}\sum _{i=1}^n{a}_i=0,\frac{1}{n}\sum _{i=1}^n{a}_i^2\to {\sigma }^2,\underset{M\to \infty }{lim}\underset{n\to \infty }{lim\; sup}\frac{1}{n}\sum _{i=1}^n{a}_i^2{\mathbb{1}}_{\left\{\right|a_i|>M\}}=0,\end{array}$$

and

$$\begin{array}{c}\hfill {\overline{\xi }}_n=\frac{1}{n}\sum _{i=1}^n{\xi }_i=0,\frac{1}{n}\sum _{i=1}^n{\xi }_i^2{\to }_{{\mathbb{P}}_{\xi }}{\alpha }^2,\frac{1}{n}\underset{1\le i\le n}{max}{\xi }_i^2{\to }_{{\mathbb{P}}_{\xi }}0.\end{array}$$

Then,

$$\frac{1}{\sqrt{n}}\sum _{i=1}^n{a}_i{\xi }_i\to \mathcal{N}\left(0,{\sigma }^2{\alpha }^2\right).$$

Applying Lemma 3 with $a_i\equiv {\psi }_q\left({\mathbf{B}}_i\right)-{\mathbb{P}}_n{\psi }_q$ and ${\xi }_i$ replaced by ${\xi }_{R_i}-1$, we can see that

$$A_{\phi \left(n\right),q}^{\left(1\right)}\to c·{\mathbb{G}}_{\mathbb{P}}\left({\psi }_q\right),a.s.,$$

where ${\mathbb{G}}_{\mathbb{P}}$ is a Gaussian process defined on $L_2^{c,1}\left(\mathbb{P}\right)$ with covariance

$$\mathbb{E}{\mathbb{G}}_{\mathbb{P}}\left(f\right){\mathbb{G}}_{\mathbb{P}}\left(g\right)=\mathbb{P}\left(fg\right),\mathrm{for}f,g\in L_2^{c,1}\left(\mathbb{P}\right).$$

For $\ell =2$: Note that

$$\begin{array}{cc}\hfill {\mathbb{E}}_{\pi }^{*,\xi }\left(A_{\phi \left(n\right),q}^{\left(2\right)}\right)& =\frac{1}{\phi \left(n\right)}\sum _{i=1}^{\phi \left(n\right)}{({\xi }_i-1)}^2·\frac{1}{\phi \left(n\right)}\sum _{i=1}^{\phi \left(n\right)}{\psi }_q^2\left({\mathbf{B}}_i\right)\hfill \\ \hfill & {\to }_{{\mathbb{P}}_{\nu ,\xi }}{c}^2\mathbb{E}{\psi }_q^2\left({\mathbf{B}}_1\right)=c^2{\mathbb{E}}_A\left[{\left(\sum _{i={\mathcal{T}}_0+1}^{{\mathcal{T}}_1}{h}_1\left(X_i\right)\right)}^2\right],\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

Furthermore,

$$\begin{array}{cc}\hfill & {\mathrm{Var}}^{*,\xi }\left(A_{\phi \left(n\right),q}^{\left(2\right)}\right)\hfill \\ \hfill & ={\mathbb{E}}^{*,\xi }{\left(A_{\phi \left(n\right),q}^{\left(2\right)}\right)}^2-{\left({\mathbb{E}}_{\pi }^{*,\xi }\left(A_{\phi \left(n\right),q}^{\left(2\right)}\right)\right)}^2\hfill \\ \hfill & ={\mathbb{E}}_{\pi }^{*,\xi }{\left[\frac{1}{\phi \left(n\right)}\sum _{i=1}^{\phi \left(n\right)}{\left({\xi }_i-1\right)}^2{\psi }_q^2\left({\mathbf{B}}_{{\pi }_i}\right)\right]}^2-{\left[\frac{1}{\phi \left(n\right)}\sum _{i=1}^{\phi \left(n\right)}{\left({\xi }_i-1\right)}^2{\mathbb{P}}_n{\psi }_q^2\right]}^2\hfill \\ \hfill & =\frac{1}{{\phi \left(n\right)}^2}\sum _{i,j}{\left({\xi }_i-1\right)}^2{\left({\xi }_j-1\right)}^2\left[{\mathbb{E}}_{\pi }^{*}{\psi }_q^2\left({\mathbf{B}}_{{\pi }_i}\right){\psi }_q^2\left({\mathbf{B}}_{{\pi }_j}\right)-{\left({\mathbb{P}}_n{\psi }_q^2\right)}^2\right]\hfill \\ \hfill & =\frac{1}{{\phi \left(n\right)}^2}\sum _i{\left({\xi }_i-1\right)}^4\left[{\mathbb{E}}_{\pi }^{*}{\psi }_q^4\left({\mathbf{B}}_{{\pi }_i}\right)-{\left({\mathbb{P}}_n{\psi }_q^2\right)}^2\right]\hfill \\ \hfill & +\frac{1}{{\phi \left(n\right)}^2}\sum _{i\ne j}{\left({\xi }_i-1\right)}^2{\left({\xi }_j-1\right)}^2\left[{\mathbb{E}}_{\pi }^{*}{\psi }_q^2\left({\mathbf{B}}_{{\pi }_i}\right){\psi }_q^2\left({\mathbf{B}}_{{\pi }_j}\right)-{\left({\mathbb{P}}_n{\psi }_q^2\right)}^2\right]\hfill \\ \hfill & \le \frac{1}{{\phi \left(n\right)}^2}\sum _i{\left({\xi }_i-1\right)}^4·{\mathbb{P}}_n{\psi }_q^4+\frac{1}{{\phi \left(n\right)}^2}{\left(\sum _i{({\xi }_i-1)}^2\right)}^2·\frac{1}{\phi \left(n\right)-1}{P}_n{\psi }_q^4\hfill \\ \hfill & \le C\frac{1}{{\phi \left(n\right)}^2}\sum _{i=1}^n{\left({\xi }_i-1\right)}^4·{\mathbb{P}}_n{\psi }_q^4\hfill \\ \hfill & \le C{\parallel {\psi }_q\parallel }_{\infty }^4\frac{{max}_i{({\xi }_i-1)}^2}{\phi \left(n\right)}·\frac{1}{\phi \left(n\right)}\sum _{i=1}^{\phi \left(n\right)}({\xi }_i{-1)}^2{\to }_{{\mathbb{P}}_{\xi }}0,\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

The first inequality in the above display follows, since

$$\begin{array}{cc}\hfill & {\mathbb{E}}_{\pi }^{*}{\psi }_q^2\left({\mathbf{B}}_{{\pi }_i}{\mathbf{B}}_{{\pi }_i}\right){\psi }_q^2\left({\mathbf{B}}_{{\pi }_j}\right)-{\left({\mathbb{P}}_n{\psi }_q^2\right)}^2\hfill \\ \hfill & =\frac{1}{\phi \left(n\right)(\phi \left(n\right)-1)}\left[\sum _{i\ne j}{\psi }_q^2\left({\mathbf{B}}_{{\pi }_i}\right){\psi }_q^2\left({\mathbf{B}}_{{\pi }_j}\right)\right]-{\left({\mathbb{P}}_n{\psi }_q^2\right)}^2\hfill \\ \hfill & \le \frac{1}{\phi \left(n\right)-1}{\left({\mathbb{P}}_n{\psi }_q^2\right)}^2\le \frac{1}{\phi \left(n\right)-1}{\mathbb{P}}_n{\psi }_q^4.\hfill \end{array}$$

This shows that

$$A_{\phi \left(n\right),q}^{\left(2\right)}{\to }_{{\mathbb{P}}_{\xi }}{c}^2\mathbb{E}{\psi }_q^{2}a.s.$$

For $\ell \ge 3$:

$$\begin{array}{cc}\hfill {\mathbb{E}}_{\pi }^{*,\xi }\left|A_{\phi \left(n\right),q}^{(\ell )}\right|& \le \frac{1}{{\phi \left(n\right)}^{\ell /2}}\sum _{i=1}^{\phi \left(n\right)}|{\xi }_i-1{|}^{\ell }·\frac{1}{\phi \left(n\right)}\sum _{i=1}^{\phi \left(n\right)}{\left|{\psi }_q\left({\mathbf{B}}_i\right)\right|}^{\ell }\hfill \\ \hfill & \le {\left(\frac{{max}_i{\left|{\xi }_i-1\right|}^2}{\phi \left(n\right)}\right)}^{\frac{\ell -2}{2}}·\frac{1}{\phi \left(n\right)}\sum _{i=1}^{\phi \left(n\right)}{\left|{\xi }_i-1\right|}^2·{\parallel {\psi }_q\parallel }_{\infty }\hfill \\ \hfill & {\to }_{{\mathbb{P}}_{\xi }}0,\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

This shows that

$$A_{\phi \left(n\right),q}^{(\ell )}{\to }_{{\mathbb{P}}_{\xi }}0,a.s.$$

Then, we have

$$R_m(A_{\phi \left(n\right),q}^{\left(1\right)},\dots ,A_{\phi \left(n\right),q}^{\left(m\right)})\to R_m({\mathbb{G}}_{\mathbb{P}}\left(c{\psi }_q\right),\mathbb{E}{\left(c{\psi }_q\right)}^2,0,\dots ,0)=c{(m!)}^{-1/2}·{\mathbb{K}}_{\mathbb{P}}\left({\psi }_q\right)a.s.,$$

where ${\mathbb{K}}_{\mathbb{P}}$ is the Gaussian chaos process defined on (⊕ is the orthogonal sum in $L_2\left(E^{\infty },{\mathcal{E}}^{\infty },P^{\infty }\right)$)

$$\mathbb{R}\oplus L_2^{c,\mathbb{N}}\left(\mathbb{P}\right)\equiv \mathbb{R}\oplus \left({\oplus }_{m=1}^{\infty }{L}_2^{c,m}\left(\mathbb{P}\right)\right).$$

Hence, it follows that, by linearity of ${\mathbb{K}}_{\mathbb{P}}$,

$${\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}{R}_{l_n}^{*}\left({\overline{f}}^{ϵ}\right)\to c·{\mathbb{K}}_{\mathbb{P}}\left(f^{ϵ}\right),\mathrm{a}.\mathrm{s}.$$

The last term in (26) follows from the definition of ${\mathbb{K}}_{\mathbb{P}}$

$$\begin{array}{cc}\hfill \left(III\right)& \le c\sqrt{\mathbb{E}{\mathbb{K}}_{\mathbb{P}}^2\left({\overline{f}}^{ϵ}\right)}\to 0(ϵ\to 0).\hfill \end{array}$$

(29)

All these final results give the finite-dimensional convergence.

Now, we take a step-by-step approach to establish stochastic equicontinuity. We assume that the class of functions must be bounded, so we suppose that $h\le H$, for H an envelope. Throughout the following, we denote by

$${\mathcal{F}}_{\delta }:=\{f,g\in \mathcal{F}:d(f,g)\le \delta \}.$$

• Step 1

Let

$$\begin{array}{ccc}\hfill Z_n^{*}& :=& {\left(\genfrac{}{}{0pt}{}{n^{*}}{m}\right)}^{1/2}\left[U_n^{*}\left(h\right)-{\mathbb{E}}^{*}\left(U_n^{*}\left(h\right)\right)\right],\hfill \end{array}$$

(30)

and

$$\begin{array}{ccc}\hfill {\breve{T}}_{l_n}^{*}& :=& {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\left(\genfrac{}{}{0pt}{}{l_n}{m}\right)\left[R_{l_n}^{*}-{\mathbb{E}}^{*}\left(R_{l_n}^{*}\right)\right].\hfill \end{array}$$

(31)

In this step, we must prove that the stochastic equicontinuity of the U-process implies that of the regenerative U-process. This is a consequence of 1, and for the weighted bootstrap Proposition 2 and part ii) of Lemma 1.

• Step 2

Define

$${\breve{T}}_{l_n}^{*}:={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_h({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})$$

and

$${\tilde{T}}_{l_n}^{*}={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}).$$

Hypothesis: The stochastic equicontinuity of ${\breve{T}}_{l_n}^{*}$ implies the stochastic equicontinuity of ${\tilde{T}}_{l_n}^{*}$.

Proof. 

In order to prove the previous implication, we only need to show that:

$$\begin{array}{ccc}& & {\mathbb{P}}^{*}\left({∥{\tilde{T}}_{l_n}^{*}-{\breve{T}}_{l_n}^{*}∥}_{{\mathcal{F}}_{\delta }}>ϵ\right)\hfill \\ & \le & {\mathbb{P}}^{*}\left({∥{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_h({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})∥}_{{\mathcal{F}}_{\delta }}>ϵ\right).\hfill \end{array}$$

Suppose that $l_n\le \mathbb{E}\left(l_n\right)$, the opposite case can be treated in a similar way. We have

$$\begin{array}{ccc}& & {\mathbb{P}}^{*}\left({∥{\tilde{T}}_{l_n}^{*}-{\breve{T}}_{l_n}^{*}∥}_{{\mathcal{F}}_{\delta }}>ϵ\right)\hfill \\ & =& {\mathbb{P}}^{*}\left(∥{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_h({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})\right.\hfill \\ & & \left.{-{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_h({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})∥}_{{\mathcal{F}}_{\delta }}>ϵ\right)\hfill \\ & & \mathrm{Define}\mathfrak{I}:=\left\{(i_1,\dots ,i_m):1\le i_1<\dots <i_m\le \mathbb{E}\left(l_n\right):i_j\ne i_k\mathrm{for}j\ne k,\right.\hfill \\ & & \left.\mathrm{such}\mathrm{that}\exists \ell \in \{1,\dots ,m\}:l_n\le i_{\ell }\le \mathbb{E}\left(l_n\right)\right\}\hfill \\ & \le & {\mathbb{P}}^{*}\left({∥{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in \mathfrak{I}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_h({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})∥}_{{\mathcal{F}}_{\delta }}>ϵ\right)\hfill \\ & =& {\mathbb{P}}^{*}(\parallel {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in \mathfrak{I}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_h({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})>ϵ\hfill \\ & & \cap \left(|\mathbb{E}\left(l_n\right)-l_n|\le n/4\right){{\displaystyle \parallel }}_{{\mathcal{F}}_{\delta }}{\displaystyle )}+{\mathbb{P}}^{*}\left(|\mathbb{E}\left(l_n\right)-l_n|>n/4\right).\hfill \end{array}$$

However, $|\mathbb{E}\left(l_n\right)-l_n|=O_{\mathbb{P}}\left(\sqrt{n}\right)$ by Lemma 1, part i). Then, the exists a constant $K>0$, such that for every $ϵ>0$,

$${\mathbb{P}}^{*}\left(|\mathbb{E}\left(l_n\right)-l_n|>n/4\right)<ϵ,$$

and the first expression in the previous expression is bounded by:

$$\begin{array}{ccc}& & {\mathbb{P}}^{*}\left(\underset{M\le n/2+\mathbb{E}\left(l_n\right)}{max}{∥{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}^{{}^{\prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})∥}_{{\mathcal{F}}_{\delta }}>ϵ\right)\hfill \\ & & \mathrm{where}{\mathfrak{I}}^{{}^{\prime }}:=\left\{(i_1,\dots ,i_m):1\le i_1<\dots <i_m\le \mathbb{E}\left(l_n\right),\exists \ell =1,\dots ,m,\mathbb{E}\left(l_n\right)<i_{\ell }\le M,\right.\hfill \\ & & \left.i_j\ne i_k\mathrm{for}j\ne k\right\}\hfill \\ & \le & C_1{\mathbb{P}}^{*}\left({∥{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})∥}_{{\mathcal{F}}_{\delta }}>C_2ϵ\right)\hfill \\ & & \mathrm{where}{\mathfrak{I}}_m^{{}^{\prime \prime }}:=\left\{(i_1,\dots ,i_m):1\le i_1<\dots <i_m\le \mathbb{E}\left(l_n\right),\exists \ell =1,\dots ,m,\right.\hfill \\ & & \left.\mathbb{E}\left(l_n\right)<i_{\ell }\le \mathbb{E}\left(l_n\right)+n/2,i_j\ne i_k\mathrm{for}j\ne k\right\}.\hfill \end{array}$$

The last expression follows from the Montgomery–Smith inequality. Since

$$\mathbb{E}\left(l_n\right)/n\to {\alpha }^{-1},$$

the last expression matches the stochastic equicontinuity condition for ${\tilde{T}}_{l_n}^{*}$. This proves this step. □

Before passing to the next step, we introduce a new bootstrap sample. Define ${\widehat{\mathbf{B}}}_i:=\left\{X_{{\mathcal{T}}_{i-1}+1},\cdots ,X_{{\mathcal{T}}_i}\right\}$ for $i=1,\dots ,\mathbb{E}\left(l_n\right)$. Now, apply the weighted bootstrap procedure on the sample ${\left\{{\widehat{\mathbf{B}}}_i\right\}}_{i=1}^{\mathbb{E}\left(l_n\right)}$. This new procedure is the same as the old one for ${\mathbf{B}}_i$, but we aim here to replace the random quantity $l_n$ with a deterministic one, which is $\mathbb{E}\left(l_n\right)$.

• Step 3

Define:

$${\widehat{T}}_{l_n}^{*}={\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})$$

Hypothesis: The stochastic equicontinuity of ${\tilde{T}}_{l_n}^{*}$ implies the stochastic equicontinuity of ${\widehat{T}}_{l_n}^{*}$.

Proof. 

First case: $l_n\le \mathbb{E}\left(l_n\right)$:

In this case, all of the terms in the following computation should be multiplied with ${\mathbb{1}}_{(l_n\le \mathbb{E}\left(l_n\right))}$. We leave it out to keep the already complex notation simple. Define

$$\begin{array}{ccc}\hfill {\mathcal{A}}_n& :=& \{{\mathbf{B}}_1,\dots ,{\mathbf{B}}_{l_n}\}\hfill \\ \hfill {\overrightarrow{T}}_{l_n}^{*}& :=& {\widehat{T}}_{l_n}^{*}{\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n}+T_{l_n}^{*}{\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n^c}.\hfill \end{array}$$

${\overrightarrow{T}}_{l_n}^{*}$ is well defined, i.i.d., and has the same distribution as $T_{l_n}^{*}$ and $(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m$. Hence, if we show that:

$$\underset{\delta \to 0}{lim}\underset{n\to \infty }{lim\; sup}{\mathbb{P}}^{*}\left({∥{\overrightarrow{T}}_{l_n}^{*}∥}_{{\mathcal{F}}_{\delta }}>ϵ\right)=0\mathrm{in}\mathrm{probability},$$

then the stochastic equicontinuity of ${\tilde{T}}_{l_n}^{*}$ is established. However, we aim to approximate the one of ${\widehat{T}}_{l_n}^{*}$. In order to achieve our goal, it is sufficient to estimate:

$$\begin{array}{c}{\displaystyle {∥{\overrightarrow{T}}_{l_n}^{*}-{\widehat{T}}_{l_n}^{*}∥}_{{\mathcal{F}}_{\delta }}}\hfill \\ ={∥{\widehat{T}}_{l_n}^{*}{\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n}+T_{l_n}^{*}{\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n^c}-\left[{\widehat{T}}_{l_n}^{*}{\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n}+{\widehat{T}}_{l_n}^{*}{\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n^c}\right]∥}_{{\mathcal{F}}_{\delta }}\hfill \\ \le {∥T_{l_n}^{*}{\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n^c}∥}_{{\mathcal{F}}_{\delta }}+{∥{\widehat{T}}_{l_n}^{*}{\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n^c}∥}_{{\mathcal{F}}_{\delta }}\hfill \\ :=I_n+I{I}_n.\hfill \end{array}$$

(32)

For $I_n$: Let

$$S_n^{*}:=\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m}{\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n^c},$$

conditioned on the sample, we have:

$$\begin{array}{ccc}& & \mathcal{L}\left(\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}){\mathbb{1}}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n^c}\right)\hfill \\ & =& \mathcal{L}\left(\sum _{(i_1,\dots ,i_m)\in I_{S_n^{*}}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})\right).\hfill \end{array}$$

Hence,

$$\begin{array}{ccc}\hfill & & {\mathbb{P}}^{*}\left(I_n>ϵ\right)\hfill \\ & =& {\mathbb{P}}^{*}\left({\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}{∥\sum _{(i_1,\dots ,i_m)\in I_{S_n^{*}}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})∥}_{{\mathcal{F}}_{\delta }}>ϵ\right)\hfill \\ & =& {\mathbb{P}}^{*}\left(\parallel {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_{S_n^{*}}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})>ϵ\right.\hfill \\ & & \left.\cap \left(S_n^{*}\le K\sqrt{n}\right){\parallel }_{{\mathcal{F}}_{\delta }}\right)+{\mathbb{P}}^{*}\left(S_n^{*}>K\sqrt{n}\right)\hfill \\ & \le & {\mathbb{P}}^{*}\left(\underset{M\le K\sqrt{n}}{max}\parallel {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}^{{}^{\prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}){\parallel }_{{\mathcal{F}}_{\delta }}>ϵ\right)\hfill \\ & & +{\mathbb{P}}^{*}\left(S_n^{*}>K\sqrt{n}\right),\mathrm{for}\mathrm{any}K>0,\hfill \\ & & \mathrm{where}{\mathfrak{I}}^{{}^{\prime }}:=\left\{(i_1,\dots ,i_m):1\le i_1<\dots <i_m\le S_n^{*},\exists \ell =1,\dots ,m,S_n^{*}<i_{\ell }\le M,\right.\hfill \\ & & \left.i_j\ne i_k\mathrm{for}j\ne k\right\}.\hfill \\ & \le & C_1{\mathbb{P}}^{*}\left(\parallel {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}){\parallel }_{{\mathcal{F}}_{\delta }}>C_2ϵ\right)\hfill \\ & & +{\mathbb{P}}^{*}\left(S_n^{*}>K\sqrt{n}\right),\mathrm{for}\mathrm{any}K>0,\hfill \end{array}$$

(33)

where

$$\begin{array}{ccc}& & {\mathfrak{I}}_m^{{}^{\prime \prime }}:=\left\{(i_1,\dots ,i_m):1\le i_1<\dots <i_m\le S_n^{*},\exists \ell =1,\dots ,m,\right.\hfill \\ & & \left.S_n^{*}<i_{\ell }\le K\sqrt{n},i_j\ne i_k\mathrm{for}j\ne k\right\}.\hfill \end{array}$$

For n large enough, we need to show that there exists $K>0$ such that

$${\mathbb{P}}^{*}\left(S_n^{*}>K\sqrt{n}\right)\to 0.$$

As ${\mathbb{1}}_{{\widehat{\mathbf{B}}}_i\in {\mathcal{A}}_n^c}$ are i.i.d and bounded,

$$\frac{S_n^{*}-\mathbb{E}\left(S_n^{*}\right)}{\sqrt{\mathbb{E}\left(l_n\right)}}\to N(0,{\eta }^2)\mathrm{in}\mathrm{probability}.$$

therefore, we can find $\mathcal{M}>0$ such that

$${\mathbb{P}}^{*}\left(S_n^{*}>\mathbb{E}\left(S_n^{*}\right)+\mathcal{M}\sqrt{n}\right)<ϵ.$$

However,

$$\mathbb{E}\left(S_n^{*}\right)=\mathbb{E}\left(l_n\right){\mathbb{P}}^{*}({\widehat{\mathbf{B}}}_i^{*}\in {\mathcal{A}}_n^c)=\mathbb{E}\left(l_n\right)-l_n=O_{\mathbb{P}}\left(\sqrt{n}\right),$$

by Lemma 1 i), then

$${\mathbb{P}}^{*}\left(S_n^{*}>K\sqrt{n}\right)\to 0.$$

Then, we only need to estimate the first part in (33). Define the following bootstrap procedure: let ${\overrightarrow{\mathbf{B}}}_i:=\left\{X_{{\mathcal{T}}_{i-1}+1},\cdots ,X_{{\mathcal{T}}_i},0,0,\dots \right\}$ and let $\overrightarrow{\mathcal{F}}$ be a class of function, related to the class of functions $\mathcal{F}$, such that, for every ${\overrightarrow{\omega }}_h\in \overrightarrow{\mathcal{F}}$:

$$\left\{\begin{array}{cc}{\displaystyle {\overrightarrow{\omega }}_h({\overrightarrow{\mathbf{B}}}_1,{\overrightarrow{\mathbf{B}}}_2,\cdots ,{\overrightarrow{\mathbf{B}}}_k)=\sum _{i_1=1}^{\infty }\dots \sum _{i_k=1}^{\infty }h(x_{i_1},\dots ,x_{i_k}){\mathbb{1}}_{x_k\ne 0}}\hfill & \hfill \mathrm{if} \mathrm{defined},\\ \infty \hfill & \hfill \mathrm{otherwise}.\end{array}\right.$$

(34)

It is classical that $\left\{{\overrightarrow{B}}_i\right\}$ are i.i.d., applying the same bootstrap method of Algorithm 1. This new sample allows us to enlarge and bound (33) by

$${\mathbb{P}}^{*}\left(\underset{\overrightarrow{h}\in \overrightarrow{\mathcal{H}}}{sup}|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\overrightarrow{\omega }}_{\overrightarrow{h}}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})|>ϵ\right),$$

(35)

where $\overrightarrow{h}\in \overrightarrow{\mathcal{H}}=\{{\overrightarrow{\omega }}_{\overrightarrow{f}}-{\overrightarrow{\omega }}_{\overrightarrow{g}},\overrightarrow{f},\overrightarrow{g}\in \overrightarrow{\mathcal{F}}\}$ and the corresponding class $\mathcal{H}=\{{\omega }_f-{\omega }_g,f,g\in \mathcal{F}\}$, with envelope $\tilde{F}$ and F, respectively. To estimate the last expression, we use bracketing. Define the bracket $[f^{\ell },f^u]$ by:

$$[f^{\ell },f^u]:=\{f\in \mathcal{F}:f^{\ell }⩽f⩽f^u\},$$

and the bracketing entropy number by $N_1(\gamma ,\mathcal{F},\mathbb{P})$, which denotes the minimal number $N\ge 1$ for which there exist functions $f_1^{\ell },\cdots ,f_N^{\ell }$ and $f_1^u,\cdots ,f_N^u$ such that:

$$\left\{\begin{array}{c}{\displaystyle \mathcal{F}\subset \bigcup _{k=1}^N\left[f_k^{\ell },f_k^u\right],}\hfill \\ \\ {\displaystyle {\int }_S\left(f_k^u-f_k^{\ell }\right)\mathbb{P}\le \gamma .}\hfill \end{array}\right.$$

(36)

For the class of functions $\mathcal{H}$, consider the bracket $[h^{\ell },h^u]$, such that ${\mathbb{E}}^{*}(h^{\ell },h^u)⩽\gamma$, where $\gamma >0$ and it is determined later. In this framework, the bracketing entropy number is $N^{*}\left(\gamma \right):=N_1(\gamma ,\mathcal{H},{\mathbb{D}}^{*})$, for

$${\mathbb{D}}^{*}={\left(\genfrac{}{}{0pt}{}{l_n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}{\xi }_{i_1,l_n}\dots {\xi }_{i_m,l_n}{\delta }_{({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})}.$$

Hence, we have the following inequalities

$$\begin{array}{ccc}\hfill & & \underset{\overrightarrow{h}\in \overrightarrow{\mathcal{H}}}{sup}|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\overrightarrow{\omega }}_{\overrightarrow{h}}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})|\hfill \\ & \le & \underset{k\le N^{*}\left(\gamma \right)}{max}|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\left(h_k^u-h_k^{\ell }\right)({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})|\hfill \\ & \le & \underset{k\le N^{*}\left(\gamma \right)}{max}|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\hfill \\ & & \left[h_k^u({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})-{\mathbb{E}}^{*}\left(h_k^u({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)\right]|\hfill \\ & & +\underset{k\le N^{*}\left(\gamma \right)}{max}|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\hfill \\ & & \left[h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})-{\mathbb{E}}^{*}\left(h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)\right]|\hfill \\ & & +\underset{k\le N^{*}\left(\gamma \right)}{max}|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\hfill \\ & & \left[{\mathbb{E}}^{*}\left(h_k^u({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)-{\mathbb{E}}^{*}\left(h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)\right]|\hfill \\ & :=& I_A+I_B+I_C.\hfill \end{array}$$

(37)

Treating each term, keeping in mind Condition (A.1), i.e., ${\sum }_{i=1}^n{\xi }_i=n$, we have

$$\begin{array}{c}{\displaystyle I_C:=\underset{1\le k\le N^{*}\left(\gamma \right)}{max}|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)}\hfill \\ \left[{\mathbb{E}}^{*}\left(h_k^u({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)-{\mathbb{E}}^{*}\left(h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)\right]|\hfill \\ \le \gamma |{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)|\hfill \\ =\gamma \left|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1}\sum _{{\mathbf{i}}_{\{1,\dots ,m-1\}}}\sum _{j=1}^m\prod _{k=1,k\ne j}^m{\xi }_{i_k,n}\left\{\sum _{i_j=1}^n({\xi }_{i_j,n}-1)\right\}\right|=0,\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill & & \mathbb{P}(I_B>ϵ):=\mathbb{P}\left(\underset{k\le N^{*}\left(\gamma \right)}{max}\right|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\hfill \\ & & \left[h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})-{\mathbb{E}}^{*}\left(h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)\right]|>ϵ)\hfill \\ & \le & N^{*}\left(\gamma \right)\underset{k\le N^{*}\left(\gamma \right)}{max}\mathbb{P}\left(\right|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\hfill \\ & & \left[h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})-{\mathbb{E}}^{*}\left(h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)\right]|>ϵ)\hfill \\ & & (\mathrm{For}{h}_k^{\ell }=h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }\le M_n}+h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }>M_n})\hfill \\ & \le & N^{*}\left(\gamma \right)\underset{k\le N^{*}\left(\gamma \right)}{max}\mathbb{P}\left(\right|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\hfill \\ & & \left[h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }\le M_n}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})-{\mathbb{E}}^{*}\left(h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }\le M_n}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)\right]|>ϵ)\hfill \\ & & +N^{*}\left(\gamma \right)\underset{k\le N^{*}\left(\gamma \right)}{max}\mathbb{P}\left(\right|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\hfill \\ & & \left[h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }>M_n}({\overrightarrow{B}}_{i_1},\dots ,{\overrightarrow{B}}_{i_m})-{\mathbb{E}}^{*}\left(h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }>M_n}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)\right]|>ϵ)\hfill \\ & \le & \frac{N^{*}\left(\gamma \right)}{{ϵ}^2{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}}{\mathbb{E}}^{*}{\left(({\xi }_{1,l_n}-1)\dots ({\xi }_{m,l_n}-1)\left[h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }\le M_n}-{\mathbb{E}}^{*}\left(h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }\le M_n}\right)\right]\right)}^2\hfill \\ & & +\frac{N^{*}\left(\gamma \right)}{ϵ}{\mathbb{E}}^{*}\left|({\xi }_{1,l_n}-1)\dots ({\xi }_{m,l_n}-1)\left[h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }>M_n}-{\mathbb{E}}^{*}\left(h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }>M_n}\right)\right]\right|\hfill \\ & \le & \frac{N^{*}\left(\gamma \right)}{{ϵ}^2{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}}{\mathbb{E}}_{\xi }\left(\prod _{i=1}^m{({\xi }_{i,l_n}-1)}^2\right){\mathbb{E}}^{*}{\left(\left[h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }\le M_n}-{\mathbb{E}}^{*}\left(h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }\le M_n}\right)\right]\right)}^2\hfill \\ & & +\frac{N^{*}\left(\gamma \right)}{ϵ}{\mathbb{E}}^{*}\left|({\xi }_{1,l_n}-1)\dots ({\xi }_{m,l_n}-1)\left[h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }>M_n}-{\mathbb{E}}^{*}\left(h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }>M_n}\right)\right]\right|\hfill \\ & \le & \frac{N^{*}\left(\gamma \right)}{{ϵ}^2{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}}{c}^2\times 4M_n^2+\frac{N^{*}\left(\gamma \right)}{ϵ}{\mathbb{E}}^{*}\left|({\xi }_{1,l_n}-1)\dots ({\xi }_{m,l_n}-1)h_k^{\ell }{\mathbb{1}}_{h_k^{\ell }>M_n}\right|\hfill \\ & \le & \frac{N^{*}\left(\gamma \right)}{{ϵ}^2{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}}{c}^2\times 4M_n^2+\frac{2N^{*}\left(\gamma \right)}{ϵ}{\mathbb{E}}^{*}\left|({\xi }_{1,l_n}-1)\dots ({\xi }_{m,l_n}-1)\tilde{F}{\mathbb{1}}_{\tilde{F}>M_n}\right|\hfill \\ & \le & \frac{N^{*}\left(\gamma \right)}{{ϵ}^2{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}}{c}^2\times 4M_n^2\hfill \\ & & +\frac{4N^{*}\left(\gamma \right)}{ϵ}{\left(\genfrac{}{}{0pt}{}{l_n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\left|\tilde{F}{\mathbb{1}}_{\tilde{F}>M_n}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right|,\hfill \end{array}$$

(38)

yet, ${\overrightarrow{\mathbf{B}}}_i$ are i.i.d. and and $\mathbb{E}\left(\tilde{F}\right)=\mathbb{E}{\left(\tau \right)}^m\mathbb{E}F<\infty$, so for any $M_n\nearrow \infty$, we have

$${\left(\genfrac{}{}{0pt}{}{\mathbb{E}\left(l_n\right)}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\tilde{F}{\mathbb{1}}_{\tilde{F}>M_n}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\to 0a.s.$$

Using the same argument as in part iii) of Lemma 1, we can prove that

$${\left(\genfrac{}{}{0pt}{}{l_n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\left|\overrightarrow{F}{\mathbb{1}}_{\overrightarrow{F}>M_n}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right|\to 0inprobability.$$

Then, it remains to find that, for every fixed $\gamma >0$, $N^{*}\left(\gamma \right)$ is bounded in probability, as the last expression in (38) does not depend on k. It is interesting to note that $N_1(\gamma ,\overrightarrow{\mathcal{H}},\mathbb{P})$ is finite, due to the boundness of $\overrightarrow{\mathcal{H}}$ by $2F$ with $\mathbb{E}\overrightarrow{F}\left(\overrightarrow{\mathbf{B}}\right)<\infty$ and the fact that ${\overrightarrow{\mathbf{B}}}_i$ are i.i.d. and discrete random variables. Under the norm $L_1\left(\mathbb{P}\right)$, define $\gamma /2$ brackets $h_1^{\ell },\cdots h_{N(\gamma /2)}^{\ell }$ and $h_1^u,\cdots ,h_{N(\gamma /2)}^u$. Observe that

$$\begin{array}{c}\hfill \underset{j\le N(\gamma /2)}{max}|{\left(\genfrac{}{}{0pt}{}{l_n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in I_{l_n}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\left(h_j^u-h_j^{\ell }\right)({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})|,\end{array}$$

(39)

converges to zero in probability, and $N(\gamma /2)$ does not depend on n. That implies that $N^{*}\left(\gamma \right)\le N(\gamma /2)<\infty$ in probability. Replacing $h^{\ell }$ by $h^u$, $I_A$ is identical to $I_B$, i.e., $I_A$ also converges to zero in probability. This proves the convergence of $I_n$ to zero in probability.

For $I{I}_n$: In the same manner, let

$$S_n^{*}:=\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m}{𝟙}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n^c}.$$

Define a new bootstrap sample $\left\{{\mathbf{B}}_i^{**}\right\}$ in $i=l_n+1,\dots ,\mathbb{E}\left(l_n\right)$. Clearly, the new sample is well-defined since we assumed at the beginning that $l_n\le \mathbb{E}\left(l_n\right)$, and it is defined independently from ${\mathbf{B}}_i^{*}$ and ${\widehat{\mathbf{B}}}_i^{*}$. In this case:

$$\begin{array}{c}{\displaystyle \mathcal{L}\left(\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m}){𝟙}_{({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})\in {\mathcal{A}}_n^c}\right)}\hfill \\ =\mathcal{L}\left(\sum _{(i_1,\dots ,i_m)\in I_{S_n^{*}}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1}^{**},\dots ,{\mathbf{B}}_{i_m}^{**})\right).\hfill \end{array}$$

Hence, as in (33), we have:

$$\begin{array}{c}\hfill {\displaystyle {\mathbb{P}}^{*}\left(I{I}_n>ϵ\right)}\\ & =& {\mathbb{P}}^{**}\left({\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}{∥\sum _{(i_1,\dots ,i_m)\in I_{S_n^{*}}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m})∥}_{{\mathcal{F}}_{\delta }}>ϵ\right)\hfill \\ & \le & C_1{\mathbb{P}}^{**}\left(\parallel {\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\omega }_{\tilde{h}}({\mathbf{B}}_{i_1},\dots ,{\mathbf{B}}_{i_m}){\parallel }_{{\mathcal{F}}_{\delta }}>C_2ϵ\right)\hfill \\ & & +{\mathbb{P}}^{*}\left(S_n^{*}>K\sqrt{n}\right),\mathrm{for}\mathrm{any}K>0,\hfill \end{array}$$

(40)

where

$$\begin{array}{ccc}& & {\mathfrak{I}}_m^{{}^{\prime \prime }}:=\left\{(i_1,\dots ,i_m):1\le i_1<\dots <i_m\le S_n^{*},\exists \ell =1,\dots ,m,\right.\hfill \\ & & \left.S_n^{*}<i_{\ell }\le K\sqrt{n},i_j\ne i_{k}forj\ne k\right\}.\hfill \end{array}$$

Using the same bootstrap procedure defined previously for $I_n$, let

$${\overrightarrow{\mathbf{B}}}_i:=\left\{X_{{\mathcal{T}}_{i-1}+1},\cdots ,X_{{\mathcal{T}}_i},0,0,\dots \right\},$$

for $i=l_n+1,\dots ,\mathbb{E}\left(l_n\right)$, and let $\overrightarrow{\mathcal{F}}$ be a class of function such that, for every ${\overrightarrow{\omega }}_h\in \overrightarrow{\mathcal{F}}$:

$$\left\{\begin{array}{cc}{\displaystyle {\overrightarrow{\omega }}_h({\overrightarrow{\mathbf{B}}}_1,{\overrightarrow{\mathbf{B}}}_2,\cdots ,{\overrightarrow{\mathbf{B}}}_k)=\sum _{i_1=1}^{\infty }\dots \sum _{i_k=1}^{\infty }h(x_{i_1},\dots ,x_{i_k}){\mathbb{1}}_{x_k\ne 0}}\hfill & \hfill \mathrm{if} \mathrm{defined}\\ \infty \hfill & \hfill \mathrm{otherwise}.\end{array}\right.$$

(41)

It is classical that $\left\{{\overrightarrow{\mathbf{B}}}_i\right\}$ are i.i.d., applying the same bootstrap method of Algorithm 1. This new sample allows us to enlarge and bound (33) by

$${\mathbb{P}}^{**}\left(\underset{\overrightarrow{h}\in \overrightarrow{\mathcal{H}}}{sup}|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1){\overrightarrow{\omega }}_{\overrightarrow{h}}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})|>ϵ\right),$$

(42)

where

$$\overrightarrow{h}\in \overrightarrow{\mathcal{H}}=\{{\overrightarrow{\omega }}_{\overrightarrow{f}}-{\overrightarrow{\omega }}_{\overrightarrow{g}},\overrightarrow{f},\overrightarrow{g}\in \overrightarrow{\mathcal{F}}\}$$

corresponding to the class

$$\mathcal{H}=\{{\omega }_f-{\omega }_g,f,g\in \mathcal{F}\},$$

with envelope $\overrightarrow{F}$ and F, respectively. As before, for the class of functions $\mathcal{H}$, consider the bracket $[h^{\ell },h^u]$, such that

$${\mathbb{E}}^{**}(h^{\ell },h^u)⩽\gamma ,$$

where $\gamma >0$ and it is determined later. In this framework, the bracketing entropy number is $N^{**}\left(\gamma \right):=N_1(\gamma ,\mathcal{H},{\mathbb{D}}^{**})$, for

$${\mathbb{D}}^{**}={\left(\genfrac{}{}{0pt}{}{\mathbb{E}\left(l_n\right)-l_n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)-l_n}^m}{\xi }_{i_1,l_n}\dots {\xi }_{i_m,l_n}{\delta }_{({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})}.$$

Following the same arguments from Equations (37) through (38), we can find that (42) is

$$\begin{array}{ccc}\hfill & \le & N^{**}\left(\gamma \right)\underset{k\le N^{**}\left(\gamma \right)}{max}\mathbb{P}\left(\right|{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{-1/2}\sum _{(i_1,\dots ,i_m)\in {\mathfrak{I}}_m^{{}^{\prime \prime }}}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\hfill \\ & & \left[h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})-{\mathbb{E}}^{**}\left(h_k^{\ell }({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right)\right]|>ϵ)\hfill \\ & \le & \frac{N^{**}\left(\gamma \right)}{{ϵ}^2{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}}{c}^2\times 4M_n^2+\frac{2N^{**}\left(\gamma \right)}{ϵ}{\mathbb{E}}^{**}\left|({\xi }_{1,l_n}-1)\dots ({\xi }_{m,l_n}-1)\tilde{F}{\mathbb{1}}_{\tilde{F}>M_n}\right|\hfill \\ & \le & \frac{N^{**}\left(\gamma \right)}{{ϵ}^2{\left(\genfrac{}{}{0pt}{}{n}{m}\right)}^{1/2}}{c}^2\times 4M_n^2+\frac{4N^{**}\left(\gamma \right)}{ϵ}\hfill \\ & & {\left(\genfrac{}{}{0pt}{}{\mathbb{E}\left(l_n\right)-l_n}{m}\right)}^{-1}\sum _{(i_1,\dots ,i_m)\in I_{\mathbb{E}\left(l_n\right)-l_n}^m}({\xi }_{i_1,l_n}-1)\dots ({\xi }_{i_m,l_n}-1)\left|\tilde{F}{\mathbb{1}}_{\tilde{F}>M_n}({\overrightarrow{\mathbf{B}}}_{i_1},\dots ,{\overrightarrow{\mathbf{B}}}_{i_m})\right|.\hfill \end{array}$$

(43)

Here, we must pay attention to the randomness of $N^{**}$ which depends on n. According to Lemma 1 i), we can see that $|\mathbb{E}\left(l_n\right)-l_n|\to \infty$ in probability, under the assumption that $l_n<\mathbb{E}\left(l_n\right)$. Now, using the same treatment of $I_n$, and for $M_n:=n^{1/3}$ ( to provide the convergence of $M_n$ to ∞), as in [47], this allows the convergence of (43) to zero in probability. Estimating now $N^{**}$ by considering the same $\gamma /2$ brackets $h_1^{\ell },\cdots h_{N(\gamma /2)}^{\ell }$ and $h_1^u,\cdots ,h_{N(\gamma /2)}^u$, we have $N^{**}\left(\gamma \right)<N(\gamma /2)$, which does not depend on n. Then, $I{I}_n$ is proved. Following the same footsteps, we can prove the case where $l_n>\mathbb{E}\left(l_n\right)$. This proves Step 3. □

The end of the previous step yields that we only need to show the stochastic equicontinuity of ${\widehat{T}}_{l_n}^{*}$, where the number of blocks is replaced by the deterministic quantity $\mathbb{E}\left(l_n\right)$. In order to achieve the equicontinuity of this statistic, Lemma 2 shows that it is sufficient to prove that:

$$\begin{array}{cc}\hfill & \mathbb{E}{∥\sum _{\begin{array}{c}1\le i_k\le {\ell }_k,1\le k\le m\left(n\right)\end{array}}{ϵ}_{i_1}^{\left(1\right)}\cdots {ϵ}_{i_m}^{\left(m\right)}{\omega }_{\overrightarrow{h}}\left({\widehat{\mathbf{B}}}_{i_1}^{\left(1\right)},\dots ,{\widehat{\mathbf{B}}}_{i_m}^{\left(m\right)}\right)∥}_{\overrightarrow{\mathcal{H}}}\hfill \\ \hfill & \le a({\ell }_1,\dots ,{\ell }_m){\left(\prod _{k=1}^m{\ell }_k\right)}^{1/2}\hfill \end{array}$$

for all $1\le {\ell }_1,\dots ,{\ell }_m\le n$. We begin to define the distance:

$$e_{\ell }^2(f,g)\equiv \frac{1}{{\prod }_{k=1}^m{\ell }_k}\sum _{1\le i_k\le {\ell }_k,1\le k\le m}{\omega }_{\overrightarrow{h}}^2\left({\widehat{\mathbf{B}}}_{i_1}^{\left(1\right)},\dots ,{\widehat{\mathbf{B}}}_{i_m}^{\left(m\right)}\right),$$

defined in $L_2$, associated with the Rademacher process

$$\left\{\frac{1}{{\left({\prod }_{k=1}^m{\ell }_k\right)}^{1/2}}\sum _{1\le i_k\le {\ell }_k,1\le k\le m}{\epsilon }_{i_1}^{\left(1\right)}\cdots {\epsilon }_{i_m}^{\left(m\right)}{\omega }_{\overrightarrow{h}}\left({\widehat{\mathbf{B}}}_{i_1}^{\left(1\right)},\dots ,{\widehat{\mathbf{B}}}_{i_m}^{\left(m\right)}\right):\overrightarrow{h}\in \overrightarrow{\mathcal{H}}\mid {\widehat{\mathbf{B}}}_1,\dots ,{\widehat{\mathbf{B}}}_m\right\}.$$

Take ${\parallel f\parallel }_{\ell }^2\equiv e_{\ell }^2(f,0)$ and

$$r_{\ell }\left(\delta \right)\equiv \underset{f\in {\overrightarrow{\mathcal{F}}}_{\delta }}{sup}{\parallel f\parallel }_{\ell }^2.$$

Using Corollary A1, we have

$$\begin{array}{c}\hfill {\displaystyle \mathbb{E}{∥\frac{1}{{\left({\prod }_{k=1}^m{\ell }_k\right)}^{1/2}}\sum _{1\le i_k\le {\ell }_k,1\le k\le m}{\epsilon }_{i_1}^{\left(1\right)}\cdots {\epsilon }_{i_m}^{\left(m\right)}{\omega }_{\overrightarrow{h}}\left({\widehat{\mathbf{B}}}_{i_1}^{\left(1\right)},\dots ,{\widehat{\mathbf{B}}}_{i_m}^{\left(m\right)}\right)∥}_{\overrightarrow{\mathcal{H}}}}\\ \le C{\int }_0^{r_{\ell }\left(\delta \right)}{\left(log\mathcal{N}\left(\epsilon ,\mathcal{F},e_{\ell }\right)\right)}^{m/2}\mathrm{d}\epsilon \hfill \\ ={C\parallel F\parallel }_{\ell }·{\int }_0^{r_{\ell }\left(\delta \right)/{\parallel F\parallel }_{\ell }}{\left(log\mathcal{N}\left({\epsilon \parallel F\parallel }_{\ell },\mathcal{F},e_{\ell }\right)\right)}^{m/2}\mathrm{d}\epsilon \hfill \\ \le {C\parallel F\parallel }_{\ell }·{\int }_0^{r_{\ell }\left(\delta \right)/{\parallel F\parallel }_{\ell }}{\left(\underset{Q}{sup}log\mathcal{N}\left({\epsilon \parallel F\parallel }_{L_2\left(Q\right)},\mathcal{F},L_2\left(Q\right)\right)\right)}^{m/2}\mathrm{d}\epsilon .\hfill \end{array}$$

(44)

Assuming that $F\ge 1$, the upper bound in the integral can be replaced by $r_{\ell }\left(\delta \right)$. The following proposition is necessary for the following.

Proposition 3

([46]). Let $\left\{X_i\right\}$ be i.i.d. random variables with law $\mathbb{P}$. Let $\mathcal{H}$ be a class of measurable real-valued functions defined on $\left({\mathcal{X}}^m,{\mathcal{A}}^m\right)$ with an ${\mathbb{P}}^m$-integrable envelope such that the following holds: for any fixed $\delta >$$0,M>0,1\le k\le m$,

$$\underset{1\le j^{\prime }\le k}{max}\mathbb{E}{\left(\frac{log\mathcal{N}\left(\delta ,{\left({\pi }_k\mathcal{H}\right)}_M,e_{\ell ,j^{\prime }}\right)}{{\ell }_{j^{\prime }}}\right)}^{1/2}\to 0$$

(45)

holds for any ${\ell }_1\wedge \cdots \wedge {\ell }_k\to \infty$. Here for $\ell =\left({\ell }_1,\dots ,{\ell }_k\right)$ and ${\left\{X_i\right\}}_{i=1}^{\infty }$,

$$e_{\ell ,j^{\prime }}(f,g)\equiv \frac{1}{{\ell }_{j^{\prime }}}\sum _{i_{j^{\prime }}=1}^{{\ell }_{j^{\prime }}}\left|\frac{1}{{\prod }_{j\ne j^{\prime }}{\ell }_j}\sum _{1\le i_j\le {\ell }_j:j\ne j^{\prime }}(f-g)\left(X_{i_1},\dots ,X_{i_k}\right)\right|$$

and

$${\left({\pi }_k\mathcal{H}\right)}_M\equiv \left\{h{1}_{H_k\le M}:h\in {\pi }_k\mathcal{H}\right\},$$

where $H_k$ is an envelope for ${\pi }_k\mathcal{H}$. Then,

$$\underset{h\in \mathcal{H}}{sup}\left|\frac{1}{{\prod }_{k=1}^m{\ell }_k}\sum _{1\le i_k\le {\ell }_k,1\le k\le m}\left(h\left(X_{i_1},\dots ,X_{i_m}\right)-{\mathbb{P}}^{m}h\right)\right|\to 0$$

in $L_1$ as ${\ell }_1\wedge \dots \wedge {\ell }_m\to \infty$. The above equation can be replaced by the decoupled version.

By this proposition, ${\parallel F\parallel }_{\ell }{\to }_{\mathbb{P}}{\parallel F\parallel }_{L_2\left(P\right)}$ as ${\ell }_1\wedge \dots \wedge {\ell }_m\to \infty$, therefore, it suffices to get $r_{\ell }\left(\delta \right){\to }_{p}0$ as ${\ell }_1\wedge \dots \wedge {\ell }_m\to \infty$ and $\delta \to 0$. It is obvious that all that is left to do now is to demonstrate that

$$\underset{f\in {\tilde{\mathcal{F}}}_{\delta }}{sup}\left|\frac{1}{{\prod }_{k=1}^m{\ell }_k}\sum _{1\le i_k\le {\ell }_k,1\le k\le m}\left({\omega }_{\tilde{h}}^2\left({\widehat{\mathbf{B}}}_{i_1}^{\left(1\right)},\dots ,{\widehat{\mathbf{B}}}_{i_m}^{\left(m\right)}\right)-{\mathbb{P}}^m{\omega }_{\tilde{h}}^2\right)\right|{\to }_{p}0.$$

(46)

Verifying condition (45)

$$\begin{array}{cc}\hfill & \underset{1\le j^{\prime }\le k}{max}\mathbb{E}{\left(\frac{log\mathcal{N}\left(\delta ,{\mathcal{F}}_M^2,e_{\ell ,j^{\prime }}\right)}{{\ell }_{j^{\prime }}}\right)}^{1/2}\hfill \\ \hfill & \le {\left({\ell }_1\wedge \cdots \wedge {\ell }_k\right)}^{-1/2}\mathbb{E}\left[{\int }_0^{\delta }{\left(log\mathcal{N}\left(\epsilon ,{\mathcal{F}}_M^2,e_{\ell ,j^{\prime }}\right)\right)}^{m/2}\mathrm{d}\epsilon \right]\hfill \\ \hfill & \le {\left(\frac{\delta }{\sqrt{2}M}\right)}^{-1}{\left({\ell }_1\wedge \cdots \wedge {\ell }_k\right)}^{-1/2}\mathbb{E}\left[{\int }_0^{\delta /\sqrt{2}M}{\left(log\mathcal{N}\left(\epsilon ,{\mathcal{F}}_M^2,e_{\ell ,j^{\prime }}\right)\right)}^{m/2}\mathrm{d}\epsilon \right]\hfill \\ \hfill & \le {(\delta /2M)}^{-1}{\left({\ell }_1\wedge \cdots \wedge {\ell }_k\right)}^{-1/2}{\int }_0^1{\left(\underset{Q}{sup}log\mathcal{N}\left({\epsilon \parallel F\parallel }_{L_2\left(Q\right)},\mathcal{F},L_2\left(Q\right)\right)\right)}^{m/2}\mathrm{d}\epsilon \hfill \\ & \times {\parallel F\parallel }_{L_2\left({\mathbb{P}}^m\right)}\to 0.\hfill \end{array}$$

(47)

The shift from the second to the third line is true because

$$\mathcal{N}\left(\delta ,{\mathcal{F}}_M^2,L_2\left(\mathbb{Q}\right)\right)\le \mathcal{N}\left(\delta /\sqrt{2}M,{\mathcal{F}}_M,L_2\left(\mathbb{Q}\right)\right).$$

As the condition is verified, as well as ${\ell }_1\wedge \cdots \wedge {\ell }_m\to \infty$, (46) follows directly using the previous proposition. Hence, there exists some sequence $\left\{a_{\ell }\right\}$, in a way that $a_{\ell }\to 0$ for any sequence $\left\{{\delta }_{\ell }\right\}$ with ${\delta }_{\ell }\to 0$ both under ${\ell }_1\wedge \cdots \wedge {\ell }_m\to \infty$, such that:

$$\begin{array}{ccc}& & \mathbb{E}{∥\sum _{1\le i_k\le {\ell }_k,1\le k\le m}{\epsilon }_{i_1}^{\left(1\right)}\cdots {\epsilon }_{i_m}^{\left(m\right)}{\omega }_{\tilde{h}}\left({\widehat{\mathbf{B}}}_{i_1}^{\left(1\right)},\dots ,{\widehat{\mathbf{B}}}_{i_m}^{\left(m\right)}\right)∥}_{\tilde{\mathcal{H}}}\le a_{\ell }{\left(\prod _{k=1}^m{\ell }_k\right)}^{1/2}.\hfill \end{array}$$

(48)

An application of Lemma 2 proves that

$$\begin{array}{c}\hfill n^{-m/2}\mathbb{E}{∥\sum _{1\le i_1,\dots ,i_m\le n}({\xi }_{i_1}-1)\cdots ({\xi }_{i_m}-1){\omega }_{\tilde{h}}({\widehat{\mathbf{B}}}_{i_1},\dots ,{\widehat{\mathbf{B}}}_{i_m})∥}_{{\mathcal{F}}_{{\delta }_n}}\to 0,n\to \infty .\end{array}$$

This completes the proof for the asymptotic equicontinuity.