Uniform in Number of Neighbor Consistency and Weak Convergence of k-Nearest Neighbor Single Index Conditional Processes and k-Nearest Neighbor Single Index Conditional U-Processes Involving Functional Mixing Data

Abstract

U-statistics are fundamental in modeling statistical measures that involve responses from multiple subjects. They generalize the concept of the empirical mean of a random variable X to include summations over each m-tuple of distinct observations of X. W. Stute introduced conditional U-statistics, extending the Nadaraya–Watson estimates for regression functions. Stute demonstrated their strong pointwise consistency with the conditional expectation $r^{\left(m\right)}(\phi ,\mathbf{t})$, defined as $\mathbb{E}\left[\phi (Y_1,\dots ,Y_m)\right|(X_1,\dots ,X_m)=\mathbf{t}]$ for $\mathbf{t}\in {\mathcal{X}}^m$. This paper focuses on estimating functional single index (FSI) conditional U-processes for regular time series data. We propose a novel, automatic, and location-adaptive procedure for estimating these processes based on k-Nearest Neighbor (kNN) principles. Our asymptotic analysis includes data-driven neighbor selection, making the method highly practical. The local nature of the kNN approach improves predictive power compared to traditional kernel estimates. Additionally, we establish new uniform results in bandwidth selection for kernel estimates in FSI conditional U-processes, including almost complete convergence rates and weak convergence under general conditions. These results apply to both bounded and unbounded function classes, satisfying certain moment conditions, and are proven under standard Vapnik–Chervonenkis structural conditions and mild model assumptions. Furthermore, we demonstrate uniform consistency for the nonparametric inverse probability of censoring weighted (I.P.C.W.) estimators of the regression function under random censorship. This result is independently valuable and has potential applications in areas such as set-indexed conditional U-statistics, the Kendall rank correlation coefficient, and discrimination problems.

1. Introduction and Motivations

U-statistics, first introduced in [1] and building on earlier work in [2], represent an essential class of statistical tools serving as unbiased estimators. Specifically, U-statistics of order m with kernel h are constructed from a sequence of random variables ${\left\{X_i\right\}}_{i=1}^{\infty }$ defined on a measurable space $(S,\mathfrak{S})$, using a measurable function $f:S^m\to \mathbb{R}$. They are given by

$$U_n\left(h\right)={\displaystyle \frac{(n-m)!}{n!}}\sum _{\left(i_1,\dots ,i_m\right)\in I_n^m}h(X_{i_1},\dots ,X_{i_m}),n\ge m,$$

where

$$I_n^m=\left\{\left(i_1,\dots ,i_m\right):i_j\in \mathbb{N},1\le i_j\le n,i_j\ne i_k\mathrm{if}j\ne k\right\}.$$

Notice that $U_n\left(h\right)$ serves as the nonparametric uniformly minimum variance estimator of $\mathit{\theta }=\mathbb{E}\left[h(X_1,\dots ,X_m)\right]$, minimizing the following expression:

$$\sum _{1\le i_1<\cdots <i_m\le n}{\left(h(X_{i_1},\dots ,X_{i_m})-\alpha \right)}^2,$$

where $\alpha$ is the minimizer. Common estimators based on U-statistics include the empirical variance, Gini’s mean difference, and Kendall’s rank correlation coefficient. A notable example is the Wilcoxon signed-rank test, a classical nonparametric method for testing zero location, as discussed in [3], Example 12.4. Asymptotic properties of U-statistics for independent and identically distributed (i.i.d.) random variables were initially developed in [1] and subsequently advanced in [4,5,6,7]. Parallel results for V-statistics were presented in [8,9]. Comprehensive reviews of U-statistics literature can be found in works [5,10,11], with a more extensive discussion in [12]. U-processes, a generalization of U-statistics where the statistics are indexed by a family of kernels, extend U-statistics to infinite-dimensional settings. These processes serve as nonlinear extensions of empirical processes and are crucial for tackling complex problems in areas such as density estimation, nonparametric regression, and goodness-of-fit testing. Transitioning from empirical processes to U-processes involves specialized techniques, particularly in stationary contexts, and finds wide applications in estimator analysis, particularly for functions of varying smoothness. Notable applications include testing qualitative features in nonparametric statistics [13,14], cross-validation in density estimation [15], and deriving limiting distributions for M-estimators [10,16,17]. Ref. [10] provided necessary and sufficient conditions for the law of large numbers and the central limit theorem for U-processes. More recent applications include normality tests using U-processes [18] and the new normality tests proposed in [19], which utilize weighted $L_1$ distances between the standard normal density and local U-statistics derived from standardized observations. Using a characterization of symmetry in terms of extremal order statistics, the authors of [20] developed several new nonparametric tests of symmetry based on novel U-empirical processes. In a related work, ref. [21] considered a class of U-statistics with kernels indexed by a multivariate parameter and, as an application, discussed an empirical characteristic function-based transformation to symmetry. The median-of-means approach, which leverages U-statistics, was introduced in [22] for estimating the mean of multivariate functions in heavy-tailed distributions. A thorough overview of the theory behind U-processes was provided in [17]. The broad utility of U-statistics extends to various fields, including random graph theory for counting subgraphs such as triangles [23] and machine learning applications like clustering, image recognition, and graph-based learning. Even with random kernels of diverging orders, U-statistics remain relevant, as explored in [24,25,26,27]. Infinite-order U-statistics are also employed to construct simultaneous prediction intervals, addressing uncertainties in ensemble methods like subbagging and random forests [28]. Specific applications of U-statistics include the MeanNN method for estimating differential entropy [29] and novel test statistics for goodness of fit [30]. In genetics, ref. [31] used U-statistics for model-free clustering and classification, while [32] applied them to analyze random compressed sensing matrices. Ref. [33] evaluated independence tests in functional data using Kendall statistics, a specific type of U-statistics. In high-dimensional clustering, ref. [34] developed a U-statistics-based framework for group classification and partition significance, while [35] focused on dimension-agnostic inference methods using variational representations of test statistics. Other innovations include U-statistics-based empirical risk minimization [36] and asymmetric U-statistics for pattern matching in random strings and permutations [37]. Ref. [38] proposed U-statistics under left truncation and right censoring for nonparametric independence tests between time to failure and failure cause in competing risks, while [39] explored quadruplet U-statistics for network analysis. The extension of U-statistics to conditional empirical U-processes presents both practical advantages and technical challenges.

We introduce Stute’s estimators for a sequence of random elements ${\left\{({\mathbf{X}}_i,Y_i)\right\}}_{i\in {\mathbb{N}}^{*}}$, where ${\mathbf{X}}_i\in {\mathbb{R}}^d$ and $Y_i\in \mathcal{Y}$, a Polish space, with ${\mathbb{N}}^{*}=\mathbb{N}\setminus \left\{0\right\}$. For a measurable function $\phi :{\mathcal{Y}}^m\to \mathbb{R}$, our goal is to estimate the conditional expectation, or regression function, $r^{\left(m\right)}(\phi ,\mathbf{t})$ for $\mathbf{t}\in {\mathbb{R}}^{dm}$, given by

$$r^{\left(m\right)}(\phi ,\mathbf{t})=\mathbb{E}\left(\phi (Y_1,\dots ,Y_m)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_m)=\mathbf{t}\right),$$

whenever it exists, i.e., when $\mathbb{E}\left(\left|\phi (Y_1,\dots ,Y_m)\right|\right)<\infty$. We let $K:{\mathbb{R}}^d\to \mathbb{R}$ be a kernel function with support in ${[-B,B]}^d$, $B>0$, satisfying

$$\underset{\mathbf{x}\in {\mathbb{R}}^d}{sup}\left|K\left(\mathbf{x}\right)\right|=:\kappa <\infty \mathrm{and}\int K\left(\mathbf{x}\right)d\mathbf{x}=1.$$

Stute’s class of estimators for $r^{\left(m\right)}(\phi ,\mathbf{t})$, known as conditional U-statistics, is defined for each $\mathbf{t}\in {\mathbb{R}}^{dm}$ as

$${\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{t};h_K)={\displaystyle \frac{{\sum }_{(i_1,\dots ,i_m)\in I(m,n)}\phi (Y_{i_1},\dots ,Y_{i_m})K\left(\frac{{\mathbf{t}}_1-{\mathbf{X}}_{i_1}}{h_K}\right)\cdots K\left(\frac{{\mathbf{t}}_m-{\mathbf{X}}_{i_m}}{h_K}\right)}{{\sum }_{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{{\mathbf{t}}_1-{\mathbf{X}}_{i_1}}{h_K}\right)\cdots K\left(\frac{{\mathbf{t}}_m-{\mathbf{X}}_{i_m}}{h_K}\right)}},$$

where

$$I(m,n)=\left\{(i_1,\dots ,i_m):1\le i_j\le n,i_j\ne i_r\mathrm{if}j\ne r\right\},$$

is the set of all m-tuples of distinct integers between 1 and n, and ${\{h_K:=h_n\}}_{n\ge 1}$ is a sequence of positive constants converging to zero such that $n{h}_K^m\to \infty$. In the particular case where $m=1$, the regression function $r^{\left(1\right)}(\phi ,t)$ simplifies to $r^{\left(1\right)}(\phi ,t)=\mathbb{E}(\phi \left(Y\right)\mid \mathbf{X}=t)$, and Stute’s estimator corresponds to the well-known Nadaraya–Watson estimator of $r^{\left(1\right)}(\phi ,t)$[40,41], as elaborated in [42,43]. The seminal work of [44] concentrated on determining the uniform convergence rate of ${\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{t};h_K)$ to $r^{\left(m\right)}(\phi ,\mathbf{t})$ as a function of $\mathbf{t}$. Subsequently, ref. [45] investigated the asymptotic distribution of ${\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{t};h_K)$, drawing comparisons with the findings of Stute. Under appropriate mixing conditions, ref. [46] extended Stute’s framework to weakly dependent data, demonstrating the Bayes risk consistency of the corresponding classification rules. As an alternative to traditional kernel-based approaches, ref. [47] introduced symmetrized Nearest Neighbor conditional U-statistics. In a complementary vein, ref. [48] analyzed functional conditional U-statistics, establishing the asymptotic normality of their finite-dimensional distributions. Despite the critical role of nonparametric estimation of conditional U-statistics in functional data analysis, this area has remained relatively underexplored. Recent developments in this field, highlighted in [49,50,51,52,53], address key challenges, particularly regarding the uniform consistency of bandwidth selection.

There has been increasing interest in regression models where the response variable is real-valued and the explanatory variables consist of smooth functions that can vary arbitrarily across observations. This form of data, referred to as functional data, is encountered across diverse fields, including climatology, medicine, economics, and linguistics. Functional time series emerge when continuous processes are divided into smaller segments such as daily intervals where each intraday curve is treated as a functional random variable. This paper delves into functional data by investigating the theory of U-processes. For an in-depth introduction to functional data analysis, we direct readers to foundational works [54,55,56], which present case studies from various disciplines alongside core analytical methods. It is important to note that the extension of probability theory to random variables within normed vector spaces, such as Banach and Hilbert spaces, precedes many contemporary developments in the field of functional data, as highlighted in [57]. Ref. [58] explored density and mode estimation in normed vector spaces, addressing the challenges posed by the curse of dimensionality in functional data. In the context of regression, ref. [56] examined nonparametric models, while key foundational contributions to the theoretical framework and practical applications can be found in works [59,60,61]. Recent advances in functional data analysis include [62], the authors of which provided consistency rates for various functionals of the conditional distribution, uniformly over a subset of the explanatory variable. Building on this, ref. [63] extended these results by establishing uniform in bandwidth (UIB) consistency rates for nonparametric models, covering key functionals such as the regression function, conditional distribution, conditional density, and conditional hazard function. Further contributions of [64] investigated local linear estimation of the regression function when the regressor is functional, demonstrating strong convergence uniformly in bandwidth parameters. Additionally, ref. [65] explored k-Nearest Neighbors (kNN) estimation for nonparametric regression models using strongly mixing functional time series data, establishing uniform nearly complete convergence rates under moderate conditions. For more recent advancements in the field, works [66,67,68,69,70,71] provide further insights and developments. Recent literature has increasingly advocated for the integration of dimension reduction techniques in regression models. One prominent approach is the use of single-index models which reduce the influence of multiple predictors to a single index—a projection in a specific direction—coupled with a nonparametric link function. These models effectively capture the essential relationship between predictors and the response while mitigating the curse of dimensionality by focusing on a one-dimensional index. The nonparametric link function, applied to this index, allows for greater flexibility than traditional linear models, extending the scope of linear regression, where the link function is simply the identity function (see [72,73,74,75,76]). Recent advances in Functional Data Analysis (FDA) have further highlighted the importance of addressing high-dimensionality in functional regression models (see [77,78,79] for comprehensive reviews). Semiparametric models, particularly functional single-index models (FSIMs), have emerged as powerful tools in this context. Refs. [80,81,82] explored the FSIM framework, extending single-index models to accommodate functional data. Ref. [83] introduced a functional single-index composite quantile regression model, utilizing B-spline basis functions to estimate both the unknown slope and link functions. Recent papers include [84], the authors of which proposed a compact FSIM where the coefficient function is restricted to be non-zero in a subregion, and [85], the authors of which developed methods for estimating general FSIMs where the conditional distribution of the response is governed by a functional predictor through a single-index structure. Ref. [86] advanced this framework by combining functional principal component analysis (FPCA) for the functional predictors with B-spline modeling for the parameters, applying profile estimation techniques for unknown functions and parameters. Additionally, refs. [87,88] investigated FSIMs in the presence of randomly missing responses in strongly mixing time series data. Ref. [89] introduced a functional single-index varying coefficient model, wherein the functional predictor forms the single index. By applying FPCA and basis function approximations, the authors proposed an iterative estimation procedure for slope and coefficient functions. In a similar vein, ref. [90] developed an automatic, location-adaptive estimation procedure for FSIM using the kNN techniques. Inspired by imaging data analysis, ref. [91] introduced a novel functional varying-coefficient single-index model to analyze functional response data with covariates of interest. Meanwhile, ref. [92] explored the nonparametric estimation of conditional cumulative distribution functions using a functional Hilbertian regressor in the single-index framework, later extending this methodology to the multi-index case in [93]. This approach avoids fixing the true parameter within a pre-specified sieve and offers a rigorous theoretical analysis of a direct kernel-based estimation method, demonstrating polynomial convergence rates. These contributions reflect the ongoing evolution of FSIMs as versatile and robust tools in the analysis of complex, high-dimensional functional data. The principal objective of this study is to establish a comprehensive framework and rigorously investigate the weak and uniform convergence properties of k-Nearest Neighbor (kNN) single-index conditional U-processes within a regular sequence of random functions. This research is driven by the fundamental statistical significance of the kNN method, a widely adopted and versatile approach. The kNN method identifies the k closest neighbors of a given point $X_i$ to x using a prescribed distance metric $d(·,·)$. A notable feature of the kNN approach is its random, locally adaptive bandwidth, which adjusts to the underlying structure of the data—a critical aspect, especially in infinite-dimensional settings. The kNN method, originally introduced in [94] and further examined in [95], has its roots in nonparametric discrimination and was later explored in [96]. For a comprehensive exposition of the method, readers are referred to [97]. In practice, the kNN method is widely used, as demonstrated in [56], and is favored for its simplicity as it involves only one parameter, k, which dictates the number of Nearest Neighbors. This parameter is typically selected from a finite set, and the method’s local adaptability enables it to respond to the specific data structure at any point. Extensive research has been conducted on the kNN method in finite-dimensional spaces. Notable contributions include works [98,99,100,101,102]. However, in infinite-dimensional spaces, particularly within functional frameworks, three primary approaches to kNN regression estimation have emerged. The first, introduced in [103], focuses on a kNN kernel estimate where the functional variable resides in a separable Hilbert space $\mathcal{H}$. In this setting, ref. [103] established weak consistency by projecting the infinite-dimensional space onto a finite-dimensional subspace. This was achieved by considering the first m coefficients of an expansion of X in an orthonormal basis of $\mathcal{H}$ and subsequently applying multivariate kNN regression techniques to the projected data. The second approach integrated kNN methods with functional local linear estimation. Consistency and convergence rate results for this method were presented in [104,105], further enriching the theoretical underpinnings of kNN in infinite-dimensional contexts. These advances underscore the growing importance of kNN methods in functional data analysis and their adaptability to high-dimensional and complex data structures.

In this paper, we first aim to establish almost sure uniform consistency and almost sure uniform-in-the-number-of-neighbors (UINN) consistency for nonparametric functional single-index regression estimators and functional single-index conditional U-processes in the dependent setting. The concept of uniform-in-bandwidth (UIB) consistency was originally introduced in [15] for kernel density estimators using empirical process methods. This investigation is motivated by a series of foundational works, including [49,51,106,107], which established UIB consistency for similar estimators in the i.i.d. finite-dimensional framework, with $h_n$ varying within intervals indexed by n. In the realm of Functional Data Analysis (FDA), numerous studies have focused on nonparametric functional estimators. For example, ref. [62] provided consistency rates for several functionals of the conditional distribution, such as the regression function, conditional cumulative distribution, and conditional density, uniformly over specific subsets of the explanatory variables. Ref. [63] extended these results by establishing uniform consistency rates for conditional models, including regression functions, conditional distributions, densities, and hazard functions. Refs. [68,108] demonstrated almost complete convergence of k-Nearest Neighbor (kNN) estimators—uniform in the number of neighbors—under classical assumptions on kernel functions, small ball probabilities of the functional variable, and an entropy condition to control space complexity. Similarly, ref. [64] studied local linear estimation of the regression function in the context of functional covariates and established strong UIB convergence. Ref. [65] explored kNN estimation for nonparametric regression models with strongly mixing functional time series data, achieving uniform almost sure convergence rates under mild conditions, while [90] provided new uniform asymptotic results for kernel estimates within functional single-index models. Despite this rich body of literature, most studies have focused exclusively on either UIB or UINN consistency or on uniform consistency over specific functional subsets. However, the simultaneous investigation of both UIB and UINN consistency, particularly in a dependent setting, remains an open problem. This gap, which was addressed in [51] in the independent case, forms the central focus of our research. By integrating results from both Functional Data Analysis and empirical process theory, we aim to advance the understanding of consistency in more general settings. A second major challenge we address concerns the weak convergence of these estimators. This problem is inherently complex, as it requires controlling asymptotic equicontinuity under minimal conditions—a challenge that remains unresolved in the current literature. Our approach draws on seminal works [109,110], combined with strategies outlined in [111,112], to handle functional data. However, our contribution extends beyond simply merging existing concepts; it involves developing intricate mathematical derivations tailored to the specific nature of functional data. The effective application of large sample theory, particularly results related to dependent empirical processes, is essential to this endeavor, and we rely heavily on the theoretical foundations laid out in [109,110,112]. Notably, even in the i.i.d. setting, weak convergence for kNN single-index conditional U-processes has not yet been established—a gap we aim to fill with this work.

In addition, this work focuses on estimating the regression function via the kNN method in the single index nonparametric regression model under the mixing dependence condition. We emphasize that the kNN method is a fundamental statistical technique with several advantages. Generally, it is computationally fast and does not require extensive parameter tuning. One of the key features is its nonparametric nature, enabling it to adapt automatically to any continuous underlying distributions without relying on specific models. For significant statistical problems, including density estimation, classification, and regression, kNN methods are proven consistent when k is appropriately selected. kNN methods are one of the main paradigms in machine learning. Historically, the kNN method was first introduced in [94]—see also [95]—in the context of nonparametric discrimination and further investigated in [96]. For more details, refer to [97]. The investigation of single index models, popular in econometrics, is motivated by two primary concerns: dimension reduction and the interpretability of index $\mathit{\theta }$ in these models. For more on this, refer to [72,80,113] in infinite-dimensional settings. Therefore, the single functional index model accumulates the advantages of single index models and the potential of functional linear models in applications (see [114,115,116]). In essence, the single functional index model reduces dimensionality effects while capturing as much information as possible from the data. Functional data analysis is a challenging research field in the statistical community. The kNN method considers the k neighbors of $X_i$ nearest to x with respect to distance $d(·,·)$. The local bandwidth of the kNN is random and depends on data $X_i$, respecting the local structure of the data, which is essential in infinite dimensions. It is commonly used in practice (see [56]) and is simple to handle because the user controls only one parameter: the number k of Nearest Neighbors, valued in a finite set. Additionally, it allows building a neighbor adapted to the data at any point. The kNN method is widely studied if the explanatory variable is in a finite-dimensional space (see [98,99,100,101,102]). In an infinite dimensional space, i.e., a functional framework, three approaches exist for kNN regression estimation. The first, published in [103], examines a kNN kernel estimate when the functional variable is an element of a separable Hilbert space $\mathcal{H}$. In [103], a weak consistency result was established by reducing the infinite dimension of $\mathcal{H}$ through projection onto a finite dimension subspace, considering only the first ℓ coefficients of an expansion of X in an orthonormal system of $\mathcal{H}$ and then applying multivariate techniques on the projected data for kNN regression. The second approach, based on the kNN procedure and functional local linear estimation, achieved consistency with convergence rate (see [104,105,117]). The third approach, in [118], is a purely functional method.

1.1. Contribution

This paper addresses challenges in high-dimensional functional data analysis by advancing classical kernel estimators for stationary random processes. It focuses on the k-Nearest Neighbor (kNN) single-index kernel estimator, examining its regression performance and uniform consistency within functional single-index regression. Introducing the concepts of Uniform In Bandwidth (UIB) and Uniform In Nearest Neighbor (UINN) consistency, the study aims to provide robust estimates for accurate relative error predictions. It extends the kNN methodology to functional single-index conditional U-statistics, analyzing their uniform and UINN consistency, asymptotic properties, and limiting behavior. A key contribution is establishing a uniform central limit theorem for function classes meeting certain moment conditions, applicable to both bounded and unbounded functions. The methodologies employed include advanced techniques such as the kNN framework, small-ball probability, Hoeffding decompositions, decoupling methods, and modern empirical process theory indexed by function classes. The results are derived under general conditions, enhancing their practical relevance. The findings have versatile applications in statistics, including time series forecasting, set-indexed conditional U-statistics, and estimation of the Kendall rank correlation coefficient. The theoretical framework utilizes maximal moment inequalities for U-processes and $\beta$-mixing results as established in [119], providing a robust foundation for further exploration of high-dimensional functional data and sophisticated statistical models.

1.2. Organization of the Paper

The structure of this article is organized as follows. Section 2 introduces the functional framework and presents the necessary definitions for our analysis, alongside a thorough discussion of the assumptions underlying our asymptotic results. In Section 3, we investigate strong uniform convergence rates, providing a detailed exploration of the conditions that ensure such convergence. Section 4.1 is devoted to the weak convergence of empirical processes within the functional data setting, laying the foundation for subsequent theoretical developments. The main theoretical contributions, including the uniform Central Limit Theorem (CLT) for conditional U-processes, are articulated in Section 4.2, where we establish the key results of the study. In Section 5, we explore potential applications of our findings, covering set-indexed conditional U-statistics (Section 5.1), the Kendall rank correlation coefficient (Section 5.2), and discrimination problems (Section 5.3). Section 6 addresses practical concerns related to bandwidth selection, offering insights into its implementation in empirical studies. Concluding remarks, along with recommendations for future research directions, are provided in Section 7. For the sake of clarity and coherence, all proofs, which primarily draw upon modern empirical process theory, are consolidated in Section 8, with a focus on the core arguments given the technical length of the demonstrations. Additionally, a collection of pertinent technical results is included in the Appendix section to support the main text.

2. The Functional Framework

2.1. Generality on the Model

Statistical data often exhibit some degree of dependence, which can profoundly impact the accuracy of statistical inference. Neglecting to account for this dependence during analysis can lead to erroneous conclusions. The concept of mixing serves as a valuable tool for quantifying the level of dependence within a sequence of random variables, enabling the extension of classical results for independent sequences to those that exhibit weak dependence or mixing behavior. This study focuses specifically on $\beta$-mixing, or absolute regularity, a measure of dependence originally introduced in [120,121]. Sequence ${\mathbf{Z}}_1,{\mathbf{Z}}_2,\dots$ is defined as $\beta$-mixing if the conditional probabilities converge to the unconditional probabilities in a specific manner as the lag between observations increases. In the present analysis, we assume that the sequence of random elements $\{(X_i,Y_i),i\in {\mathbb{N}}^{*}\}$ is absolutely regular, which ensures that the dependence structure of the data is controlled and behaves predictably. For instance, Markov chains are known to be $\beta$-mixing under the mild Harris recurrence condition, particularly in scenarios where the underlying state space is finite [27,122,123,124]. We let $⟨·,·⟩$ denote the inner product and $\parallel ·\parallel$ the corresponding norm in the Hilbert space $\mathcal{H}$. We utilize ${\left(e_p\right)}_{p\ge 1}$ as a complete orthonormal system for $\mathcal{H}$. We consider a sequence of stationary random copies of the random vector $(X,Y)$, where X assumes values in the abstract space $\mathcal{X}$ and Y takes values in the abstract space $\mathcal{Y}$. The Hilbert space $\mathcal{H}$ is equipped with a semi-metric $d(·,·)$, which induces a topology to measure the proximity between two elements within $\mathcal{H}$. Importantly, this metric is independent of the specific definition of X to circumvent issues related to measurability. Furthermore, we define a semi-metric $d_{\mathit{\theta }}(·,·)$ associated with a single index $\mathit{\theta }\in \mathcal{H}$ given by $d_{\mathit{\theta }}(u,v)=\left|⟨\mathit{\theta },u-v⟩\right|$ for $u,v\in \mathcal{H}$. This construction facilitates the analysis of functional data through a single-index structure while accounting for the dependence introduced by the $\beta$-mixing condition. We assume that fixed values ${\mathit{\theta }}_1,\dots ,{\mathit{\theta }}_m\in \mathbf{\Theta }\subset \mathcal{H}$ exist, where observations ${(X_i,Y_i)}_{i=1,\dots ,n}$ satisfy the following relation:

$$\phi (Y_{i_1},\dots ,Y_{i_m})=r^{\left(m\right)}(\phi ,⟨{\mathit{\theta }}_1,{\mathbf{t}}_1⟩,\dots ,⟨{\mathit{\theta }}_m,{\mathbf{t}}_m⟩)+{\epsilon }_{i_1,\dots ,i_m},\forall i_j=1,\dots ,n,j=1,\dots ,m,$$

(1)

where ${\epsilon }_{i_1,\dots ,i_m}$ is a real random variable satisfying $\mathbb{E}({\epsilon }_{i_1,\dots ,i_m}\mid (X_{i_1},\dots ,X_{i_m}))=0$ almost surely. To ensure identifiability, we impose the assumption that the regression function is differentiable, and for $i=1,\dots ,m$, we require that $⟨{\mathit{\theta }}_i,e_1⟩=1$, where $e_1$ denotes the first element of the orthonormal basis in $\mathcal{H}$. This condition ensures the identifiability of the single functional index model. For a comprehensive discussion on the identifiability issues related to single functional index models, we refer to [80]. The primary objective of this study is to establish the weak convergence of the single-index conditional U-process, which is constructed from the single-index conditional U-statistic in the kNN framework introduced in [51]. This investigation is pivotal for advancing the theoretical understanding of nonparametric methods in high-dimensional functional data, particularly within the context of U-processes indexed by single indices. This kNN estimator is defined as

$${\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))={\displaystyle \frac{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}\phi (Y_{i_1},\dots ,Y_{i_m})K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}},$$

(2)

which serves as an estimator for the multivariate regression function:

$$r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta }):=\mathbb{E}\left(\phi (Y_1,\dots ,Y_m)\mid (⟨X_1,{\mathit{\theta }}_1⟩,\dots ,⟨X_m,{\mathit{\theta }}_m⟩)=(⟨t_1,{\mathit{\theta }}_1⟩,\dots ,⟨t_m,{\mathit{\theta }}_m⟩)\right).$$

The kernel function $K(·)$ and $\phi :{\mathcal{Y}}^m\to \mathbb{R}$ is a symmetric measurable function from class ${\mathcal{F}}_m$. Further, ${\mathbf{h}}_{n,k}\left(\mathbf{t}\right)$ is a vector of positive random variables depending on $\left(X_1,\dots ,X_n\right)$, defined for $x_j\in {\mathcal{X}}^m$ by

$$H_{n,k}(x_j,\mathit{\theta })=min\left\{h\in {\mathbb{R}}^{+}:\sum _{i=1}^n{\mathbb{1}}_{B_{\mathit{\theta }}(x_j,h)}\left(X_i\right)=k\right\},$$

where $B_{\mathit{\theta }}(t,r)$ is a ball in $\mathcal{X}$ and ${\mathbb{1}}_A$ denotes the indicator function. This kNN estimator generalizes the functional conditional U-statistic defined as

$${\widehat{r}}_n^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };h_K\left(\mathbf{t}\right))={\displaystyle \frac{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}\phi (Y_{i_1},\dots ,Y_{i_m})K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{h_K\left(t_1\right)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{h_K\left(t_m\right)}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{h_K\left(t_1\right)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{h_K\left(t_m\right)}\right)}}},$$

(3)

where $h_K\left(\mathbf{t}\right)$ are positive real numbers decreasing as $n\to \infty$. This estimator traces its origins to the Akaike–Parzen–Rosenblatt kernel density estimation [125,126,127], with early applications discussed in [94] and later republished in [95]. The extensive body of literature surrounding this topic includes seminal works such as [128,129,130,131], among others. The k-Nearest Neighbor (kNN) method remains a fundamental statistical tool, valued for its computational efficiency and minimal requirement for parameter tuning. Our objective is to extend the results from [49,112,132,133] to a multivariate framework, with a particular focus on VC-subgraph classes and small-ball probabilities, as introduced in [56,58,111]. For a comprehensive understanding of VC-theory, we refer readers to the foundational work in [134] and related literature. This extension is critical for advancing the theoretical understanding and practical application of nonparametric methods in multivariate functional data contexts.

 Definition 1. 

A class of subsets $\mathcal{C}$ on a set C is called a VC-class if there exists a polynomial $\mathfrak{P}(·)$ such that for every set of N points in C, the class $\mathcal{C}$ selects at most $\mathfrak{P}\left(N\right)$ distinct subsets.

 Definition 2. 

A class of functions $\mathcal{F}$ is called a VC-subgraph class if the graphs of the functions in $\mathcal{F}$ form a VC-class of sets. Specifically, if we define the subgraph of a real-valued function f on S as the subset ${\mathcal{G}}_f$ on $S\times \mathbb{R}$ such that

$${\mathcal{G}}_f=\{(s,t):0\le t\le f\left(s\right)orf\left(s\right)\le t\le 0\},$$

then the class $\{{\mathcal{G}}_f:f\in \mathcal{F}\}$ forms a VC-class of sets on $S\times \mathbb{R}$. Informally, a VC-class of functions is characterized by having a polynomial covering number, which is the minimal number of functions required to cover the entire class.

 Definition 3. 

Let ${\mathcal{S}}_{\mathcal{E}}$ be a subset of a semi-metric space $\mathcal{E}$, and let $N_{\epsilon }$ be a positive integer. A finite set of points $\{e_1,\dots ,e_{N_{\epsilon }}\}\subset \mathcal{E}$ is called an ε-net of ${\mathcal{S}}_{\mathcal{E}}$ for a given $\epsilon >0$ if

$${\mathcal{S}}_{\mathcal{E}}\subseteq \bigcup _{j=1}^{N_{\epsilon }}{B}_{\mathit{\theta }}(e_j,\epsilon ),$$

where $B_{\mathit{\theta }}(e_j,\epsilon )$ represents a ball of radius ε around the point $e_j$. If $N_{\epsilon }\left({\mathcal{S}}_{\mathcal{E}}\right)$ is the cardinality of the smallest ε-net, i.e., the minimal number of open balls of radius ε needed to cover ${\mathcal{S}}_{\mathcal{E}}$, then the Kolmogorov entropy (metric entropy) of the set ${\mathcal{S}}_{\mathcal{E}}$ is defined as

$${\psi }_{{\mathcal{S}}_{\mathcal{E}}}\left(\epsilon \right):=log{N}_{\epsilon }\left({\mathcal{S}}_{\mathcal{E}}\right).$$

The concept of metric entropy, introduced by Kolmogorov (cf. [135]), has been the subject of extensive research across various metric spaces. Dudley [136] applied this concept to establish sufficient conditions for the continuity of Gaussian processes, thereby laying the foundation for significant generalizations of Donsker’s theorem concerning the weak convergence of empirical processes. Let ${\mathcal{B}}_{\mathcal{H}}$ and ${\mathcal{S}}_{\mathcal{H}}$ represent two subsets of the semi-metric space $\mathcal{H}$, and denote the Kolmogorov entropy for a given radius $\epsilon$ by ${\psi }_{{\mathcal{B}}_{\mathcal{H}}}\left(\epsilon \right)$ and ${\psi }_{{\mathcal{S}}_{\mathcal{H}}}\left(\epsilon \right)$, respectively. The Kolmogorov entropy for the product subset ${\mathcal{B}}_{\mathcal{H}}\times {\mathcal{S}}_{\mathcal{H}}$ within the semi-metric space ${\mathcal{H}}^2$ is expressed as

$${\psi }_{{\mathcal{B}}_{\mathcal{H}}\times {\mathcal{S}}_{\mathcal{H}}}\left(\epsilon \right)={\psi }_{{\mathcal{B}}_{\mathcal{H}}}\left(\epsilon \right)+{\psi }_{{\mathcal{S}}_{\mathcal{H}}}\left(\epsilon \right).$$

Consequently, $m{\psi }_{{\mathcal{S}}_{\mathcal{H}}}\left(\epsilon \right)$ denotes the Kolmogorov entropy of the subset ${\mathcal{S}}_{\mathcal{H}}^m$ within the semi-metric space ${\mathcal{H}}^m$. Given a semi-metric d on $\mathcal{H}$, define a semi-metric on ${\mathcal{H}}^m$ as

$$d_{{\mathcal{H}}^m}(\mathbf{x},\mathbf{y}):={\displaystyle \frac{1}{m}}\sum _{i=1}^{m}d(x_i,y_i),$$

where $\mathbf{x}=(x_1,\dots ,x_m)$ and $\mathbf{y}=(y_1,\dots ,y_m)$ are elements of ${\mathcal{H}}^m$. Furthermore, it is possible to define another semi-metric, $d_{{\mathcal{H}}^m,\mathit{\theta }}(\mathbf{x},\mathbf{y})$, on ${\mathcal{H}}^m$, parameterized by $\mathit{\theta }=({\mathit{\theta }}_1,\dots ,{\mathit{\theta }}_m)$ in ${\mathbf{\Theta }}^m$, and based on the individual semi-metrics $d_{{\mathit{\theta }}_1},\dots ,d_{{\mathit{\theta }}_m}$. The selection of the appropriate semi-metric is critical in such analyses, as it significantly impacts the theoretical properties and results. For an in-depth discussion on these considerations, refer to [56], particularly Chapters 3 and 13, which provide comprehensive insights into the role of semi-metrics in functional data analysis.

2.2. Conditions and Comments

Let us present the conditions that we need in our analysis in the case of known ${\mathit{\theta }}_0=({\mathit{\theta }}_{0,1},\dots ,{\mathit{\theta }}_{0,m})$.

 (C.1.) 

On the distributions/small-ball probabilities

 (C.1.1) 

For $\mathbf{t}=(t_1,\dots ,t_m)\in {\mathcal{X}}^m$ and $\mathbf{h}\left(\mathbf{t}\right)=(h_1\left(t_1\right),\dots ,h_m\left(t_m\right))\in {\mathbb{R}}_{+}^m\setminus \left\{0\right\}$, we have

$${\varphi }_{{\mathit{\theta }}_0,\mathbf{t}}\left(\mathbf{h}\left(\mathbf{t}\right)\right):=\mathbb{P}\left(X_1\in B_{{\mathit{\theta }}_{0,1}}(t_1,h_1\left(t_1\right)),\dots ,X_m\in B_{{\mathit{\theta }}_{0,m}}(t_m,h_m\left(t_m\right))\right)$$

$$0<C_1\tilde{\varphi }\left(\mathbf{h}\right)f_1\left(\mathbf{t}\right)\le {\varphi }_{{\mathit{\theta }}_0,\mathbf{t}}\left(\mathbf{h}\left(\mathbf{t}\right)\right)\le C_2\tilde{\varphi }\left(\mathbf{h}\right)f_1\left(\mathbf{t}\right)<\infty ,$$

where $f_1\left(\mathbf{t}\right)$ is a non-negative functional in $\mathbf{t}=(t_1,\dots ,t_m)\in {\mathcal{H}}^m$, ${\displaystyle \tilde{\varphi }\left(\mathbf{h}\right):=\prod _{j=1}^m\varphi \left(h_j\left(t_j\right)\right)}$ and $\varphi \left(0\right)=0$ and $\varphi \left(u\right)$ is an invertible function absolutely continuous in a neighbor of the origin.

 (C.1.2) 

For $i\ne j,$ let ${\mathbf{X}}_{\mathbf{i}}=\left(X_{i_1},\dots ,X_{i_m}\right),{\mathbf{X}}_{\mathbf{j}}=\left(X_{j_1},\dots ,X_{j_m}\right)and\mathbf{t}=\left(t_1,\dots ,t_m\right)\in {\mathcal{H}}^m$, we have

$$\underset{\mathbf{i}\ne \mathbf{j}}{sup}\mathbb{P}\left\{{\mathbf{X}}_{\mathbf{i}}\in \prod _{i=1}^{m}B\left(t_i,h_i\left(t_i\right)\right),{\mathbf{X}}_{\mathbf{j}}\in \prod _{i=1}^{m}B\left(t_i,h_i\left(t_i\right)\right)\right\}\le {\tilde{\Psi }}_{{\mathit{\theta }}_0}\left(\mathbf{h}\left(\mathbf{t}\right)\right)f_2\left(\mathbf{t}\right),$$

where $f_2\left(\mathbf{t}\right)$ is a non-negative function, ${\displaystyle {\tilde{\Psi }}_{{\mathit{\theta }}_0}\left(\mathbf{h}\left(\mathbf{t}\right)\right):=\prod _{i=1}^m{\psi }_{{\mathit{\theta }}_i}\left(h_i\left(t_i\right)\right)}$ and ${\tilde{\Psi }}_{{\mathit{\theta }}_0}\left(\mathbf{h}\left(\mathbf{t}\right)\right)\to 0$ as $n\to \infty$ satisfying ${\tilde{\Psi }}_{{\mathit{\theta }}_0}\left(\mathbf{h}\left(\mathbf{t}\right)\right)/{\tilde{\varphi }}^2\left(\mathbf{h}\left(\mathbf{t}\right)\right)$ is bounded.

 (C.2.) 

On the smoothness of the model

 (C.2.1) 

The regression satisfies for $\phi (·)\in {\mathcal{F}}_m,$ and some $\gamma >0$ for $1\le m\le n$,

$$\exists \gamma >0,\forall {\mathbf{t}}_1,{\mathbf{t}}_2\in S_{\mathcal{H}}^m:\left|r^{\left(m\right)}(\phi ,\mathbf{t},{\mathit{\theta }}_1)-r^{\left(m\right)}(\phi ,\mathbf{t},{\mathit{\theta }}_2)\right|\le C_3{d}_{{\mathcal{H}}^m,{\mathit{\theta }}_0}^{\gamma }\left({\mathbf{t}}_{\mathbf{1}},{\mathbf{t}}_{\mathbf{2}}\right);$$

 (C.2.2) 

The conditional variance, defined for $\mathbf{u}\in {\mathcal{H}}^m,\mathrm{Var}\left[\phi \left(\mathbf{Y}\right)|\mathbf{X}=\mathbf{u}\right]=:g_2\left(\mathbf{u},\phi \right)$ is continuous in some neighborhood of $\mathbf{t}$,

$$\underset{d_{{\mathcal{H}}^m,{\mathit{\theta }}_0}\left(\mathbf{t},\mathbf{u}\right)⩽h_n}{sup}\left|g_2\left(\mathbf{t},\phi \right)-g_2\left(\mathbf{u},\phi \right)\right|=o\left(1\right)asn\to \infty .$$

Further, we assume that for some $p>2,\mathbb{E}{\left|F\left(\mathbf{Y}\right)\right|}^p<\infty ,$ and $\phi (·)\in {\mathcal{F}}_m,$

$$g_p\left(\mathbf{t},\mathbf{u},\phi \right):=\mathbb{E}\left({\left|\phi \left(\mathbf{Y}\right)-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|}^p\left|\phantom{\mathbf{Y}-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })}\right.\mathbf{X}=\mathbf{u}\right),$$

is continuous in some neighborhood of $\mathbf{t}.$

 (C.2.3) 

For $\mathbf{u},\mathbf{v}\in {\mathcal{H}}^m,$ the function $g_{\phi }\left(\mathbf{t},\mathbf{u},\mathbf{v}\right)$ does not depend on $\mathbf{i},\mathbf{j}$ and is continuous in some neighborhood of $\left(\mathbf{t},\mathbf{t}\right)$,

$$g_{\phi }\left(\mathbf{t},\mathbf{u},\mathbf{v}\right):=\mathbb{E}\left(\left(\phi \left({\mathbf{Y}}_{\mathbf{i}}\right)-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right)\left(\phi \left({\mathbf{Y}}_{\mathbf{j}}\right)-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right)\left|\phantom{\mathbf{Y}-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })}\right.{\mathbf{X}}_{\mathbf{i}}=\mathbf{u},{\mathbf{X}}_{\mathbf{j}}=\mathbf{v}\right).$$

 (C.3) 

On the kernel function

 (C.3.1) 

The kernel functions $K(·)$ is supported within $[0,1]$, and there exist some constants $0<{\kappa }_1\le {\kappa }_2<\infty$ such that

$${\int }_0^{1}K\left(x\right)dx=1,$$

and

$$0<{\kappa }_1{\mathbb{1}}_{[0,1]}(·)\le K(·)\le {\kappa }_2{\mathbb{1}}_{[0,1]}(·).$$

 (C.3.2) 

The kernel $K(·)$ is a positive function and differentiable function on $[0,1]$ with derivative $K^{\prime }(·)$ such that

$$-\infty <{\kappa }_3<K^{\prime }(·)<{\kappa }_4<0.$$

(4)

 (C.4) 

On the classes of functions

 (C.4.1) 

The class of functions ${\mathcal{F}}_m$ is bounded and its envelope function satisfies, for some $0<M<\infty$,

$$F\left(\mathbf{y}\right)\le M,\mathbf{y}\in {\mathcal{Y}}^m$$

 (C.4.2) 

The class of functions ${\mathcal{F}}_m$ is unbounded and its envelope function satisfies, for some $p>2$,

$${\mathit{\theta }}_p:=\underset{\mathbf{t}\in S_{\mathcal{X}}^m}{sup}\mathbb{E}\left(F^p\left(\mathbf{Y}\right)|\mathbf{X}=\mathbf{t}\right)<\infty .$$

 (C.4.3) 

The metric entropy of the class ${\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m$ satisfies, for some $1\le p<\infty$,

$$\begin{array}{cc}\hfill & {\int }_0^{\infty }(logN(u,{\mathcal{F}}_m{\mathfrak{K}}_{{\mathit{\theta }}_0}^m{,\parallel ·\parallel }_p{\left)\right)}^{{\displaystyle \frac{1}{2}}}du<\infty ,\hfill \end{array}$$

where

$${\mathcal{F}}_m{\mathfrak{K}}_{{\mathit{\theta }}_0}^m=\left\{fg:f\in {\mathcal{F}}_m,g\in {\mathfrak{K}}_{{\mathit{\theta }}_0}^m\right\}.$$

 (C.4.4) 

The class of functions ${\mathcal{F}}_m{\mathfrak{K}}_{{\mathit{\theta }}_0}^m$ is supposed to be of VC-type with envelope function previously defined. Hence, there are two finite constants b and $\nu$ such that

$$N\left(ϵ,{\mathcal{F}}_m{\mathfrak{K}}_{{\mathit{\theta }}_0}^m,{\parallel ·\parallel }_{L_2\left(Q\right)}\right)\le {\left({\displaystyle \frac{{\displaystyle b\parallel F{\kappa }^m{\parallel }_{L_2\left(Q\right)}}}{{\displaystyle ϵ}}}\right)}^{\nu }$$

for any $ϵ>0$ and each probability measure such that $Q{\left(F\right)}^2<\infty$.

 (C.5) 

On the dependence of the random variables

 (C.5.1) 

Absolute regularity

$$\sum _{s=1}^{\infty }{s}^{\delta }{\left(log\left(s\right)\right)}^{\delta (1-1/p)}{\left(\beta \left(s\right)\right)}^{1-2/p}<\infty ,$$

for some $p>2$ and $\delta >1-2/p.$

 (C.5.2) 

There is a sequence of positive integers ${\left\{s_n\right\}}_{n\in {\mathbb{N}}^{*}}$ such that, as $n\to \infty$,

$$s_n\to \infty ,s_n=o\left(\sqrt{n\tilde{\varphi }\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\right),{\left({\displaystyle \frac{n}{\tilde{\varphi }\left(\mathbf{h}\left(\mathbf{t}\right)\right)}}\right)}^{1/2}\beta \left(s_n\right)⟶0.$$

 (C.6) 

On the entropy

For n large enough and for some $\omega >1$, Kolomogorov’s entropy satisfies

$${\displaystyle \frac{{(logn)}^2}{n\varphi \left(h_K\right)}}\le m{\psi }_{S_{\mathcal{H}}}\left({\displaystyle \frac{logn}{n}}\right)+m{\psi }_{\mathbf{\Theta }}\left({\displaystyle \frac{logn}{n}}\right)\le {\displaystyle \frac{n}{logn}}\varphi \left(h_K\right),$$

(5)

$$\sum _{n=1}^{\infty }exp\left\{m(1-\omega )\left\{{\psi }_{S_{\mathcal{H}}}\left({\displaystyle \frac{logn}{n}}\right)+{\psi }_{\mathbf{\Theta }}\left({\displaystyle \frac{logn}{n}}\right)\right\}\right\}<\infty ,$$

(6)

where ${\psi }_A\left(\epsilon \right):=log{N}_{\epsilon }\left(A\right),$ and $N_{\epsilon }\left(A\right)$ is the minimal number of open balls of radius $\epsilon$ in $\mathcal{H}$ needed to cover $A.$

 (C.7.) 

The sequences $\left\{{\tilde{h}}_n\right\}$ and $\left\{{\widetilde{h^{\prime }}}_n\right\}$ (respectively, $\left\{h_{n,1}\right\}$ and $\left\{h_{n,2}\right\}$) verify

$${\widetilde{h^{\prime }}}_n⟶0\mathrm{and}{\displaystyle \frac{(logn/n)}{min\{{\tilde{h}}_n^2,{\varphi }^2\left({\tilde{h}}_n\right)\}}}⟶0\mathrm{as}n⟶\infty .$$

(7)

 (C.8.) 

There exist sequences $\left\{{\rho }_{n,1},\dots ,{\rho }_{n,m}\right\}\subset {(0,1)}^m$, $\left\{k_{1,n}\right\}\subset {\mathbb{Z}}^{+}$ and $\left\{k_{2,n}\right\}\subset {\mathbb{Z}}^{+}$ ($k_{1,n}\le k\le k_{2,n}$) and constants $\mathit{\mu }=({\mu }_1,\dots ,{\mu }_m)and\mathit{\nu }=({\nu }_1,\dots ,{\nu }_m)$, such that

$$0<{\mu }_j\le {\nu }_j<\infty ,\mathrm{for} \mathrm{all}j=1,\dots ,m(\mathrm{and} \mathrm{we} \mathrm{note}\mathit{\mu }\le \mathit{\nu }),$$

and

$$\begin{array}{cc}\hfill & {\mu }_j{\varphi }^{-1}\left({\displaystyle \frac{{\rho }_{n,j}{k}_{1,n}}{n}}\right)\le {\varphi }_{t_j,{\mathit{\theta }}_j}^{-1}\left({\displaystyle \frac{{\rho }_{n,j}{k}_{1,n}}{n}}\right)\mathrm{and}{\varphi }_{t_j,{\mathit{\theta }}_j}^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n,j}n}}\right)\le {\nu }_j{\varphi }^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n,j}n}}\right),\hfill \end{array}$$

(8)

$$\begin{array}{cc}\hfill & {\varphi }^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n,j}n}}\right)⟶0,\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill & min\left\{{\displaystyle \frac{1-{\rho }_{n,j}}{4}}{\displaystyle \frac{k_{1,n}}{lnn}},{\displaystyle \frac{{(1-{\rho }_{n,j})}^2}{4{\rho }_{n,j}}}{\displaystyle \frac{k_{1,n}}{lnn}}\right\}>2,\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill & {\displaystyle \frac{{\displaystyle (logn/n)}}{{\displaystyle min\left\{{\mu }_j{\varphi }^{-1}\left(\frac{{\rho }_{n,j}{k}_{1,n}}{n}\right),\varphi \left({\mu }_j{\varphi }^{-1}\left(\frac{{\rho }_{n,j}{k}_{1,n}}{n}\right)\right)\right\}}}}⟶0.\hfill \end{array}$$

(11)

• Additional/alternative conditions

   (C.1’.) 

  For ${\mathbf{X}}_{\mathbf{i}}=\left(X_{i_1},\dots ,X_{i_m}\right),{\mathbf{X}}_{\mathbf{j}}=\left(X_{j_1},\dots ,X_{j_m}\right)and\mathbf{t}=(t_1,\dots ,t_m)\in {\mathcal{H}}^m,$

   (C.1’.1) 

  We have

  $${\varphi }_{\mathbf{t},{\mathit{\theta }}_0}\left(\mathbf{x}\right)=\tilde{\varphi }\left(\mathbf{x}\right)f_1\left(\mathbf{t}\right)<\infty \mathrm{as}\mathbf{x}\to \infty .$$

  with $\mathbf{x}=(x_1,\dots ,x_m)\in {\mathbb{R}}_{+}^m$, $\varphi \left(0\right)=0,$ and $\varphi \left(u\right)$ is absolutely continuous in a neighborhood of the origin.

   (C.1’.2) 

  We have

  $$\underset{\mathbf{i}\ne \mathbf{j}}{sup}\mathbb{P}\left\{{\mathbf{X}}_{\mathbf{i}}\in \prod _{i=1}^m{B}_{{\mathit{\theta }}_i}\left(t_i,h_i\left(t_i\right)\right),{\mathbf{X}}_{\mathbf{j}}\in \prod _{i=1}^m{B}_{{\mathit{\theta }}_i}\left(t_i,h_i\left(t_i\right)\right)\right\}\le {\tilde{\Psi }}_{{\mathit{\theta }}_0}\left(\mathbf{h}\left(\mathbf{t}\right)\right)f_2\left(\mathbf{t}\right),$$

  where $f_2\left(\mathbf{t}\right)$ is a non-negative function, ${\displaystyle {\tilde{\Psi }}_{{\mathit{\theta }}_0}\left(\mathbf{h}\left(\mathbf{t}\right)\right):=\prod _{i=1}^m{\psi }_{{\mathit{\theta }}_i}\left(h_i\left(t_i\right)\right)}$, and ${\tilde{\Psi }}_{{\mathit{\theta }}_0}\left(\mathbf{h}\left(\mathbf{t}\right)\right)\to 0$ as $n\to \infty$ satisfying ${\tilde{\Psi }}_{{\mathit{\theta }}_0}\left(\mathbf{h}\left(\mathbf{t}\right)\right)/{\tilde{\varphi }}^2\left(\mathbf{h}\left(\mathbf{t}\right)\right)$ is bounded.

   (C.2’.) 

  The kernel function $K(·)$ is supported within $[0,1]$, and there exist some constants $0<{\kappa }_2,$$0<{\kappa }_1^{\prime }\le {\kappa }_2^{\prime }<\infty$ such that for $j=1,2$

  $${\int }_0^{1}K\left(x\right)dx=1,K(·)\le {\kappa }_2{\mathbb{1}}_{[0,1]}(·),{\displaystyle \frac{h_n}{\varphi \left(h_K\left(t\right)\right)}}{\int }_0^1{K}^j\left(x\right){\varphi }^{\prime }\left(v{h}_K\right)dv\to {\kappa }_j^{\prime }\mathrm{as}n\to \infty .$$

Comments

In our nonparametric functional regression model, a significant theoretical challenge arises in establishing functional central limit theorems for conditional empirical processes and conditional U-processes under functional absolute regularity. Additionally, we incorporate random (or data-dependent) bandwidths based on the k-Nearest Neighbor (kNN) approach. Traditional statistical methods cannot be directly applied in this functional setting, so many of the conditions we impose are tailored to the characteristics of infinite-dimensional spaces, such as the topological structure of ${\mathcal{H}}^m$, the probability distribution of X, and the concept of measurability for the classes ${\mathcal{F}}_m$ and ${\mathcal{K}}^m$. It is worth noting that these conditions draw inspiration from [56,58,65,111,112]. We begin with assumption (C.1.1), adapted from [111], which itself is influenced by [58]. As explained in [111], when ${\mathcal{H}}^m={\mathbb{R}}^m$, Condition (C.1.1) aligns with the fundamental axioms of probability calculus. Moreover, if ${\mathcal{X}}^m$ is an infinite-dimensional Hilbert space, then $\varphi \left(h_K\right)$ can decay exponentially as $n\to \infty$. Condition (C.1.1) is a standard assumption on small ball probability, which controls the behavior of ${\varphi }_{\mathbf{t},\mathit{\theta }}(·)$ around zero. This condition allows us to express the small ball probability as the product of two independent functions, $\tilde{\varphi }(·)$ and $f_1(·)$; for further details, see, for example, ref. [137] for diffusion processes, [138] for Gaussian measures, and [139] for general Gaussian processes. A widely recognized result in the literature expresses small ball probability in the form ${\phi }_t\left(\epsilon \right)\sim g\left(t\right)\varphi \left(\epsilon \right)$, where $\varphi \left(\epsilon \right)={\epsilon }^{\gamma }exp\left(-C/{\epsilon }^p\right)$, with $\gamma \ge 0$ and $p\ge 0$. This formulation corresponds to various processes, such as Ornstein–Uhlenbeck and general diffusion processes (with $p=2$ and $\gamma =0$) and fractal processes (with $\gamma >0$ and $p=0$). For additional examples, refer to [140]. When dealing with functional data, it is crucial to gather information about the variability of the small ball probability to adapt it to the bias of nonparametric estimators. This information is typically obtained by assuming

 (C.1.1”) 

$$\forall u\in [0,1]:\underset{r\to \infty }{lim}{\displaystyle \frac{{\varphi }_{t,\mathit{\theta }}\left(ur\right)}{{\varphi }_{t,\mathit{\theta }}\left(r\right)}}=\underset{r\to \infty }{lim}\mathbb{P}\left(d_{\mathit{\theta }}(X,t)\le ur\mid d_{\mathit{\theta }}(X,t)\le r\right)=:{\tau }_{t,\mathit{\theta }}\left(u\right)<\infty .$$

In our model, we also consider Condition (C.2), which relates to the regularity of the model. This condition incorporates mild assumptions about the continuity of certain conditional moments and imposes the standard Lipschitz condition on Regression Function (C.2.1). Assumption (C.3) pertains to the kernel function $K(·)$, where (C.3.1) can be replaced by (C.3’). For bounded functions, we employ (C.4.1), while for conditional U-processes indexed by an unbounded class, we utilize (C.4.2). It is essential to balance the moment order p in (C.4.2) with the decay rate of the mixing coefficient $\beta \left(s\right)$ in (C.5). Condition (C.6) addresses topological concerns by controlling the Kolmogorov entropy of the set $S_{\mathcal{X}}^m$, which is critical for studying uniform consistency and uniform-in-bandwidth consistency. Assumption (C.7) plays a crucial role in establishing the consistency rates of the estimator, while (C.8) adapts (C.7) to the specific case of functional conditional U-statistics within the kNN framework. These conditions collectively form the theoretical backbone for ensuring the rigor and robustness of our nonparametric functional regression model.

 Remark 1. 

It is important to highlight that condition(C.4.2)may be substituted with more general assumptions regarding the moments of $\mathbf{Y}$, as detailed in [141]. Specifically:

•  (M.1)″  We let $\left\{\mathcal{M}\right(x):x\ge 0\}$ be a nonnegative continuous function, increasing on $[0,\infty )$, such that for some $s>2$, as $x\uparrow \infty$,

  $$\left(i\right)x^{-s}\mathcal{M}\left(x\right)\downarrow ;\left(ii\right)x^{-1}\mathcal{M}\left(x\right)\uparrow .$$

  (12)
  For each $t\ge \mathcal{M}\left(0\right)$, we define ${\mathcal{M}}^{inv}\left(t\right)\ge 0$ such that $\mathcal{M}\left({\mathcal{M}}^{inv}\left(t\right)\right)=t$. We assume further that

  $$\mathbb{E}\left(\mathcal{M}\left(\left|F\left(\mathbf{Y}\right)\right|\right)\right)<\infty .$$
• The following choices of $\mathcal{M}(·)$ are of particular interest:

  (i)

  $\mathcal{M}\left(x\right)=x^p$ for some $p>2$;

  (ii)

  $\mathcal{M}\left(x\right)=exp\left(sx\right)$ for some $s>0$.

  For a positive constant $\mathfrak{L}$, we set

  $${\rho }_{\mathfrak{L}}^2\left(f\right)=2{\mathfrak{L}}^2\mathbb{E}\left(e^{\left|f\right(X\left)\right|/\mathfrak{L}}-1-\left|f\left(X\right)\right|/\mathfrak{L}\right)=2{\mathfrak{L}}^2\sum _{k=2}^{\infty }{\displaystyle \frac{{\mathbb{E}\left|f\left(X\right)\right|}^k}{{\mathfrak{L}}^{k}k!}}$$

  where X has probability law $\mathbb{P}$ and define the Bernstein size of f as the nonnegative square root of ${\rho }_{\mathfrak{L}}^2\left(f\right)$. Note that ${\rho }_{\mathfrak{L}}\left(f\right)<\infty$ if and only if $\mathbb{E}{e}^{\left|f\right(X\left)\right|/\mathfrak{L}}<\infty$, and that if this holds for some K, then ${lim}_{\mathfrak{L}\to \infty }{\rho }_{\mathfrak{L}}\left(f\right)=\mathbb{E}{f}^2\left(X\right)$. One can use the following condition ${\rho }_{\mathfrak{L}}\left(f\right)\le R$ for some positive constant R.

In the following section, we comprehensively present the main results concerning the uniform consistency of the regression estimators and the conditional U-statistics. This includes detailed discussions on the theoretical foundations and the conditions under which these estimators achieve uniform consistency. We also explore the implications of these findings for practical applications, highlighting how they enhance the reliability and accuracy of statistical inference in regression analysis and conditional U-statistics.

3. Uniform Consistency

For simplicity reasons, Condition (C.1.1) on the small ball probability is replaced by

 (H.1) 

For $\mathbf{h}\in {\mathbb{R}}_{+}^m\setminus \left\{0\right\}$ and $\mathbf{t}\in {\mathcal{X}}^m$

$$0<C_1\tilde{\varphi }\left(\mathbf{h}\right)\le {\varphi }_{\mathbf{t},{\mathit{\theta }}_0}\left(\mathbf{h}\left(\mathbf{t}\right)\right)\le C_2\tilde{\varphi }\left(\mathbf{h}\right)<\infty .$$

(13)

This standard condition can be considered an extension of the multivariate case where we assume that the density function of the variable $\mathbf{X}$ is strictly positive. Also, it is worth mentioning that if we particularly denote ${\mathbf{h}}_K:=(h_K,\dots ,h_K)\in {({\tilde{h}}_{n,1},{\tilde{h}}_{n,2})}^m$, we can find two positive constants $C_1^{\prime },C_2^{\prime }$ such that

$$0<C_1^{\prime }\varphi \left(h_K\right)\le {\varphi }_{\mathbf{t},{\mathit{\theta }}_0}\left({\mathbf{h}}_K\right)\le C_2^{\prime }\varphi \left(h_K\right)<\infty ,$$

(14)

which is similar to Condition (C.1) used in [49,142], so we deal with $\varphi \left(h_K\right)$ instead of ${\varphi }^m\left(h_K\right),$ (whenever we encounter a similar situation in the proofs). This approach does not merely serve as a notational convenience but also facilitates the integration of the UIB and UINN results, enhancing the theoretical coherence between the two frameworks.

3.1. Uniform Consistency of the  kNN Kernel Estimator for Regression

In this section, we investigate the uniform consistency of the functional regression operator in its general form, which is expressed for all $t\in \mathcal{H}$ as

$${\widehat{r}}_n^{*\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,H_{n,k}(t,\mathit{\theta }))={\displaystyle \frac{{\displaystyle \sum _{i=1}^n\phi \left(Y_i\right)K\left(\frac{d_{\mathit{\theta }}(t,X_i)}{H_{n,k}(t,\mathit{\theta })}\right)}}{{\displaystyle \sum _{i=1}^{n}K\left(\frac{d_{\mathit{\theta }}(t,X_i)}{H_{n,k}(t,\mathit{\theta })}\right)}}},$$

(15)

where $k=k_n$ depending on n and

$$H_{n,k}(t,\mathit{\theta })=min\left\{h\in {\mathbb{R}}^{+}\mathrm{such} \mathrm{that}\sum _{i=1}^n{\mathbb{1}}_{B_{\mathit{\theta }}(t,h)}\left(X_i\right)=k\right\}.$$

In fact, the kNN operator presented in (15) can be regarded as a natural generalization, broadening its applicability and enhancing its flexibility in handling more complex, high-dimensional functional data, of the traditional kernel regression estimator

$${\widehat{r}}_n^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,h_K)={\displaystyle \frac{{\displaystyle \sum _{i=1}^n\phi \left(Y_i\right)K\left(\frac{d_{\mathit{\theta }}(t,X_i)}{h_K}\right)}}{{\displaystyle \sum _{i=1}^{n}K\left(\frac{d_{\mathit{\theta }}(t,X_i)}{h_K}\right)}}},\mathrm{for} \mathrm{each}t\in \mathcal{H},$$

(16)

where the bandwidth $h_K\in {\mathbb{R}}_{+}^{*}$ depends on n (but does not depend on t).

3.1.1. UIB Consistency for Functional Regression

Recall the bandwidths $h_{n,1}$ and $h_{n,2}$ given in Condition (C.7.). The following theorem plays an instrumental role in the sequel.

 Theorem 1. 

Under Assumptions(H.1.),  (C.2.1),  (C.3.1),  (C.4.1),  (C.4.3),  (C.6.)   and  (C.7.)   (for $m=1$), we have, as $n\to \infty$,

$$\begin{gathered} {\displaystyle \underset{\phi K\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\underset{h_{n,1}\le h_K\le h_{n,2}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\underset{\mathit{\theta }\in \mathbf{\Theta }}{sup}\left|{\widehat{r}}_n^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K)-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right|} \\ =O\left(h_{n,2}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left(h_{n,1}\right)}}}}\right). \end{gathered}$$

(17)

The following result gives uniform consistency when the class of functions is unbounded.

 Corollary 1. 

Under Assumptions  (H.1.),  (C.2.1),  (C.3.1),  (C.4.2),  (C.4.3),  (C.6.)   and  (C.7.)   (for $m=1$), we have

$$\begin{gathered} {\displaystyle \underset{\phi K\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\underset{h_{n,1}\le h_K\le h_{n,2}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\underset{\mathit{\theta }\in \mathbf{\Theta }}{sup}\left|{\widehat{r}}_n^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K)-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right|} \\ =O\left(h_{n,2}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left(h_{n,1}\right)}}}}\right). \end{gathered}$$

(18)

3.1.2. UINN Consistency for Functional Regression

Now, we can state the main results of this section concerning the kNN functional regression. Recall the bandwidths $k_{1,n}$ and $k_{2,n}$ given in Condition (C.8.).

 Theorem 2. 

Under Assumptions  (H.1.),  (C.2.1),  (C.3.1),  (C.4.1),  (C.4.3),  (C.6.)   and  (C.7.)   (for $m=1$), if, in addition, Condition  (C.8.)   is satisfied, we have

$$\begin{gathered} {\displaystyle \underset{\phi K\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\underset{\mathit{\theta }\in \mathbf{\Theta }}{sup}\left|{\widehat{r}}_n^{*\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right|} \\ =O\left({\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n}n}}\right)}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left(\mu {\varphi }^{-1}\left(\frac{{\rho }_n{k}_{1,n}}{n}\right)\right)}}}}\right). \end{gathered}$$

The following result gives uniform consistency when the class of functions is unbounded.

 Corollary 2. 

Under Assumptions  (H.1.),  (C.2.1),  (C.3.1),  (C.4.2),  (C.4.3),  (C.6.)   and  (C.7.)   (for $m=1$) and if Condition  (C.8.)   is satisfied, we have

$$\begin{gathered} {\displaystyle \underset{\phi K\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\underset{\mathit{\theta }\in \mathbf{\Theta }}{sup}\left|{\widehat{r}}_n^{*\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right|} \\ =O\left({\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n}n}}\right)}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left(\mu {\varphi }^{-1}\left(\frac{{\rho }_n{k}_{1,n}}{n}\right)\right)}}}}\right). \end{gathered}$$

(19)

3.2. Relative-Error Prediction

Recall that the operator $r(·)$ is typically estimated by minimizing the expected squared loss function $\mathbb{E}\left[{(Y-r\left(X\right))}^2\mid X\right]$. While this loss function is commonly used to assess prediction performance, it may not be suitable in all contexts. Least-squares regression assigns equal weight to all variables, potentially making the results highly sensitive to outliers. In this paper, we address the limitations of classical regression by proposing the estimation of the operator m through the minimization of the mean squared relative error (MSRE), which offers a more suited alternative:

$$\mathbb{E}\left[{\left({\displaystyle \frac{Y-r\left(X\right)}{Y}}\right)}^2\mid X\right],\mathrm{for}Y>0\mathrm{almost}\mathrm{surely}.$$

(20)

This criterion provides a more meaningful measure of prediction performance than the least-squares error, particularly in cases where the range of predicted values is large. Moreover, the solution to (20) can be explicitly formulated as the ratio of the first two conditional inverse moments of Y given X. To develop a regression estimator that optimally minimizes the MSRE, we assume that the first two conditional inverse moments of Y given X, defined as $g_{\gamma }\left(x\right):=\mathbb{E}\left(Y^{-\gamma }\mid X=x\right)$ for $\gamma =1,2$, exist and are finite almost surely. As demonstrated in [143,144,145], the optimal mean squared relative error predictor of Y given X is

$$\breve{r}\left(⟨t,\mathit{\theta }⟩\right)={\displaystyle \frac{\mathbb{E}\left(Y^{-1}\mid ⟨X,\mathit{\theta }⟩=⟨t,\mathit{\theta }⟩\right)}{\mathbb{E}\left(Y^{-2}\mid ⟨X,\mathit{\theta }⟩=⟨t,\mathit{\theta }⟩\right)}},\mathrm{almost}\mathrm{surely}.$$

Therefore, we estimate the regression operator $\breve{r}(·)$, which minimizes the MSRE, using

$${\breve{r}}_n^{\left(1\right)}(⟨t,\mathit{\theta }⟩,h_K)={\displaystyle \frac{{\displaystyle \sum _{i=1}^n{Y}_i^{-1}K\left(\frac{d_{\mathit{\theta }}(t,X_i)}{h_K}\right)}}{{\displaystyle \sum _{i=1}^n{Y}_i^{-2}K\left(\frac{d_{\mathit{\theta }}(t,X_i)}{h_K}\right)}}},\mathrm{for}\mathrm{each}t\in \mathcal{H},$$

(21)

and

$${\breve{r}}_n^{*\left(1\right)}(⟨t,\mathit{\theta }⟩,h_K)={\displaystyle \frac{{\displaystyle \sum _{i=1}^n{Y}_i^{-1}K\left(\frac{d_{\mathit{\theta }}(t,X_i)}{H_{n,k}(t,\mathit{\theta })}\right)}}{{\displaystyle \sum _{i=1}^n{Y}_i^{-2}K\left(\frac{d_{\mathit{\theta }}(t,X_i)}{H_{n,k}(t,\mathit{\theta })}\right)}}},\mathrm{for}\mathrm{each}t\in \mathcal{H}.$$

(22)

By considering the special cases $\phi \left(y\right)=y^{-1}$ and $\phi \left(y\right)=y^{-2}$ in Corollaries 1 and 2, we obtain results that complement the work in [145,146,147].

 Corollary 3. 

Under Assumptions  (H.1),  (C.2.1),  (C.3.1),  (C.4.2),  (C.4.3),  (C.6), and (C.7)   (for $m=1$), we have

$$\begin{gathered} {\displaystyle \underset{K\in {\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{h_{n,1}\le h_K\le h_{n,2}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\underset{\mathit{\theta }\in \mathbf{\Theta }}{sup}\left|{\breve{r}}_n^{\left(1\right)}(⟨t,\mathit{\theta }⟩,h_K)-\breve{r}\left(⟨t,\mathit{\theta }⟩\right)\right|} \\ =O\left(h_{n,2}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}{n\varphi \left(h_{n,1}\right)}}}\right). \end{gathered}$$

(23)

This result has not been previously addressed in the literature.

 Corollary 4. 

Under Assumptions  (H.1.),  (C.2.1),  (C.3.1),  (C.4.2),  (C.4.3),  (C.6.)   and  (C.7.)   (for $m=1$), and if Condition  (C.8.)   is satisfied, we have

$$\begin{gathered} {\displaystyle \underset{K\in {\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\underset{\mathit{\theta }\in \mathbf{\Theta }}{sup}\left|{\breve{r}}_n^{*\left(1\right)}(⟨t,\mathit{\theta }⟩,h_K)-\breve{r}\left(⟨t,\mathit{\theta }⟩\right)\right|} \\ =O\left({\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n}n}}\right)}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left(\mu {\varphi }^{-1}\left(\frac{{\rho }_n{k}_{1,n}}{n}\right)\right)}}}}\right). \end{gathered}$$

(24)

3.3. Uniform Consistency of the  kNN Functional Conditional U-Statistics

In addition to the conditions previously established, the following assumptions are crucial for obtaining exponential inequalities for dependent data which are used later in the proofs:

 (A1) 

Assume that the sequence ${\left\{X_i\right\}}_{i\in {\mathbb{N}}^{*}}$ is strictly stationary, and there exists an absolute constant $\delta \ge 1$ such that, for any $n\ge 1$, the $\beta$-mixing coefficient corresponding to ${\left\{X_i\right\}}_{i\in {\mathbb{N}}^{*}}$ satisfies $\beta \left(n\right)\lesssim n^{-\delta }$.

 (A2) 

Assume that, uniformly, for any integer J such that $1\le J\le m-1$ and arbitrary indices $1\le i_1<\cdots <i_J\le n$, the sequence ${\left\{X_i\right\}}_{i=i_J+1}^{\infty }$, conditional on $X_{i_1},\dots ,X_{i_J}$, satisfies, for the corresponding $\alpha$-mixing coefficient,

$$\alpha \left(n;X_{i_1},\dots ,X_{i_J}\right):=\underset{j\ge i_J+1}{sup}\alpha \left({\sigma }_{i_J+1}^j,{\sigma }_{j+n}^{\infty };\mathbb{P}\left(·\mid X_{i_1},\dots ,X_{i_J}\right)\right)\lesssim exp(-\gamma n),\mathrm{almost}\mathrm{surely},$$

where $\mathbb{P}\left(·\mid X_{i_1},\dots ,X_{i_J}\right)$ represents the conditional probability. In particular, for the $\alpha$-mixing coefficient of the sequence ${\left\{X_i\right\}}_{i\in {\mathbb{N}}^{*}}$, one obtains

$$\alpha \left(n\right)\lesssim exp(-\gamma n).$$

The $\beta$-mixing Condition (A1) plays a crucial role in establishing the asymptotic normality of U-statistics in scenarios where strict Lipschitz continuity of the kernel functions is not present. This condition has been extensively discussed in the literature, including works such as [148,149], and in Remarks 2.2 and 2.3 of [150]. In contrast, Assumption (A2) is generally less restrictive than the $\varphi$-mixing condition. As shown in [150], sequences of finite-state and vector-valued absolutely continuous data with an exponentially decaying $\varphi$-mixing rate satisfy (A2). Nevertheless, it should be noted that this assumption is more restrictive compared to the polynomially decaying mixing rate discussed in [149]. The requirement to compute higher moments of U-statistics for obtaining sharp concentration inequalities underpins the necessity of an “exponentially decaying rate”, which is a commonly imposed condition in the literature to derive concentration inequalities for weakly dependent data, as outlined in [151] and further elaborated in Remarks 2.4 and 2.5 of [150]. This section focuses on investigating the uniform consistency, UIB (uniform in bandwidth) consistency, and UINN (uniform in Nearest Neighbor) consistency of the functional conditional U-statistic defined in (2). Before proceeding, we first introduce some notation. For an interval ${\mathfrak{H}}_n^{\left(m\right)}\subset {\mathbb{R}}_{+}^m\setminus \left\{0\right\}$, we define

$${\mathfrak{H}}_n^{\left(m\right)}:=\prod _{j=1}^m(h_{n,j},h_{n,j}^{\prime }),$$

where

$$0<h_{n,j}<h_{n,j}^{\prime },\underset{n\to \infty }{lim}{h}_{n,j}=\underset{n\to \infty }{lim}{h}_{n,j}^{\prime }=0,\forall j=1,\dots ,m.$$

Subsequently, we denote

$${\tilde{h}}_n=\underset{1\le j\le m}{min}{h}_{n,j},{\tilde{h}}_n^{\prime }=\underset{1\le j\le m}{max}{h}_{n,j}^{\prime }.$$

For any $\mathbf{b}=(b_1,\dots ,b_m)\in {(0,1)}^m$, we denote

$${\mathfrak{H}}_0^{\left(m\right)}:=\prod _{j=1}^m\left(h_{n,j},b_j\right),$$

and

$${\tilde{b}}_0:=\underset{1\le j\le m}{max}{b}_j.$$

For the sake of notational simplicity, when $m=1$, we reduce ${\mathfrak{H}}_n^{\left(1\right)}$ and ${\mathfrak{H}}_0^{\left(1\right)}$ to ${\mathfrak{H}}_n$ and ${\mathfrak{H}}_0$, respectively. This convention similarly applies to other related notations, unless explicitly stated otherwise. We now proceed to define

$$\begin{array}{cc}\hfill \mathbf{X}& :=(X_1,\dots ,X_m)\in {\mathcal{H}}^m,\mathbf{Y}:=(Y_1,\dots ,Y_m)\in {\mathcal{Y}}^m,\hfill \\ \hfill {\mathbf{X}}_i& :=(X_{i_1},\dots ,X_{i_m}),{\mathbf{Y}}_i:=(Y_{i_1},\dots ,Y_{i_m}),\hfill \\ \hfill {\mathbf{h}}_{n,k}\left(\mathbf{t}\right)& :=(H_{n,k}\left(t_1\right),\dots ,H_{n,k}\left(t_m\right)),\mathrm{for}\mathbf{t}=(t_1,\dots ,t_m)\in S_{\mathcal{H}}^m,\hfill \end{array}$$

and

$$G_{\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}}(\mathbf{x},\mathbf{y}):={\displaystyle \frac{{\displaystyle \phi (y_1,\dots ,y_m)\prod _{i=1}^{m}K\left(\frac{d_{{\mathit{\theta }}_i}(x_i,t_i)}{h_i\left(t_i\right)}\right)}}{{\displaystyle \prod _{i=1}^m\mathbb{E}\left[K\left(\frac{d_{{\mathit{\theta }}_i}(X_i,t_i)}{h_i\left(t_i\right)}\right)\right]}}},$$

for $\mathbf{t},\mathbf{x}\in {\mathcal{H}}^m$, $\mathit{\theta }\in {\mathbf{\Theta }}^m$, and $\mathbf{y}\in {\mathcal{Y}}^m$. We also define

$$\mathcal{G}:=\left\{G_{\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}}(·,·):\phi \in {\mathcal{F}}_m,\mathbf{t}=(t_1,\dots ,t_m)\in {\mathcal{H}}^m,\mathit{\theta }\in {\mathbf{\Theta }}^m\right\},$$

and

$${\mathcal{G}}^{\left(p\right)}:=\left\{{\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}}(·,·):\phi \in {\mathcal{F}}_m,\mathbf{t}=(t_1,\dots ,t_m),\mathit{\theta }\in {\mathbf{\Theta }}^m\right\}.$$

The functional conditional U-statistic $u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))$, given by

$$u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))={\displaystyle \frac{(n-m)!}{n!}}\sum _{i\in I(m,n)}{G}_{\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k},\mathit{\theta }}({\mathbf{X}}_i,{\mathbf{Y}}_i),$$

is a classical U-statistic with the U-kernel $G_{\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k},\mathit{\theta }}(\mathbf{x},\mathbf{y})$. The investigation of the uniform consistency of ${\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))$ relative to $r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })$ presents notable challenges due to the randomness inherent in the bandwidth vector ${\mathbf{h}}_{n,k}\left(\mathbf{t}\right)$, which introduces significant technical complexities. To address these difficulties, we begin by analyzing the uniform consistency of ${\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h},\mathit{\theta })$, where $\mathbf{h}=(h_1,\dots ,h_m)\in {\mathfrak{H}}_n^{\left(m\right)}$ represents a fixed multivariate bandwidth independent of $\mathbf{t}$ and k. Our analysis extends to both the uniform consistency and UIB consistency of $u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})$ relative to $\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)$, in cases where $\phi \in {\mathcal{F}}_m$ as well as when $\phi \equiv 1$. Furthermore, we propose a more suitable centering factor than the expectation $\mathbb{E}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)$, defined as follows:

$$\widehat{\mathbb{E}}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)={\displaystyle \frac{\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)}{\mathbb{E}\left(u_n(1,\mathbf{t},\mathit{\theta };\mathbf{h})\right)}}.$$

The second step involves utilizing a general lemma from [51], suitably adapted to our specific framework, following an approach akin to that in [152] (see Appendix A.1). This allows us to rigorously derive the necessary results for the bandwidth ${\mathbf{h}}_{n,k}\left(\mathbf{t}\right)$.

Uniform Consistency and UIB Consistency for a Multivariate Bandwidth

We now present the Uniform In Bandwidth (UIB) results for all $\mathbf{t}\in S_{\mathcal{H}}^m$, $\mathit{\theta }\in {\mathbf{\Theta }}^m$, and $\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}$. To begin, we establish the result concerning the uniform deviation of the estimator $u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h},\mathit{\theta })$ from $\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h},\mathit{\theta })\right)$ under the condition that the class of functions is bounded.

 Theorem 3. 

We suppose that Conditions  (H.1),  (C.3.1),  (C.4.1),  (C.4.4),  (C.6), and  (C.7)   hold. Then, as $n\to \infty$, we have

$$\begin{gathered} {\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)\right|} \\ =O_{a.co}\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}{n\varphi \left({\tilde{h}}_n\right)}}}\right). \end{gathered}$$

(25)

The following theorem establishes the uniform deviation of the estimator $u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h},\mathit{\theta })$ from its expectation $\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h},\mathit{\theta })\right)$, specifically for an unbounded class of functions that satisfy general moment conditions.

 Theorem 4. 

Suppose that Conditions  (H.1),  (C.3.1),  (C.4.2),  (C.4.4),  (C.6), and  (C.7) hold. For all $0<{\tilde{b}}_0<1$, as $n\to \infty$, we have

$$\begin{gathered} {\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_0^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)\right|} \\ =O_{a.co}\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}{n\varphi \left({\tilde{h}}_n\right)}}}\right). \end{gathered}$$

(26)

The following theorem provides the uniform deviation result for the estimate ${\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})$ from $\widehat{\mathbb{E}}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)$ for both bounded and unbounded classes of functions satisfying general moment conditions.

 Theorem 5. 

We suppose that Conditions  (H.1),  (C.3.1),  (C.4.1),  (C.4.4),  (C.6), and  (C.7)   (or alternatively  (H.1),  (C.3.1),  (C.4.2),  (C.4.4),  (C.6), and  (C.7)) hold. For all $0<{\tilde{b}}_0<1$, as $n\to \infty$, we have

$$\begin{gathered} {\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_0^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-\widehat{\mathbb{E}}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)\right|} \\ =O_{a.co}\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}{n\varphi \left({\tilde{h}}_n\right)}}}\right). \end{gathered}$$

(27)

 Theorem 6. 

We suppose that Conditions  (H.1),  (C.2.1),  (C.3.1), and  (C.6)   hold. For all $0<{\tilde{h}}_n^{\prime }<1$, with ${\tilde{h}}_n^{\prime }\to 0$ as $n\to \infty$, we have

$$\underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|\widehat{\mathbb{E}}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)-r^{\left(m\right)}\left(\phi ,\mathbf{t},\mathit{\theta }\right)\right|=O\left({\widetilde{h^{\prime }}}_n^{\gamma }\right).$$

(28)

 Corollary 5. 

Under the assumptions of Theorems 5 and 6, it follows that, as $n\to \infty$,

$$\begin{gathered} {\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|} \\ =O\left({\widetilde{h^{\prime }}}_n^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left({\tilde{h}}_n\right)}}}}\right). \end{gathered}$$

(29)

 Remark 2. 

Following the methodology outlined in [110,112,153], we decompose the U-statistics into distinct components. Certain components can be approximated by U-statistics derived from independent blocks, while others, when conditioned on a particular block, behave as empirical processes of independent blocks. To demonstrate the insignificance of the nonlinear terms, we utilize symmetrization techniques and apply maximal inequalities, as thoroughly discussed in [10,154].

3.4. Uniform Consistency and UINN Consistency of Functional Conditional U-Statistics

Let $0<{\mu }^{*}\le {\nu }^{*}<\infty$ be constants and let ${\rho }_n^{*}\in (0,1)$ be a sequence chosen such that

$$\underset{1\le j\le m}{min}{\mu }_j{\varphi }^{-1}\left({\displaystyle \frac{{\rho }_{n,j}{k}_{1,n}}{n}}\right)={\mu }^{*}{\varphi }^{-1}\left({\displaystyle \frac{{\rho }_n^{*}{k}_{1,n}}{n}}\right)$$

and

$$\underset{1\le j\le m}{max}{\nu }_j{\varphi }^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n,j}n}}\right)={\nu }^{*}{\varphi }^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_n^{*}n}}\right).$$

The following theorem addresses the uniform deviation of the estimator $u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))$ from its expected value $\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right)$, under the assumption that the class of functions is bounded.

 Theorem 7. 

Suppose that Conditions  (H.1.),  (C.3.1),  (C.4.1),  (C.4.4),  (C.6.), and  (C.7.)   are satisfied. Additionally, if assumption  (C.8.)   holds, then, as $n\to \infty$,

$$\begin{array}{c}{\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))-\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right)\right|}\\ \hfill =O_{a.co}\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}{n\varphi \left({\mu }^{*}{\varphi }^{-1}\left(\frac{{\rho }_n^{*}{k}_{1,n}}{n}\right)\right)}}}\right).\end{array}$$

(30)

The following theorem addresses the uniform deviation of the estimator $u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))$ from its expected value $\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right)$ when the class of functions is unbounded but adheres to general moment conditions.

 Theorem 8. 

Suppose that Conditions  (H.1.),  (C.3.1),  (C.4.2),  (C.4.4),  (C.6.), and  (C.7.)   hold. Additionally, if Assumption  (C.8.)   is satisfied, then, as $n\to \infty$,

$$\begin{array}{c}{\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))-\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right)\right|}\\ \hfill =O_{a.co}\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}{n\varphi \left({\mu }^{*}{\varphi }^{-1}\left(\frac{{\rho }_n^{*}{k}_{1,n}}{n}\right)\right)}}}\right).\end{array}$$

(31)

The subsequent results are centered on establishing the uniform consistency of the estimator $u_n$ for both bounded and unbounded classes of functions, ensuring that the consistency holds across varying function spaces.

 Theorem 9. 

Suppose that Conditions  (H.1.),  (C.3.1),  (C.4.1),  (C.4.4),  (C.6.), and  (C.7.)   (or alternatively,  (H.1.),  (C.3.1),  (C.4.2),  (C.4.4),  (C.6.), and  (C.7.)) hold. If, in addition, Assumption  (C.8.)   is satisfied, then, as $n\to \infty$,

$$\begin{array}{c}\hfill {\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))-\widehat{\mathbb{E}}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right)\right|}\\ \hfill =O_{a.co}\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}{n\varphi \left({\mu }^{*}{\varphi }^{-1}\left(\frac{{\rho }_n^{*}{k}_{1,n}}{n}\right)\right)}}}\right).\end{array}$$

(32)

 Theorem 10. 

Suppose that Conditions(H.1.),(C.2.1),(C.3.1), and(C.6.)hold. If, in addition, Assumption(C.8.)is satisfied, then, as $n\to \infty$,

$$\underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|\widehat{\mathbb{E}}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right)-r^{\left(m\right)}\left(\phi ,\mathbf{t},\mathit{\theta }\right)\right|=O\left({\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{{\rho }_n^{*}n}}\right)}^{\gamma }\right).$$

(33)

 Corollary 6. 

Under the assumptions of Theorems 9 and 10, it follows that, as $n\to \infty$,

$$\begin{gathered} {\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|} \\ =O\left({\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{{\rho }_n^{*}n}}\right)}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}{n\varphi \left({\mu }^{*}{\varphi }^{-1}\left(\frac{{\rho }_n^{*}{k}_{1,n}}{n}\right)\right)}}}\right). \end{gathered}$$

(34)

 Remark 3. 

The selection of parameters ${\mu }^{*}$ and ${\rho }_n^{*}$, as defined analogously in Condition  (C.8.), is crucial in determining the convergence rate of the kNN estimator. These parameters are chosen based on the small ball probability function ${\varphi }_{t,\mathit{\theta }}\left(h\right)$, offering flexibility in adjusting the estimator to enhance performance, depending on the specific characteristics of the data.

 Remark 4. 

This study builds upon the foundational work in [49,51,155], introducing several important advancements. One of the primary distinctions of our approach is the minimal restrictions placed on the choice of the kernel function, with only mild conditions required, as detailed later in the paper. However, determining the optimal bandwidth or the number of neighbors presents a more complex challenge. The selection of bandwidth is pivotal, as it directly impacts the estimator’s consistency rate and bias. Our goal is to identify bandwidth and neighbor parameters that strike an optimal balance between bias and variance. Unlike traditional approaches, we propose a more flexible framework in which bandwidth and the number of neighbors are determined based on specific criteria, data characteristics, and local considerations. For a deeper exploration of this topic, readers are referred to works such as [156] and related literature. In this section, we address these challenges within the context of Functional Data Analysis (FDA), particularly under the assumption of dependent data. Specifically, we extend the analysis to single-index models and, for the first time, explore the application of conditional U-statistics in this setting. Our methodology involves decomposing U-statistics into several components: some of these components are approximated by U-statistics derived from independent blocks, while others, when conditioned on a single block, behave as empirical processes of independent blocks. While this decomposition is a powerful tool, it complicates the proof process—a necessary trade-off for extending the methodology to dependent structures. By utilizing this decomposition, we are able to adapt techniques traditionally applied to independent variables, drawing significantly from the results in [51,155]. Furthermore, this paper introduces a novel exponential inequality specifically designed for the dependent data setting, as presented in [150], representing a notable contribution to the existing literature.

In the following section, we provide a comprehensive presentation of the main results concerning uniform central limit theorems for both regression processes and conditional U-processes. This includes detailed statements of the theorems as well as thorough and meticulous explanations of the methodologies employed in proving these results.

4. Uniform Central Limit Theorems

4.1.  kNN Conditional Empirical Process

We define the functional conditional empirical process for univariate bandwidth $h_K$ by

$$\left\{{\nu }_n(\psi ;h_K\mid ⟨t,\mathit{\theta }⟩)=\sqrt{k}\left({\widehat{r}}_n^{*\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩;h_K)-r^{\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩)\right),\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right\},$$

(35)

where ${\widehat{r}}_n^{*\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩;h_K)$ designates (3) when $m=1$, and $r^{\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩)$ refers to Regression Function (1), with

$$\psi (·,·)\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}:={\mathcal{F}}_1{\mathcal{K}}^1=\left\{\phi (·)K\left({\displaystyle \frac{d_{\mathit{\theta }}(·,t)}{h_K}}\right):\phi \in {\mathcal{F}}_1,K\left({\displaystyle \frac{d_{\mathit{\theta }}(·,t)}{h_K}}\right)\in {\mathfrak{K}}_{\mathbf{\Theta }}\right\}.$$

If for $\mathbb{P}\psi =\int \psi d\mathbb{P}$, where $\mathbb{P}$ is the probability measure and for each $(\mathbf{x},\mathbf{y})$

$$\underset{\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\left|\psi (\mathbf{x},\mathbf{y})-\mathbb{P}\psi \right|<\infty ,$$

then $\left\{{\nu }_n(\psi \mid \mathbf{t},\mathit{\theta }):\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right\}$ is a random element with values in $l_{\infty }\left(\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right),$ consisting of all functional ${\nu }_{\infty }$ on $\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}$ such that

$$\underset{\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\left|{\nu }_{\infty }\left(\psi \right)\right|<\infty .$$

Then, it is important to investigate the following weak convergence:

$$\left\{{\nu }_n(\psi \mid ⟨t,\mathit{\theta }⟩):\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}},\mathbf{t}\in \mathbb{I},\mathit{\theta }\in \mathbf{\Theta }\right\}\stackrel{w}{⟶}\left\{\mathbb{G}\left(\psi \right):\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right\}\mathrm{in}{l}_{\infty }\left(\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right).$$

It is well established that weak convergence to a Gaussian limit, with uniformly bounded and uniformly continuous paths (with respect to the ${\parallel ·\parallel }_2$ norm), is equivalent to the combination of finite-dimensional convergence and the existence of a pseudo-metric $d_{\mathrm{p}.\mathrm{m}}$ on $\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}$. This pseudo-metric ensures that $(\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}},d_{\mathrm{p}.\mathrm{m}})$ forms a totally bounded pseudo-metric space:

$$\underset{r\to 0}{lim}\underset{n\to \infty }{lim\; sup}{\mathbb{P}}^{*}\left\{\underset{d_{\mathrm{p}.\mathrm{m}}({\psi }_1,{\psi }_2)⩽r}{sup}\left|{\nu }_n\left(({\psi }_1-{\psi }_2)\mid \mathbf{t}\right)\right|>\epsilon \right\}=0.$$

(36)

Below, we denote $Z\stackrel{d}{=}\mathcal{N}(\mu ,{\Sigma }^2)$ to indicate that the random vector Z follows a normal distribution with mean vector $\mu$ and covariance matrix ${\Sigma }^2$, and use $\stackrel{d}{\to }$ to denote convergence in distribution. The following theorem, adapted from [111] for the context of kNN estimators, aims to investigate central limit theorems for the functional conditional empirical process defined as

$$\left\{{\nu }_n(\psi ;H_{n,k}(t,\mathit{\theta }))\mid ⟨t,\mathit{\theta }⟩)=\sqrt{k}\left({\widehat{r}}_n^{*\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-r^{\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩)\right):\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right\}.$$

(37)

 Theorem 11. 

Let us consider the class of functions $\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}$ and suppose that Conditions  (C.1’.),  (C.1.2),  (C.2.1),  (C.2.2),  (C.3’.),  (C.5.), and  (C.8.)   hold. If the smoothing parameter k satisfies, for all $\gamma >0$,

$$k{\left({\varphi }^{-1}\left({\displaystyle \frac{k}{n}}\right)\right)}^{2\gamma }\to 0asn\to \infty ,$$

we obtain, for $\mathfrak{l}\ge 1:{\psi }_1,\dots ,{\psi }_{\mathfrak{l}}\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}},$

$$\left\{{\nu }_n({\psi }_i;H_{n,k}(t,\mathit{\theta }))\mid ⟨t,\mathit{\theta }⟩):i=1,\dots ,\mathfrak{l}\right\}\stackrel{d}{⟶}\mathcal{N}\left(0,\Sigma \right),$$

where $\Sigma :={\left({\sigma }_{i,j}\right)}_{i,j=1,\dots ,\mathfrak{l}}$ is the covariance matrix with

$${\sigma }_{i,j}:={\displaystyle \frac{{\kappa }_2^{\prime }{r}^{\left(1\right)}({\phi }_i{\phi }_j,⟨t,\mathit{\theta }⟩)-r^{\left(1\right)}({\phi }_i,⟨t,\mathit{\theta }⟩)r^{\left(1\right)}({\phi }_j,t)}{{\displaystyle {\kappa }_1^{\prime }{f}_1\left(t\right)}}}.$$

 Theorem 12. 

We suppose that Conditions  (C.3.1),  (C.4.2)–(C.5.1), and  (C.8.)   hold, and for each $\phi \in \mathcal{F},$

$$\mathbb{E}\left({\phi }^2\left(Y_1\right)\right)<\infty .$$

Then, we have

$$\underset{b\to 0}{lim}\underset{n\to \infty }{lim\; sup}\mathbb{P}\left\{\underset{\underset{{\psi }_1,{\psi }_2\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{\parallel {\psi }_1-{\psi }_2{)\parallel }_p⩽b}}{sup}\left|{\nu }_n\left(\left({\psi }_1-{\psi }_2\right)\mid t\right)\right|>\epsilon \right\}=0.$$

The two previous theorems can be summarized as follows:

 Theorem 13. 

Under Conditions  (C.1’.),  (C.1.2),  (C.2.),  (C.3.1),  (C.3’.),  (C.4.4)(C.5.1),  (C.5.2), and  (C.8.), the process as $n\to \infty$,

$$\left\{{\nu }_n(\psi ;H_{n,k}\left(t\right)))\mid ⟨t,\mathit{\theta }⟩)=\sqrt{k}\left({\widehat{r}}_n^{*\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩;H_{n,k}\left(t\right)))-r^{\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩)\right):\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right\},$$

converges in law to a Gaussian process $\left\{{\mathbb{G}}_n\left(\psi \right):\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right\}$ that admits a version with uniformly bounded and uniformly continuous paths with respect to the ${\parallel ·\parallel }_2-$norm.

 Remark 5. 

We note that additional applications can be derived from Theorem 13, including the conditional distribution, conditional density, and conditional hazard function. However, these and other relevant applications are not addressed here due to space constraints.

 Remark 6. 

Paper [157] introduces a comprehensive semiparametric functional regression model designed for cases where the predictor sets include a combination of functional and multivariate elements within the i.i.d. framework. This model integrates single-index techniques to handle functional predictors and partial–linear methods to address multivariate predictors. Similarly, ref. [90] conducts an in-depth exploration of the functional semiparametric model (FSIM), providing general asymptotic results for the kNN procedure with a focus on uniformity across all model parameters. In contrast, our study distinguishes itself from the aforementioned works in several key ways. First, while prior studies operate within an independent data setting, our research addresses a mixing data framework. Additionally, we extend the analysis by incorporating uniformity considerations for both classes of functions and the parameter “t”, as presented in Corollaries 4 and 1. Allowing function φ to vary within a class of functions enables results for the conditional distribution to emerge as a special case, among others. Another significant difference lies in our inclusion of weak convergence, as detailed in Theorem 13, which hinges on the intricate result of tightness outlined in Theorem 12, accompanied by rigorous and lengthy proofs. Furthermore, while earlier studies primarily focus on the case where $m=1$, extending the results to the nonlinear domain where $m\ge 2$ requires utilizing a range of techniques from empirical process theory and U-processes in the conditional setting poses a more complex and fascinating challenge compared to unconditional processes. This paper presents all results for arbitrary values of m. Finally, we explore the domain of censored data, a subject of independent interest that merits further investigation.

4.2.  kNN Conditional U-Processes

In this section, we focus on investigating the weak convergence of conditional U-processes under the assumption of absolute regular observations. The class of functions under consideration is ${\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m$, as defined in Section 2. The conditional U-process, indexed by ${\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m$, is given by

$${\left\{U_n^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right)):=\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right)\right\}}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m},$$

(38)

The U-empirical process is defined by

$${\mu }_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right)):=\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left\{u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))-\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))\right)\right\}.$$

It is important to note that in order to establish the weak convergence of (38), it is first necessary to address the convergence of (40) below. In fact, we elaborate on certain key details that are utilized later in the analysis. Since Condition (C.6.) holds, for each $\lambda >0$, we have

$$\begin{array}{ccc}\hfill G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}(\mathbf{x},\mathbf{y})& =& G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}(\mathbf{x},\mathbf{y}){\mathbb{1}}_{\left\{{\kappa }^{m}F\left(\mathbf{y}\right)\le \lambda {\left(n/\tilde{\varphi }\left(\mathbf{h}\right)\right)}^{1/2(p-1)}\right\}}\hfill \\ & & +G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}(\mathbf{x},\mathbf{y}){\mathbb{1}}_{\left\{{\kappa }^{m}F\left(\mathbf{y}\right)>\lambda {\left(n/\tilde{\varphi }\left(\mathbf{h}\right)\right)}^{1/2(p-1)}\right\}}\hfill \\ & =:& G_{\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}}^{\left(T\right)}(\mathbf{x},\mathbf{y})+G_{\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}}^{\left(R\right)}(\mathbf{x},\mathbf{y}).\hfill \end{array}$$

(39)

We can write the U-statistic as follows:

$$\begin{array}{ccc}\hfill {\mu }_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))& =& \sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left\{u_n^{\left(m\right)}\left(G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)-\mathbb{E}\left(u_n^{\left(m\right)}\left(G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)\right)\right\}\hfill \\ & & +\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left\{u_n^{\left(m\right)}\left(G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(R\right)}\right)-\mathbb{E}\left(u_n^{\left(m\right)}\left(G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(R\right)}\right)\right)\right\}\hfill \\ & =:& \sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left\{u_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))-\mathbb{E}\left(u_n^{\left(T\right)}\left(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right)\right)\right)\right\}\hfill \\ \hfill & & +\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left\{u_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))-\mathbb{E}\left(u_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))\right)\right\}\hfill \\ & =:& {\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))+{\mu }_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right)).\hfill \end{array}$$

(40)

We refer to the first term on the right-hand side of (40) as the truncated part, denoted by ${\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))$, and the second term as the remainder part, denoted by ${\mu }_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))$. Our initial focus is on ${\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))$. By applying Hoeffding’s decomposition, we obtain

$$\begin{array}{ccc}\hfill u_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))& =& \sum _{p=0}^m{\displaystyle \frac{m!}{(m-p)!}}{u}_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)\hfill \\ & =& \mathbb{E}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\left({\mathbf{X}}^{\prime },{\mathbf{Y}}^{\prime }\right)+\sum _{p=1}^m{\displaystyle \frac{m!}{(m-p)!}}{u}_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right),\hfill \end{array}$$

(41)

where ${\left\{({\mathbf{X}}_i^{\prime },{\mathbf{Y}}_i^{\prime })\right\}}_{i\in \mathbb{N}}$ is a sequence of i.i.d. r.v. with $\mathcal{L}\left({\mathbf{X}}_i^{\prime },{\mathbf{Y}}_i^{\prime }\right)=\mathcal{L}\left({\mathbf{X}}_i,{\mathbf{Y}}_i\right)$ for each i, and ${\mathbf{X}}^{\prime }$ and ${\mathbf{Y}}^{\prime }$ are defined, respectively, as $\mathbf{X}$ and $\mathbf{Y}$. In view of (41), we have

$$\begin{array}{cc}\hfill {\mu }_n^{\left(T\right)}& (\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))\hfill \\ & =\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left\{\mathbb{E}{G}_{\phi ,\mathbf{t},\mathit{\theta }}^{\left(T\right)}({\mathbf{X}}^{\prime },{\mathbf{Y}}^{\prime })+\sum _{p=1}^m{\displaystyle \frac{m!}{(m-p)!}}{u}_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)-\mathbb{E}\left(u_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta })\right)\right\};\hfill \end{array}$$

(42)

the stationarity assumption and some algebras show that

$$\mathbb{E}\left(u_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))\right)=\mathbb{E}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}({\mathbf{X}}^{\prime },{\mathbf{Y}}^{\prime }).$$

Therefore,

$$\begin{array}{ccc}\hfill {\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))& =& \sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left\{\sum _{p=1}^m{\displaystyle \frac{m!}{(m-p)!}}{u}_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)\right\}\hfill \\ & =& \sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left\{m{u}_n^{\left(1\right)}\left({\pi }_{1,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)+\sum _{p=2}^m{\displaystyle \frac{m!}{(m-p)!}}{u}_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)\right\}.\hfill \end{array}$$

(43)

Because of the fact that ${\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}$ is $\mathbb{P}$-canonical, we have to show that

$$\left(\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\sum _{p=2}^m{u}_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)\right)\stackrel{\mathbb{P}}{⟶}0.$$

So, to establish the weak convergence of the U-process ${\left\{{\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))\right\}}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}$, it is enough to show

$$m\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}{u}_n^{\left(1\right)}\left({\pi }_{1,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)\stackrel{w}{⟶}\mathbb{G}\left(\phi \right)in{\ell }_{\infty }\left(m{\mathcal{G}}^{\left(1\right)}\right),$$

where ${\left\{\mathbb{G}\left(\phi \right)\right\}}_{m{\mathcal{G}}^{\left(1\right)}}$ is a Gaussian process indexed by $m{\mathcal{G}}^{\left(1\right)}$, and for $2\le p\le m$,

$${∥\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}{u}_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}^{\left(T\right)}\right)∥}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}\stackrel{\mathbb{P}}{⟶}0.$$

We have to prove, after, that the remaining part is negligible, in the sense that

$${∥\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left\{u_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))-\mathbb{E}\left(u_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))\right)\right\}∥}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}\stackrel{\mathbb{P}}{⟶}0.$$

Nevertheless, when dealing with finite-dimensional convergence, the truncation becomes inconsequential. This implies that establishing the finite-dimensional convergence of ${\mu }_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))$ is equivalent to establishing the convergence of ${\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))$.

 Theorem 14. 

(a)   Under Conditions  (C.1’.),  (C.1.2),  (C.2.),  (C.3’.),  (C.5.1),  (C.5.2)   and if $r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })$ is continuous at $\mathbf{t},$ then, as $n\to \infty$,

$$\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))-\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))\right)\right)\stackrel{d}{⟶}\mathcal{N}\left(0,{\rho }^2\right),$$

(44)

where

$$\begin{array}{ccc}\hfill {\rho }^2& :=& m^2\left({\sigma }_{\mathbf{t},\mathit{\theta }}^2\left(\phi ,\phi \right)-2r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta }){\sigma }_{\mathbf{t},\mathit{\theta }}^2(\phi ,1)+{\left(r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right)}^2{\sigma }_{\mathbf{t},\mathit{\theta }}^2(1,1)\right),\hfill \\ \hfill {\sigma }_{\mathbf{t},\mathit{\theta }}^2\left({\phi }_i,{\phi }_j\right)& :=& \underset{n\to \infty }{lim}\tilde{\varphi }\left(\mathbf{h}\right)\mathbb{E}\left({\pi }_{1,m}{G}_{{\phi }_i,\mathbf{t},\mathit{\theta },\mathbf{h}},{\pi }_{1,m}{G}_{{\phi }_j,\mathbf{t},\mathit{\theta },\mathbf{h}}\right).\hfill \end{array}$$

(45)

(b)   If, in addition, the smoothing parameter k satisfies Condition  (C.8.), then we have, as $n\to \infty$,

$$\sqrt{k}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))-\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right)\right)\stackrel{d}{⟶}\mathcal{N}\left(0,{\rho }^2\right),$$

(46)

where ${\rho }^2$ is defined as in (45), with

$${\sigma }_{\mathbf{t},\mathit{\theta }}^2\left({\phi }_i,{\phi }_j\right):=\underset{n\to \infty }{lim}\left({\displaystyle \frac{k}{n}}\right)\mathbb{E}\left({\pi }_{1,m}{G}_{{\phi }_i,\mathbf{t},\mathit{\theta },\mathbf{h}},{\pi }_{1,m}{G}_{{\phi }_j,\mathbf{t},\mathit{\theta },\mathbf{h}}\right).$$

 Corollary 7. 

Under Conditions  (C.1’.),  (C.1.2),  (C.2.),  (C.3’.),  (C.5.)   and if $n{\widetilde{h^{\prime }}}_n^{2\gamma }\tilde{\varphi }\left(\mathbf{h}\left(\mathbf{t}\right)\right)\to 0$ as $n\to \infty$, then we infer that

$$\sqrt{k}\left\{{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right\}\stackrel{d}{⟶}\mathcal{N}\left(0,{\rho }^2\right).$$

(47)

 Theorem 15. 

Under Conditions  (C.1’.),  (C.1.2),  (C.2.),  (C.3’.),  (C.5.1),  (C.5.2),  (C.8.)   and $n{\widetilde{h^{\prime }}}_n^{2\gamma }{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)\to 0$ as $n\to \infty$, we let ${\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m$ be a measurable VC-subgraph class of functions from $({\mathcal{X}}^m,{\mathcal{Y}}^m)⟶\mathbb{R}$ such that Condition  (C.4.2)   is satisfied and, if the β-coefficients of the mixing stationary sequence ${\left\{(X_i,Y_i)\right\}}_{i\in {\mathbb{N}}^{*}}$ fulfill

$${\beta }_s{S}^r\to 0, as s\to \infty$$

(48)

for some $r>1$, then ${\left\{U_n^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right\}}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}$ converges in law to a Gaussian process ${\left\{\mathbb{G}\left(\phi \right)\right\}}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}$ which has a version with uniformly bounded and uniformly continuous paths with respect to the ${\parallel ·\parallel }_2-$norm.

 Remark 7. 

The distinctive characteristics of kNN-based estimators introduce inherent complexities in analyzing their asymptotic properties. Specifically, the random variable $H_{n,k}\left(X\right)$, which depends on $(X_1,\dots ,X_n)$, adds technical intricacies to the proofs. A crucial observation is that the random elements involved in (2) for $m=1$ cannot be simply decomposed into sums of independent variables, as is typically the case with kernel-based estimators. This challenge requires more advanced probabilistic techniques, extending beyond the scope of standard limit theorems for sums of i.i.d. variables. Moreover, the direct application of Hoeffding’s decomposition—a fundamental tool for studying U-statistics—to (2) is not feasible in this context. The first step in proving Theorem 15 involves extending the work in [110,112] to address the multivariate bandwidth problem. It is also important to note that this paper extends the framework of [155] to single-index models. Additionally, we explore new applications, including the set-indexed conditional U-statistic, Kendall’s rank correlation coefficient, and time series prediction based on a continuous set of past values. Another layer of complexity arises from the fact that certain maximal inequalities and symmetrization techniques, as presented in [10,154], cannot be directly applied within this framework. This limitation necessitates a lengthier proof, particularly in establishing the equicontinuity of the empirical processes, a critical component in our analysis.

 Remark 8. 

It is straightforward to adapt the proofs of our results to demonstrate that they remain valid when the entropy condition is replaced by the bracketing condition. Specifically, for some $C_0>0$ and $v_0>0$,

$${\mathcal{N}}_{\left[\right]}\left(ϵ,{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m,L_2\left(\mathbb{P}\right)\right)\le C_0{ϵ}^{-v_0},0<ϵ<1.$$

For the definition of ${\mathcal{N}}_{\left[\right]}\left(ϵ,{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m,L_2\left(\mathbb{P}\right)\right)$, refer to p. 270 of [3].

 Remark 9. 

We can also consider the scenario where $\mathbf{\Theta }={\mathbf{\Theta }}_n$, with ${\mathbf{\Theta }}_n$ satisfying the condition $card\left({\mathbf{\Theta }}_n\right)=n^{\alpha }$, where $\alpha >0$. For each $\mathit{\theta }\in {\mathbf{\Theta }}_n$, we assume ${⟨\mathit{\theta }-{\mathit{\theta }}_0,\mathit{\theta }-{\mathit{\theta }}_0⟩}^{1/2}\le C_7{b}_n$, where $b_n$ converges to zero, as discussed in [90].

The set of functional directions, ${\mathbf{\Theta }}_n$, is constructed using an approach similar to that of [81,90], as outlined below:

(i) Each direction $\mathit{\theta }\in {\mathbf{\Theta }}_n$ is derived from a $d_n$-dimensional space spanned by $\mathrm{B}$-spline basis functions, denoted by $\left\{e_1(·),\dots ,e_{d_n}(·)\right\}$. Thus, we represent directions as

$$\begin{array}{c}\hfill \mathit{\theta }(·)=\sum _{j=1}^{d_n}{\alpha }_j{e}_j(·),where\left({\alpha }_1,\dots ,{\alpha }_{d_n}\right)\in \mathcal{V}\end{array}$$

(49)

(ii) The set of coefficient vectors in (49), denoted by $\mathcal{V}$, is generated through the following steps:

 Step 1 

For each $\left({\beta }_1,\dots ,{\beta }_{d_n}\right)\in {\mathcal{C}}^{d_n}$, where $\mathcal{C}=\left\{c_1,\dots ,c_J\right\}\subset {\mathbb{R}}^J$ represents a set of J ’seed-coefficients’, we construct the initial functional direction as

$${\mathit{\theta }}_{init}(·)=\sum _{j=1}^{d_n}{\beta }_j{e}_j(·).$$

 Step 2 

For each ${\mathit{\theta }}_{init}$ from   Step 1   that satisfies ${\mathit{\theta }}_{init}\left(t_0\right)>0$, where $t_0$ is a fixed value in the domain of ${\mathit{\theta }}_{init}(·)$, we compute $⟨{\mathit{\theta }}_{init},{\mathit{\theta }}_{init}⟩$ and normalize to obtain $\left({\alpha }_1,\dots ,{\alpha }_{d_n}\right)=\left({\beta }_1,\dots ,{\beta }_{d_n}\right)/{⟨{\mathit{\theta }}_{init},{\mathit{\theta }}_{init}⟩}^{1/2}$.

 Step 3 

We define $\mathcal{V}$ as the collection of vectors $({\alpha }_1,\dots ,{\alpha }_{d_n})$ obtained inStep 2. Consequently, the final set of admissible functional directions is given by

$${\mathbf{\Theta }}_n=\left\{\mathit{\theta }(·)=\sum _{j=1}^{d_n}{\alpha }_j{e}_j(·);({\alpha }_1,\dots ,{\alpha }_{d_n})\in \mathcal{V}\right\}.$$

5. Some Potential Applications

Although only six examples are presented here, they serve as archetypes for a wide range of problems that can be explored using similar methods.

5.1. Set Indexed Conditional U-Statistics

Our objective is to analyze the relationships between X and Y by estimating functional operators associated with the conditional distribution of Y given X. This includes estimating operators such as the regression operator for $C_1\times \cdots \times C_m:=\tilde{\mathbb{C}}$ within a class of sets ${\mathcal{C}}^m$. Specifically, for $\mathbf{t}\in {\mathcal{H}}^m$, the conditional distribution is defined as

$${\mathbb{G}}^{\left(m\right)}(C_1\times \cdots \times C_m\mid \mathbf{t},\mathit{\theta })=\mathbb{E}\left(\prod _{i=1}^m{\mathbb{1}}_{\{Y_i\in C_i\}}\mid (⟨{\mathit{\theta }}_1,X_1⟩,\dots ,⟨{\mathit{\theta }}_m,X_m⟩)=(⟨{\mathit{\theta }}_1,{\mathbf{t}}_1⟩,\dots ,⟨{\mathit{\theta }}_m,{\mathbf{t}}_m⟩)\right).$$

We define the metric entropy of the class of sets $\mathcal{C}$ by considering the covering number for each $\epsilon >0$ given by

$$\mathcal{N}(\epsilon ,\mathcal{C},{\mathbb{G}}^{\left(1\right)}(·\mid ⟨\mathit{\theta },x⟩))=inf\{n\in \mathbb{N}:\exists C_1,\dots ,C_n\in \mathcal{C},\forall C\in \mathcal{C},\exists 1\le i,j\le n:C_i\subset C\subset C_j \mathrm{and} {\mathbb{G}}^{\left(1\right)}(C_j\setminus C_i\mid x)<\epsilon \}.$$

The logarithm of this covering number is referred to as the metric entropy of the class $\mathcal{C}$ with respect to ${\mathbb{G}}^{\left(1\right)}(·\mid ⟨\mathit{\theta },x⟩)$. Estimates for such covering numbers are available for various classes of sets (see, e.g., [158]). Often, we assume that the behavior of the covering number is governed by powers of ${\epsilon }^{-1}$. Specifically, Condition $\left(R_{\gamma }\right)$ holds if

$$log\mathcal{N}(\epsilon ,\mathcal{C},{\mathbb{G}}^{\left(1\right)}(·\mid ⟨\mathit{\theta },x⟩))\le H_{\gamma }\left(\epsilon \right),\mathrm{for}\mathrm{all}\epsilon >0,$$

where

$$H_{\gamma }\left(\epsilon \right)=\left\{\begin{array}{cc}log\left(A\epsilon \right)\hfill & \mathrm{if}\gamma =0,\hfill \\ A{\epsilon }^{-\gamma }\hfill & \mathrm{if}\gamma >0,\hfill \end{array}\right.$$

for some constants A and $r>0$. As shown in [159], Condition $\left(R_0\right)$ with $\gamma =0$ is satisfied for intervals, rectangles, balls, ellipsoids, and classes formed through finite applications of set operations like unions, intersections, and complements. In the case of convex sets in ${\mathbb{R}}^d$ for $d\ge 2$, the condition holds with $\gamma =(d-1)/2$. Further examples of sets satisfying $\left(R_{\gamma }\right)$ for $\gamma >0$ are found in [158]. An important application of this theory is the estimation of ${\mathbb{G}}^{\left(m\right)}(C_1\times \cdots \times C_m\mid \mathbf{t},\mathit{\theta })$ using the following empirical estimator:

$${\widehat{\mathbb{G}}}_n^{\left(m\right)}(\tilde{\mathbb{C}},\mathbf{t},\mathit{\theta })={\displaystyle \frac{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}\prod _{j=1}^m{\mathbb{1}}_{\{Y_{i_j}\in C_j\}}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}\left(t_1\right)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}\left(t_m\right)}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}\left(t_1\right)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}\left(t_m\right)}\right)}}}.$$

Using Corollary 6, we conclude that

$$\underset{\tilde{\mathbb{C}}\times \tilde{K}\in {\mathcal{C}}^m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|{\widehat{\mathbb{G}}}_n^{\left(m\right)}(\tilde{\mathbb{C}},\mathbf{t},\mathit{\theta })-{\mathbb{G}}^{\left(m\right)}(\tilde{\mathbb{C}}\mid \mathbf{t},\mathit{\theta })\right|⟶0,\mathrm{almost}\mathrm{completely}.$$

 Remark 10. 

An alternative perspective involves considering a compact set $\mathbf{J}\subset {\mathbb{R}}^{dm}$, where

$${\mathbf{G}}^{\left(m\right)}(y_1,\dots ,y_m\mid \mathbf{t},\mathit{\theta })=\mathbb{E}\left(\prod _{i=1}^m{\mathbb{1}}_{\{Y_i\le y_i\}}\mid (⟨{\mathit{\theta }}_1,X_1⟩,\dots ,⟨{\mathit{\theta }}_m,X_m⟩)=(⟨{\mathit{\theta }}_1,{\mathbf{t}}_1⟩,\dots ,⟨{\mathit{\theta }}_m,{\mathbf{t}}_m⟩)\right).$$

The estimator for ${\mathbf{G}}^{\left(m\right)}(y_1,\dots ,y_m\mid \mathbf{t},\mathit{\theta })$ is given by

$${\widehat{\mathbf{G}}}_n^{\left(m\right)}(\mathbf{y},\mathbf{t},\mathit{\theta })={\displaystyle \frac{{\sum }_{(i_1,\dots ,i_m)\in I(m,n)}L\left(\frac{t_1-Y_{i_1}}{{\tilde{H}}_{n,\tilde{k}}\left(t_1\right)}\right)\cdots L\left(\frac{t_m-Y_{i_m}}{{\tilde{H}}_{n,\tilde{k}}\left(t_m\right)}\right)K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}{{\sum }_{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}.$$

As $n\to \infty$, Corollary 6 implies that

$$\underset{\tilde{K}\in {\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k,\tilde{k}\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\underset{\mathbf{y}\in \mathbf{J}}{sup}\left|{\widehat{\mathbf{G}}}_n^{\left(m\right)}(\mathbf{y},\mathbf{t},\mathit{\theta })-{\mathbf{G}}^{\left(m\right)}(\mathbf{y}\mid \mathbf{t},\mathit{\theta })\right|⟶0,\mathrm{almost}\mathrm{completely}.$$

5.2. Kendall Rank Correlation Coefficient

To assess the independence between the one-dimensional random variables $Y_1$ and $Y_2$, Kendall [160] proposed a method based on a U-statistic, $K_n$, with the kernel function:

$$\phi \left(\left(s_1,t_1\right),\left(s_2,t_2\right)\right)={\mathbb{1}}_{\left\{(s_2-s_1)(t_2-t_1)>0\right\}}-{\mathbb{1}}_{\left\{(s_2-s_1)(t_2-t_1)\le 0\right\}}.$$

The rejection region for the independence test is defined as $\left\{\sqrt{n}{K}_n>\gamma \right\}$. In this example, we extend Kendall’s method to the multivariate setting. Specifically, to test the conditional independence of $\mathit{\xi }$ and $\mathit{\eta }$ (where $Y=(\mathit{\xi },\mathit{\eta })$) given X, we introduce a method based on the conditional U-statistic:

$${\widehat{r}}_n^{\left(2\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))={\displaystyle \frac{{\displaystyle \sum _{i\ne j}^n\phi (Y_i,Y_j)K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_i)}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)K\left(\frac{d_{{\mathit{\theta }}_2}(t_2,X_j)}{H_{n,k}(t_2,{\mathit{\theta }}_2)}\right)}}{{\displaystyle \sum _{i\ne j}^{n}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_i)}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)K\left(\frac{d_{{\mathit{\theta }}_2}(t_2,X_j)}{H_{n,k}(t_2,{\mathit{\theta }}_2)}\right)}}},$$

where $\mathbf{t}=(t_1,t_2)\in \mathbb{I}\subset {\mathbb{R}}^2$ and $\phi (·)$ is Kendall’s kernel. We assume that $\mathit{\xi }$ and $\mathit{\eta }$ are $d_1$- and $d_2$-dimensional random vectors, respectively, with $d_1+d_2=d$. Suppose we have observations $Y_1,\dots ,Y_n$ of $(\mathit{\xi },\mathit{\eta })$, and we wish to test the following hypothesis:

$$H_0:\mathit{\xi }\mathrm{and}\mathit{\eta }\mathrm{are}\mathrm{conditionally}\mathrm{independent}\mathrm{given}X\mathrm{vs}{H}_a:H_0\mathrm{is}\mathrm{false}.$$

Let $\mathbf{a}=({\mathbf{a}}_1,{\mathbf{a}}_2)\in {\mathbb{R}}^d$ be a unit vector, where ${\mathbf{a}}_1\in {\mathbb{R}}^{d_1}$, ${\mathbf{a}}_2\in {\mathbb{R}}^{d_2}$, and $\parallel \mathbf{a}\parallel =1$. Let $F(·)$ and $G(·)$ be the distribution functions of $\mathit{\xi }$ and $\mathit{\eta }$, respectively. Assume that $F^{{\mathbf{a}}_1}(·)$ and $G^{{\mathbf{a}}_2}(·)$ are continuous for any unit vector $\mathbf{a}=({\mathbf{a}}_1,{\mathbf{a}}_2)$, where $F^{{\mathbf{a}}_1}\left(t\right)=\mathbb{P}({\mathbf{a}}_1^{\top }\mathit{\xi }<t)$ and $G^{{\mathbf{a}}_2}\left(t\right)=\mathbb{P}({\mathbf{a}}_2^{\top }\mathit{\eta }<t)$. Here, ${\mathbf{a}}_1^{\top }$ denotes the transpose of the vector ${\mathbf{a}}_1$. For the case where $n=2$, let $Y^{\left(1\right)}=({\mathit{\xi }}^{\left(1\right)},{\mathit{\eta }}^{\left(1\right)})$ and $Y^{\left(2\right)}=({\mathit{\xi }}^{\left(2\right)},{\mathit{\eta }}^{\left(2\right)})$, where ${\mathit{\xi }}^{\left(i\right)}\in {\mathbb{R}}^{d_1}$ and ${\mathit{\eta }}^{\left(i\right)}\in {\mathbb{R}}^{d_2}$ for $i=1,2$. Define

$${\phi }^a\left(Y^{\left(1\right)},Y^{\left(2\right)}\right)=\phi \left(\left({\mathbf{a}}_1^{\top }{\mathit{\xi }}^{\left(1\right)},{\mathbf{a}}_2^{\top }{\mathit{\eta }}^{\left(1\right)}\right),\left({\mathbf{a}}_1^{\top }{\mathit{\xi }}^{\left(2\right)},{\mathbf{a}}_2^{\top }{\mathit{\eta }}^{\left(2\right)}\right)\right).$$

By applying Corollary 6, conclude that as $n\to \infty$,

$$\underset{\phi \tilde{K}\in {\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^2}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^2}{sup}\left|{\widehat{r}}_n^{*\left(2\right)}({\phi }^a,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))-r^{\left(2\right)}({\phi }^a,\mathbf{t},\mathit{\theta })\right|⟶0,\mathrm{almost}\mathrm{completely}.$$

5.3. Discrimination Problems

We now apply the results to the discrimination problem outlined in Section 3 of [161], with reference to [162]. We adopt similar notation and framework. We let $\phi (·)$ be a function that takes a finite number of values, say $1,\dots ,M$. The sets

$$A_j=\left\{({\mathbf{y}}_1,\dots ,{\mathbf{y}}_k):\phi ({\mathbf{y}}_1,\dots ,{\mathbf{y}}_k)=j\right\},1\le j\le M,$$

partition the feature space. Predicting the value of $\phi ({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_k)$ is equivalent to predicting the partition set to which $({\mathbf{Y}}_1,\dots ,{\mathbf{Y}}_k)$ belongs. For any discrimination rule g, we have the following inequality:

$$\mathbb{P}(g\left(⟨\mathbf{X},\mathit{\theta }⟩\right)=\phi \left(\mathbf{Y}\right))\le \sum _{j=1}^M{\int }_{\mathbf{t}:g\left(\mathbf{t}\right)=j}max{\mathfrak{M}}^j\left(⟨\mathbf{t},\mathit{\theta }⟩\right)d\mathbb{P}\left(\mathbf{t}\right),$$

where

$${\mathfrak{M}}^j\left(⟨\mathbf{t},\mathit{\theta }⟩\right)=\mathbb{P}(\phi \left(\mathbf{Y}\right)=j\mid ⟨\mathbf{X},\mathit{\theta }⟩=⟨\mathbf{t},\mathit{\theta }⟩),\mathbf{t}\in {\mathcal{X}}^m.$$

This inequality becomes an equality if

$$g_0\left(⟨\mathbf{t},\mathit{\theta }⟩\right)=arg\underset{1\le j\le M}{max}{\mathfrak{M}}^j\left(⟨\mathbf{t},\mathit{\theta }⟩\right),$$

where $g_0(·)$ is the Bayes rule. The associated probability of error, known as the Bayes risk, is given by

$${\mathbf{L}}^{*}=1-\mathbb{P}(g_0\left(⟨\mathbf{X},\mathit{\theta }⟩\right)=\phi \left(\mathbf{Y}\right))=1-\mathbb{E}\left\{\underset{1\le j\le M}{max}{\mathfrak{M}}^j\left(⟨\mathbf{t},\mathit{\theta }⟩\right)\right\}.$$

Each of the unknown functions ${\mathfrak{M}}^j$ can be consistently estimated using the methods discussed earlier. For $1\le j\le M$, the estimator is defined as

$${\mathfrak{M}}_n^j(\mathbf{t},\mathit{\theta })={\displaystyle \frac{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}\mathbb{1}\{\phi (Y_{i_1},\dots ,Y_{i_k})=j\}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}}.$$

We define the estimated rule as

$$g_{0,n}(\mathbf{t},\mathit{\theta })=arg\underset{1\le j\le M}{max}{\mathfrak{M}}_n^j(\mathbf{t},\mathit{\theta }),$$

and introduce the estimated probability of error:

$${\mathbf{L}}_n^{*}=\mathbb{P}(g_{0,n}(\mathbf{X},\mathit{\theta })\ne \phi \left(\mathbf{Y}\right)).$$

The discrimination rule $g_{0,n}(·)$ is asymptotically Bayes risk consistent, meaning that as $n\to \infty$,

$${\mathbf{L}}_n^{*}\to {\mathbf{L}}^{*}.$$

This result follows from Corollary 6 and the fact that

$$\left|{\mathbf{L}}^{*}-{\mathbf{L}}_n^{*}\right|\le 2\mathbb{E}\left[\underset{1\le j\le M}{max}\left|{\mathfrak{M}}_n^j(\mathbf{X},\mathit{\theta })-{\mathfrak{M}}^j(\mathbf{X},\mathit{\theta })\right|\right].$$

5.4. Generalized U-Statistics

The extension to the case of multiple samples is straightforward. Consider k independent collections of independent observations:

$$\left\{\left(X_1^{\left(1\right)},Y_1^{\left(1\right)}\right),\left(X_2^{\left(1\right)},Y_2^{\left(1\right)}\right),\dots \right\},\dots ,\left\{\left(X_1^{\left(k\right)},Y_1^{\left(k\right)}\right),\left(X_2^{\left(k\right)},Y_2^{\left(k\right)}\right),\dots \right\}.$$

Let, for $\mathbf{t}\in {\mathcal{X}}^{(m_1+\cdots +m_k)}$,

$$r^{(m,k)}(\phi ,\mathbf{t},\mathit{\theta })=r^{(m,k)}(\phi ,{\mathbf{t}}_1,\dots ,{\mathbf{t}}_k,{\mathit{\theta }}_1,\dots ,{\mathit{\theta }}_k)$$

be defined as

$$\mathbb{E}\left(\phi \left(Y_1^{\left(1\right)},\dots ,Y_{m_1}^{\left(1\right)};\dots ;Y_1^{\left(k\right)},\dots ,Y_{m_k}^{\left(k\right)}\right)\mid ⟨\left(X_1^{\left(j\right)},\dots ,X_{m_j}^{\left(j\right)}\right),{\mathit{\theta }}_j⟩=⟨{\mathbf{t}}_j,{\mathit{\theta }}_j⟩,j=1,\dots ,k\right),$$

where $\phi$ is assumed, without loss of generality, to be symmetric within each of the k blocks of its arguments. The conditional U-statistic for estimating $r^{(m,k)}(\phi ,\mathbf{t})$, corresponding to the kernel $\phi$ and assuming $n_1\ge m_1,\dots ,n_k\ge m_k$, is given by

$${\widehat{r}}_n^{(m,k)}(\phi ,\mathbf{t},\mathit{\theta },{\tilde{\mathbf{h}}}_{n,k}\left(\mathbf{t}\right))={\displaystyle \frac{{\sum }_c\phi \left(Y_{i_{11}}^{\left(1\right)},\dots ,Y_{i_{1m_1}}^{\left(1\right)};\dots ;Y_{i_{k1}}^{\left(k\right)},\dots ,Y_{i_{k{m}_k}}^{\left(k\right)}\right)\mathbf{K}\left(X_{i_{11}}^{\left(1\right)},\dots ,X_{i_{k{m}_k}}^{\left(k\right)}\right)}{{\sum }_c\mathbf{K}\left(X_{i_{11}}^{\left(1\right)},\dots ,X_{i_{k{m}_k}}^{\left(k\right)}\right)}},$$

where

$$\mathbf{K}\left(X_{i_{11}}^{\left(1\right)},\dots ,X_{i_{k{m}_k}}^{\left(k\right)}\right)=\prod _{j=1}^{k}K\left({\displaystyle \frac{d_{{\mathit{\theta }}_1^{\left(j\right)}}(t_1^{\left(j\right)},X_{i_{j1}}^{\left(j\right)})}{H_{n,k}\left(t_1\right)}}\right)\cdots K\left({\displaystyle \frac{d_{{\mathit{\theta }}_{m_j}^{\left(j\right)}}(t_{m_j}^{\left(j\right)},X_{i_{j{m}_j}}^{\left(j\right)})}{H_{n,k}\left(t_m\right)}}\right),$$

and

$${\tilde{\mathbf{h}}}_{n,k}\left(\mathbf{t}\right)=\left(H_{n,k}\left(t_1^{\left(1\right)}\right),\dots ,H_{n,k}\left(t_{m_1}^{\left(1\right)}\right),\dots ,H_{n,k}\left(t_1^{\left(k\right)}\right),\dots ,H_{n,k}\left(t_{m_k}^{\left(k\right)}\right)\right)\in {\mathcal{H}}_n^{m_1+\cdots +m_k}\subset {\mathbb{R}}_{+}^{m_1+\cdots +m_k}$$

is a vector of positive random variables depending on the sets

$$\left\{X_1^{\left(1\right)},\dots ,X_{n_1}^{\left(1\right)}\right\},\dots ,\left\{X_1^{\left(k\right)},\dots ,X_{n_k}^{\left(k\right)}\right\}.$$

Here, $\left\{i_{j1},\dots ,i_{j{m}_j}\right\}$ denotes a set of $m_j$ distinct elements from the set $\{1,2,\dots ,n_j\}$ for $1\le j\le k$, and ${\sum }_c$ represents summation over all such combinations. The extension of Hoeffding’s treatment of one-sample U-statistics to the multi-sample case is due to [163] and [164]. Using Corollary 6, it is possible to infer that

$$\underset{1\le j\le k}{max}\underset{k_{1,n}^j\le k^j\le k_{2,n}^j}{sup}\underset{\mathbf{t}\in {\mathcal{S}}_{\mathcal{X}}^{m_1+\cdots +m_k}}{sup}\left|{\widehat{r}}_n^{(m,k)}(\phi ,\mathbf{t},\mathit{\theta },{\tilde{\mathbf{h}}}_{n,k}\left(\mathbf{t}\right))-r^{(m,k)}(\phi ,\mathbf{t},\mathit{\theta })\right|⟶0,\mathrm{almost}\mathrm{completely}.$$

5.5. Conditional U-Statistics for Censored Data

We consider a triple $(Y,C,X)$ of random variables defined on $\mathbb{R}\times \mathbb{R}\times \mathcal{H}$, where Y is the variable of interest, C is a censoring variable, and X is a concomitant variable. We work with a sample ${\left\{(Y_i,C_i,X_i)\right\}}_{1\le i\le n}$ consisting of independent and identically distributed replications of $(Y,C,X)$, with $n\ge 1$. In the right-censorship model, the pairs $(Y_i,C_i)$, $1\le i\le n$ are not directly observed. Instead, the available information is provided by $Z_i:=min\{Y_i,C_i\}$ and ${\Delta }_i:=\mathbb{1}\{Y_i\le C_i\}$ for $1\le i\le n$. Therefore, the observed sample is

$${\mathcal{D}}_n=\{(Z_i,{\Delta }_i,X_i),i=1,\dots ,n\}.$$

Such censoring occurs frequently in practical settings, for example, in survival data from clinical trials or failure time data in reliability studies. Often, even under well-controlled conditions, samples are incomplete. For instance, clinical data tracking survival from many diseases are typically censored due to competing risks, like death from other causes. In the following, we impose several assumptions on the distribution of $(X,Y)$. For $-\infty <t<\infty$, we let

$$F_Y\left(t\right)=\mathbb{P}(Y\le t),G\left(t\right)=\mathbb{P}(C\le t),\mathrm{and}H\left(t\right)=\mathbb{P}(Z\le t),$$

denote the right-continuous distribution functions of Y, C, and Z, respectively. For any right-continuous distribution function $\mathfrak{L}$ defined on $\mathbb{R}$, we let

$$T_{\mathfrak{L}}=sup\{t\in \mathbb{R}:\mathfrak{L}\left(t\right)<1\},$$

be the upper point of the corresponding distribution. Now, we consider a pointwise measurable class $\mathcal{F}$ of real-valued measurable functions defined on $\mathbb{R}$ and assume that $\mathcal{F}$ is of VC type. We recall the regression function of $\psi \left(Y\right)$ evaluated at $⟨X,\mathit{\theta }⟩=⟨t,\mathit{\theta }⟩$, for $\psi \in \mathcal{F}$ and $t\in \mathcal{H}$, given by

$$r^{\left(1\right)}(\psi ,t,\mathit{\theta })=\mathbb{E}(\psi \left(Y_i\right)\mid ⟨X_i,\mathit{\theta }⟩=⟨t,\mathit{\theta }⟩),$$

in the case where Y is right-censored. To estimate $r^{\left(1\right)}(\psi ,·)$, we use the Inverse Probability of Censoring Weighted (I.P.C.W.) estimators, which have gained popularity in the censored data literature (see [165,166]). The key concept behind I.P.C.W. estimators involves introducing the real-valued function ${\Phi }_{\psi }(·,·)$ defined on ${\mathbb{R}}^2$ as

$${\Phi }_{\psi }(y,c)={\displaystyle \frac{\mathbb{1}\{y\le c\}\psi (y\wedge c)}{1-G(y\wedge c)}}.$$

(50)

This function incorporates both the observed data and the censoring mechanism to provide consistent estimates in the presence of censoring. Assuming that function $G(·)$ is known, we observe that

$${\Phi }_{\psi }(Y_i,C_i)={\displaystyle \frac{{\Delta }_i\psi \left(Z_i\right)}{1-G\left(Z_i\right)}}$$

is directly observable for every $1\le i\le n$. Furthermore, under Assumption ($\mathbf{I}$) outlined below,

($\mathbf{I}$)

C and $(Y,\mathbf{X})$ are independent.

We have

$$\begin{array}{ccc}\hfill r^{\left(1\right)}({\Phi }_{\psi },t,\mathit{\theta })& :=& \mathbb{E}({\Phi }_{\psi }(Y_i,C_i)\mid ⟨X_i,\mathit{\theta }⟩=⟨t,\mathit{\theta }⟩)\hfill \\ \hfill & =& \mathbb{E}\left\{{\displaystyle \frac{\mathbb{1}\{Y_i\le C_i\}\psi \left(Z_i\right)}{1-G\left(Z_i\right)}}\mid ⟨X_i,\mathit{\theta }⟩=⟨t,\mathit{\theta }⟩\right\}\hfill \\ \hfill & =& \mathbb{E}\left\{{\displaystyle \frac{\psi \left(Y_i\right)}{1-G\left(Y_i\right)}}\mathbb{E}(\mathbb{1}\{Y_i\le C_i\}\mid {\mathbf{X}}_i,Y_i)\mid ⟨X_i,\mathit{\theta }⟩=⟨t,\mathit{\theta }⟩\right\}\hfill \\ & =& r^{\left(1\right)}(\psi ,t,\mathit{\theta }).\hfill \end{array}$$

(51)

Therefore, any estimate of $r^{\left(1\right)}({\Phi }_{\psi },·)$, which is constructed based on fully observed data, also serves as an estimate for $r^{\left(1\right)}(\psi ,·)$. This key property allows most statistical procedures used to estimate the regression function in the uncensored case to be naturally extended to the censored case. For example, kernel-type estimates can be readily constructed. For $\mathbf{x}\in \mathcal{I}$, $h\ge 0$, and $1\le i\le n$, we define

$$\begin{array}{c}\hfill {\overline{\omega }}_{n,h,j}^{\left(1\right)}\left(x\right):=K\left({\displaystyle \frac{d_{{\mathit{\theta }}_k}(x,X_{j,n})}{h_K\left(x\right)}}\right)/\sum _{j=1}^{n}K\left({\displaystyle \frac{d_{{\mathit{\theta }}_k}(x,X_{j,n})}{h_K\left(x\right)}}\right).\end{array}$$

(52)

Based on Equations (50)–(52), when $G(·)$ is known, a kernel estimator for $r^{\left(1\right)}(\psi ,·)$ is given by

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(1\right)}(\psi ,x;h_n)=\sum _{i=1}^n{\overline{\omega }}_{n,h,i}^{\left(1\right)}\left(x\right){\displaystyle \frac{{\Delta }_i\psi \left(Z_i\right)}{1-G\left(Z_i\right)}}.\end{array}$$

(53)

Function $G(·)$ is generally unknown and has to be estimated. We denote by $G_n^{*}(·)$ the Kaplan–Meier estimator of function $G(·)$ [167]. Namely, adopting conventions

$$\prod _{\varnothing }=1$$

and $0^0=1$ and setting

$$N_n\left(u\right)=\sum _{i=1}^n\mathbb{1}\{Z_i\ge u\},$$

we have

$$G_n^{*}\left(u\right)=1-\prod _{i:Z_i\le u}{\left\{{\displaystyle \frac{N_n\left(Z_i\right)-1}{N_n\left(Z_i\right)}}\right\}}^{(1-{\Delta }_i)},\mathrm{for}u\in \mathbb{R}.$$

Given this notation, we investigate the following estimator of $r^{\left(1\right)}(\psi ,·)$

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(1\right)*}(\psi ,x;h_n)=\sum _{i=1}^n{\overline{\omega }}_{n,h,i}^{\left(1\right)}\left(x\right){\displaystyle \frac{{\Delta }_i\psi \left(Z_i\right)}{1-G_n^{*}\left(Z_i\right)}},\end{array}$$

(54)

referring to [165,168]. Adopting the convention that $0/0=0$, the given quantity is well-defined, since $G_n^{*}\left(Z_i\right)=1$ only if $Z_i=Z_{\left(n\right)}$ and ${\Delta }_{\left(n\right)}=0$, where $Z_{\left(k\right)}$ represents the kth order statistic from the sample $(Z_1,\dots ,Z_n)$ for $k=1,\dots ,n$, and ${\Delta }_{\left(k\right)}$ corresponds to ${\Delta }_j$ such that $Z_k=Z_j$. A right-censored version of an unconditional U-statistic, with a kernel of degree $m\ge 1$, can be introduced via the principle of a mean-preserving reweighting scheme, as outlined in [169]. The almost sure convergence of multi-sample U-statistics under random censorship was established by the authors of [170] who also explored the consistency of a new class of tests designed to evaluate equality in distribution. To mitigate potential biases from right-censoring and confounding covariates, ref. [171] proposed modifications to classical U-statistics. Moreover, ref. [172] introduced an alternative estimation method for the U-statistic by employing a substitution estimator for the conditional kernel, derived from observed data. For additional references, see [42]. To the best of our knowledge, the problem of estimating conditional U-statistics in the presence of censoring—particularly when using variable bandwidths—remains an unresolved issue. This provides the primary motivation for the present study. A natural extension of the function defined in (50) is given by

$$\begin{array}{c}\hfill {\Phi }_{\psi }(y_1,\dots ,y_k,c_1,\dots ,c_k)={\displaystyle \frac{{\displaystyle \prod _{i=1}^k\{\mathbb{1}\{y_i\le c_i\}\psi (y_1\wedge c_1,\dots ,y_k\wedge c_m)}}{{\displaystyle \prod _{i=1}^k\{1-G(y_i\wedge c_i)\}}}}.\end{array}$$

(55)

From this, we have an analogous relation to (51) given by

$$\begin{array}{cc}\hfill \mathbb{E}& ({\Phi }_{\psi }(Y_1,\dots ,Y_k,C_1,\dots ,C_k)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{t})\hfill \\ & =\mathbb{E}\left({\displaystyle \frac{{\displaystyle \prod _{i=1}^k\mathbb{1}\{Y_i\le C_i\}\psi (Y_1\wedge C_1,\dots ,Y_k\wedge C_k)}}{{\displaystyle \prod _{i=1}^k\{1-G(Y_i\wedge C_i)\}}}}\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{t}\right)\hfill \\ & =\mathbb{E}\left({\displaystyle \frac{{\displaystyle \psi (Y_1,\dots ,Y_k)}}{{\displaystyle \prod _{i=1}^k\{1-G\left(Y_i\right)\}}}}\mathbb{E}\left(\prod _{i=1}^k\mathbb{1}\{Y_i\le C_i\}\mid (Y_1,X_1),\dots (Y_k,X_k)\right)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{t}\right)\hfill \\ & =\mathbb{E}\left(\psi (Y_1,\dots ,Y_k)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{t}\right).\hfill \end{array}$$

(56)

An analog estimator to (3) in the censored case is given by

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(k\right)}(\psi ,\mathit{\theta },\mathbf{x})=\sum _{(i_1,\dots ,i_k)\in I_n^k}{\displaystyle \frac{{\displaystyle {\Delta }_{i_1}\cdots {\Delta }_{i_k}\psi (Z_{i_1},\dots ,Z_{i_k})}}{{\displaystyle (1-G\left(Z_{i_1}\right)\cdots (1-G\left(Z_{i_k}\right))}}}{\overline{\omega }}_{n,\mathbf{i}}^{\left(k\right)}(\mathit{\theta },\mathbf{x}),\end{array}$$

(57)

where, for $\mathbf{i}=(i_1,\dots ,i_k)\in I(k,n)$,

$${\overline{\omega }}_{n,\mathbf{i}}^{\left(k\right)}(\mathit{\theta },\mathbf{t})={\displaystyle \frac{{\displaystyle K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{h_K\left(t_1\right)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{h_K\left(t_m\right)}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{h_K\left(t_1\right)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{h_K\left(t_m\right)}\right)}}}.$$

(58)

The estimator that we investigate is given by

$$\begin{array}{c}\hfill {\breve{r}}_n^{\left(k\right)*}(\psi ,\mathit{\theta },\mathbf{x})=\sum _{(i_1,\dots ,i_k)\in I_n^k}{\displaystyle \frac{{\displaystyle {\Delta }_{i_1}\cdots {\Delta }_{i_k}\psi (Z_{i_1},\dots ,Z_{i_k})}}{{\displaystyle (1-G_n^{*}\left(Z_{i_1}\right)\cdots (1-G_n^{*}\left(Z_{i_k}\right))}}}{\overline{\omega }}_{n,\mathbf{i}}^{\left(k\right)}(\mathit{\theta },\mathbf{x}).\end{array}$$

(59)

The main result of this section is given in the following corollary.

 Corollary 8. 

Under the assumptions of Theorems 9 and 10, it follows that, as $n\to \infty$,

$$\begin{gathered} {\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|{\breve{r}}_n^{\left(k\right)*}(\psi ,\mathit{\theta },\mathbf{t})-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|} \\ =O\left({\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{{\rho }_n^{*}n}}\right)}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left({\mu }^{*}{\varphi }^{-1}\left(\frac{{\rho }_n^{*}{k}_{1,n}}{n}\right)\right)}}}}\right). \end{gathered}$$

(60)

This last result is a direct consequence of Corollary 6, and the law of iterated logarithm for $G_n^{*}(·)$ established in [173] ensures that

$$\underset{t\le \tau }{sup}|G_n^{*}-G\left(t\right)|=O\left(\sqrt{{\displaystyle \frac{loglogn}{n}}}\right)\mathrm{almost}\mathrm{surely}\mathrm{as}n\to \infty .$$

For more details, refer to [131].

5.6. Conditional  U-Statistics for Left Truncated and Right Censored Data

Building on the notation from the previous section, we now introduce a truncation variable, denoted by L, and assume that $(L,C)$ is independent of Y. We consider a scenario where we have random vectors $(Z_i,{ϵ}_i,{\Delta }_i)$, with ${ϵ}_i=\mathbb{1}(L_i\le Z_i)$. In this section, our goal is to define conditional U-statistics for data that are left truncated and right censored (LTRC), drawing on ideas from [38], which dealt with the unconditional setting. To this end, we propose the following extension of Function $\left(50\right)$ for LTRC data:

$${\tilde{\Phi }}_{\psi }(y_1,\dots ,y_k,l_1,\dots ,l_k,c_1,\dots ,c_k)={\displaystyle \frac{{\displaystyle \psi (y_1\wedge c_1,\dots ,y_k\wedge c_k)\prod _{i=1}^k\mathbb{1}\{y_i\le c_i\}\mathbb{1}\{l_i\le z_i\}}}{{\displaystyle \prod _{i=1}^k\mathbb{P}\left(l_i<z_i<c_i\right)}}}.$$

According to $\left(56\right)$, we determine that

$$\begin{array}{ccc}& & \mathbb{E}({\Phi }_{\psi }(Y_1,\dots ,Y_k,L_1,\dots ,L_k,C_1,\dots ,C_k)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{t})\hfill \\ & & =\mathbb{E}\left(\psi (Y_1,\dots ,Y_k)\mid ({\mathbf{X}}_1,\dots ,{\mathbf{X}}_k)=\mathbf{t}\right).\hfill \end{array}$$

An analog estimator to (3) for LTRC data can be expressed as follows:

$$\begin{array}{c}\hfill {\breve{\stackrel{˘}{r}}}_n^{\left(k\right)}(\psi ,\mathit{\theta },\mathbf{x})=\sum _{(i_1,\dots ,i_k)\in I_n^k}{\displaystyle \frac{{\displaystyle {\Delta }_{i_1}\cdots {\Delta }_{i_k}{ϵ}_{i_1}\cdots {ϵ}_{i_k}\psi (Z_{i_1},\dots ,Z_{i_k})}}{{\displaystyle \mathbb{P}\left(L_{i_1}<Z_{i_1}<C_{i_1}\right)\cdots \mathbb{P}\left(L_{i_k}<Z_{i_k}<C_{i_k}\right)}}}{\overline{\omega }}_{n,\mathbf{i}}^{\left(k\right)}(\mathit{\theta },\mathbf{x}),\end{array}$$

(61)

where ${\overline{\omega }}_{n,\mathbf{i}}^{\left(k\right)}(\mathit{\theta },\mathbf{x})$ is defined as in (58). Since $\mathbb{P}(L_i<Z_i<C_i)$ is unknown, it needs to be estimated. To achieve this, we introduce $N_i\left(t\right)=\mathbb{1}(L_i<Z_i\le t,{\Delta }_i=1)$ and $N_i^c\left(t\right)=\mathbb{1}(L_i<Z_i\le t,{\Delta }_i=0)$, representing the counting processes for the variable of interest and the censoring variable, respectively. Additionally, we define $N\left(t\right)={\sum }_{i=1}^n{N}_i\left(t\right)$ and $N^c\left(t\right)={\sum }_{i=1}^n{N}_i^c\left(t\right)$. We introduce the risk indicators as $R_i\left(t\right)=\mathbb{1}(Z_i\ge t\ge L_i)$ and $R\left(t\right)={\sum }_{i=1}^n{R}_i\left(t\right)$, where $R\left(t\right)$ represents the risk set at time t, consisting of subjects who entered the study before time t and are still under observation at that time. Importantly, $N_i^c\left(t\right)$ is a local sub-martingale with respect to the appropriate filtration ${\mathbb{F}}_t$. The martingale associated with the censoring counting process under filtration ${\mathbb{F}}_t$ is given by

$$M_i^c\left(t\right)=N_i^c\left(t\right)-{\int }_0^t{R}_i\left(u\right){\lambda }_c\left(u\right)\mathrm{d}u,i=1,2,\dots ,n,$$

where ${\lambda }_c(·)$ represents the hazard function associated with the censoring variable C under left truncation. The cumulative hazard function for the censoring variable is defined as ${\Lambda }_c\left(t\right)={\int }_0^t{\lambda }_c\left(u\right)\mathrm{d}u$. We let $M^c\left(t\right)={\sum }_{i=1}^n{M}_i^c\left(t\right)$. Next, we define the sub-distribution function of $T_1$ corresponding to ${\Delta }_1=1$ and ${ϵ}_1=1$ as follows:

$$S\left(x\right)=\mathbb{P}\left(T_1\le x,{\Delta }_1{ϵ}_1=1\right).$$

We let

$$w\left(t\right)={\int }_0^{\infty }{\displaystyle \frac{h_1\left(x\right)}{\mathbb{P}\left(L_1\le x\le C_1\right)}}\mathbb{1}(x>t)\mathrm{d}S\left(x\right),$$

where $h_1\left(x\right)=\mathbb{E}\left(\psi \left(\left(T_1,{\Delta }_1\right),\dots ,\left(T_k,{\Delta }_k\right)\right)\mid \left(T_1,{\Delta }_1\right)=\left(x,{\Delta }_1\right)\right)$. Also, we denote

$$\tilde{z}\left(t\right)=\mathbb{P}\left(T_1\ge t\ge L_1\right).$$

Then, an estimate for the survival function of the censoring variable C under left truncation, denoted as ${\widehat{K}}_c(·)$—see [174]—can be formulated as follows:

$$\begin{array}{c}\hfill {\widehat{K}}_c\left(\tau \right)=\prod _{t\le \tau }\left(1-{\displaystyle \frac{d{N}^c\left(t\right)}{\tilde{Z}\left(t\right)}}\right).\end{array}$$

(62)

Similar to the Nelson–Aalen estimator— for instance, see [175]—the estimator for the cumulative hazard function of the censoring variable C under left truncation is represented as

$$\begin{array}{c}\hfill {\widehat{\Lambda }}_c\left(\tau \right)={\int }_0^{\tau }{\displaystyle \frac{d{N}^c\left(t\right)}{\tilde{Z}\left(t\right)}}.\end{array}$$

(63)

In both the definitions presented in (62) and (63), we make the assumption that $\tilde{Z}\left(t\right)$ is non-zero with probability one. The interrelation between ${\widehat{K}}_c\left(\tau \right)$ and ${\widehat{\Lambda }}_c\left(\tau \right)$ can be expressed as

$${\widehat{K}}_c\left(\tau \right)=exp\left[-{\widehat{\Lambda }}_c\left(\tau \right)\right].$$

We let $a_K=inf\{t:K\left(t\right)>0\}$ and $b_K=sup\{t:K\left(t\right)<1\}$ denote the left and right endpoints of the support. For LTRC data, as in [176], $F(·)$ is identifiable if $a_G⩽a_W$ and $b_G⩽b_W$. By Corollary 2.2. [176], for $b<b_W$, we readily infer that

$$\begin{array}{c}\hfill \underset{a_W<\tau <b}{sup}|{\widehat{K}}_c\left(\tau \right)-K_c\left(\tau \right)|=O\left(\sqrt{n^{-1}loglogn}\right).\end{array}$$

(64)

From the above, Estimator (61) can be rewritten directly as follows:

$$\begin{array}{c}\hfill {\breve{\stackrel{˘}{r}}}_n^{\left(k\right)*}(\psi ,\mathit{\theta },\mathbf{x})=\sum _{(i_1,\dots ,i_k)\in I_n^k}{\displaystyle \frac{{\displaystyle {\Delta }_{i_1}\cdots {\Delta }_{i_k}{ϵ}_{i_1}\cdots {ϵ}_{i_k}\psi (Z_{i_1},\dots ,Z_{i_k})}}{{\displaystyle {\widehat{K}}_c\left(Z_{i_1}\right)\cdots {\widehat{K}}_c\left(Z_{i_k}\right)}}}{\overline{\omega }}_{n,\mathbf{i}}^{\left(k\right)}(\mathit{\theta },\mathbf{x}).\end{array}$$

(65)

The last estimator is the conditional version of that studied in [38]. Following the same reasoning of Corollary 8, one can infer that as $n\to \infty ,$

$$\begin{gathered} {\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{k_{1,n}\le k\le k_{2,n}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|{\breve{\stackrel{˘}{r}}}_n^{\left(k\right)*}(\psi ,\mathit{\theta },\mathbf{x})-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|} \\ =O\left({\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{{\rho }_n^{*}n}}\right)}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left({\mu }^{*}{\varphi }^{-1}\left(\frac{{\rho }_n^{*}{k}_{1,n}}{n}\right)\right)}}}}\right). \end{gathered}$$

(66)

Remark 11.

• In the study by the authors of [177], the practical utility of U-statistics is exemplified through an analysis of data from a case–control study among African Americans. The researchers investigated whether the similarity of haplotypes between a case and a control participant differs from the similarity between two control participants. By employing U-statistics, they effectively measured and compared haplotype similarities, highlighting the method’s applicability in genetic association studies and its capacity to handle complex genetic data.
• Expanding on the traditional use of U-statistics, the authors of [178] introduced a new family that includes the well-known Wilcoxon signed-rank statistic and extends to other statistics with substantially higher power in sensitivity analyses of observational studies. These enhanced statistics were applied in three diverse examples drawn from epidemiology, clinical medicine, and genetic toxicology. The results demonstrated not only improved performance but also the flexibility of these new U-statistics in addressing various research questions across different fields.
• In another significant contribution, the authors of [179] combined the properties of U-statistics with large-dimensional random matrix theory to identify group structures among numerous variables. By analyzing the eigenvalues and eigenvectors of the scaled sample matrix, they uncovered underlying patterns and groupings within high-dimensional data sets. This approach showcases the power of U-statistics in dimensionality reduction and the exploration of complex data structures, which are common challenges in modern data analysis.
• Furthermore, the authors of [180] proposed the heterogeneity-weighted U (HWU)-method for association analyses that consider genetic heterogeneity. This innovative method leverages U-statistics to provide a computationally efficient solution suitable for high-dimensional genetic data and various types of phenotypes. The HWU method addresses the significant challenge of genetic heterogeneity in association studies, offering researchers a powerful tool to detect associations that might be overlooked by traditional methods.

5.7. Examples of Classes of Functions

 Example 1. 

The set $\mathcal{F}$ of all indicator functions ${1\mathrm{I}}_{\left\{\right(-\infty ,t\left]\right\}}$ representing intervals in $\mathbb{R}$ satisfies the following inequality:

$$N\left(ϵ,\mathcal{F},d_{\mathbb{P}}^{\left(2\right)}\right)\le {\displaystyle \frac{2}{{ϵ}^2}},$$

for any probability measure $\mathbb{P}$ and for $ϵ\le 1$. We observe that

$${\int }_0^1\sqrt{log\left({\displaystyle \frac{1}{ϵ}}\right)}dϵ\le {\int }_0^{\infty }{u}^{1/2}exp(-u)du\le 1.$$

For further details, refer to Example 2.5.4 in [181] and [182] (p. 157). The covering numbers for the class of intervals $(-\infty ,t]$ in higher dimensions also satisfy a similar bound, though with a higher power of $(1/ϵ)$; see Theorem 9.19 in [182].

 Example 2 

(Classes of functions that are Lipschitz in a parameter, Section 2.7.4 in [181]). We let $\mathcal{F}$ be the class of functions $x\mapsto \phi (t,x)$ that are Lipschitz continuous with respect to the parameter $t\in T$. We suppose

$$|\phi (t_1,x)-\phi (t_2,x)|\le d(t_1,t_2)\kappa \left(x\right),$$

for some metric d on the index set T and a function $\kappa (·)$ defined on the sample space $\mathcal{X}$, for all x. According to Theorem 2.7.11 in [181] and Lemma 9.18 in [182], it follows that for any norm ${\parallel ·\parallel }_{\mathcal{F}}$ on $\mathcal{F}$

$$N(ϵ\parallel F{\parallel }_{\mathcal{F}},\mathcal{F},\parallel ·{\parallel }_{\mathcal{F}})\le N(ϵ/2,T,d).$$

Thus, if $(T,d)$ satisfies

$$J(\infty ,T,d)={\int }_0^{\infty }\sqrt{logN(ϵ,T,d)}dϵ<\infty ,$$

the result holds for $\mathcal{F}$.

 Example 3. 

We consider the class of functions that are smooth up to order α, as discussed in Section 2.7.1 of [181] and Section 2 of [183]. For $0<\alpha <\infty$, we let $\lfloor \alpha \rfloor$ denote the greatest integer smaller than α. For any vector $k=(k_1,\dots ,k_d)$ of d integers, we define the differential operator:

$$D^k:={\displaystyle \frac{{\partial }^{k_1+\cdots +k_d}}{{\partial }^{k_1}\cdots {\partial }^{k_d}}},$$

where $k_{.}:={\sum }_{i=1}^d{k}_i$. For a function $f:\mathcal{X}\to \mathbb{R}$, we define the norm

$${\parallel f\parallel }_{\alpha }:=\underset{k_{.}\le \lfloor \alpha \rfloor }{max}\underset{x}{sup}\left|D^{k}f\left(x\right)\right|+\underset{k_{.}=\lfloor \alpha \rfloor }{max}\underset{x\ne y}{sup}{\displaystyle \frac{|D^{k}f\left(x\right)-D^{k}f\left(y\right)|}{{\parallel x-y\parallel }^{\alpha -\lfloor \alpha \rfloor }}},$$

where the suprema are taken over all $x,y$ in the interior of $\mathcal{X}$. We let $C_M^{\alpha }\left(\mathcal{X}\right)$ denote the set of continuous functions $f:\mathcal{X}\to \mathbb{R}$ such that ${\parallel f\parallel }_{\alpha }\le M$. For $\alpha \le 1$, this class consists of bounded functions satisfying a Lipschitz condition. According to [135], the entropy of class $C_M^{\alpha }\left(\mathcal{X}\right)$ for the uniform norm was computed, and [183] showed that there exists a constant K, depending on α, d, and the diameter of $\mathcal{X}$, such that for every measure γ and every $ϵ>0$,

$$log{\mathcal{N}}_{\left[\right]}(ϵM\gamma \left(\mathcal{X}\right),C_M^{\alpha }\left(\mathcal{X}\right),L_2\left(\gamma \right))\le K{\left({\displaystyle \frac{1}{ϵ}}\right)}^{d/\alpha },$$

where ${\mathcal{N}}_{\left[\right]}$ denotes the bracketing number (see Definition 2.1.6 in [181]). A variant of this inequality is given in Theorem 2.7.1 of [181]. Lemma 9.18 of [182] also implies

$$log\mathcal{N}(ϵM\gamma \left(\mathcal{X}\right),C_M^{\alpha }\left(\mathcal{X}\right),L_2\left(\gamma \right))\le K{\left({\displaystyle \frac{1}{2ϵ}}\right)}^{d/\alpha }.$$

5.8. Examples of  U-Kernels

In this section, we present some classical U-kernels.

 Example 4. 

Ref. [1] introduced the parameter

$$\Delta ={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }{D}^2(y_1,y_2)dF(y_1,y_2),$$

where $D(y_1,y_2)=F(y_1,y_2)-F(y_1,\infty )F(\infty ,y_2)$ and $F(·,·)$ is the joint distribution function of $Y_1$ and $Y_2$. Parameter Δ has the property that $\Delta =0$ if and only if $Y_1$ and $Y_2$ are independent. As shown in [11], an alternative expression for Δ can be developed by introducing the following functions:

$$\psi (y_1,y_2,y_3)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{y}_2\le y_1<y_3,\hfill \\ 0\hfill & \mathrm{if}{y}_1<y_2\mathrm{or}{y}_1\ge y_2,\hfill \\ -1\hfill & \mathrm{if}{y}_3\le y_1<y_2,\hfill \end{array}\right.$$

and

$$h(y_{1,1},y_{1,2},\dots ,y_{5,1},y_{5,2})={\displaystyle \frac{1}{4}}\psi (y_{1,1},y_{1,2},y_{1,3})\psi (y_{1,1},y_{1,4},y_{1,5})\psi (y_{1,2},y_{2,2},y_{3,2})\psi (y_{1,2},y_{4,2},y_{5,2}).$$

This leads to expression

$$\Delta =\int \dots \int h(y_{1,1},y_{1,2},\dots ,y_{5,1},y_{5,2})dF(y_{1,1},y_{1,2})\dots dF(y_{1,5},y_{2,5}).$$

 Example 5 

(Hoeffding’s D). The symmetric kernel

$$\begin{array}{cc}\hfill h_D& (z_1,\dots ,z_5):={\displaystyle \frac{1}{16}}\sum _{\left(i_1,\dots ,i_5\right)\in {\mathcal{P}}_5}\left[\mathbb{1}\left(z_{i_1,1}\le z_{i_5,1}\right)-\mathbb{1}\left(z_{i_2,1}\le z_{i_5,1}\right)\right]\left[\mathbb{1}\left(z_{i_3,1}\le z_{i_5,1}\right)-\mathbb{1}\left(z_{i_4,1}\le z_{i_5,1}\right)\right]\hfill \\ & \times \left[\mathbb{1}\left(z_{i_1,2}\le z_{i_5,2}\right)-\mathbb{1}\left(z_{i_2,2}\le z_{i_5,2}\right)\right]\left[\mathbb{1}\left(z_{i_3,2}\le z_{i_5,2}\right)-\mathbb{1}\left(z_{i_4,2}\le z_{i_5,2}\right)\right]\hfill \end{array}$$

recovers Hoeffding’s D statistic, a rank-based U-statistic of Order 5, which leads to Hoeffding’s D correlation measure $\mathbb{E}{h}_D$.

 Example 6 

(Blum–Kiefer–Rosenblatt’s R). The symmetric kernel

$$\begin{array}{c}\hfill h_R(z_1,\dots ,z_6):={\displaystyle \frac{1}{32}}\sum _{\left(i_1,\dots ,i_6\right)\in {\mathcal{P}}_6}\left[\mathbb{1}\left(z_{i_1,1}\le z_{i_5,1}\right)-\mathbb{1}\left(z_{i_2,1}\le z_{i_5,1}\right)\right]\left[\mathbb{1}\left(z_{i_3,1}\le z_{i_5,1}\right)-\mathbb{1}\left(z_{i_4,1}\le z_{i_5,1}\right)\right]\\ \times \left[\mathbb{1}\left(z_{i_1,2}\le z_{i_6,2}\right)-\mathbb{1}\left(z_{i_2,2}\le z_{i_6,2}\right)\right]\left[\mathbb{1}\left(z_{i_3,2}\le z_{i_6,2}\right)-\mathbb{1}\left(z_{i_4,2}\le z_{i_6,2}\right)\right]\hfill \end{array}$$

yields Blum–Kiefer–Rosenblatt’s R statistic [184,185,186,187].

 Example 7 

(Bergsma–Dassios–Yanagimoto’s ${\tau }^{*}$). In [188], a rank correlation statistic was introduced as a U-statistic of Order 4 with the symmetric kernel:

$$\begin{array}{cc}\hfill h_{{\tau }^{*}}\left(z_1,\dots ,z_4\right):=& {\displaystyle \frac{1}{16}}\sum _{\left(i_1,\dots ,i_4\right)\in {\mathcal{P}}_4}[\mathbb{1}(z_{i_1,1},z_{i_3,1}<z_{i_2,1},z_{i_4,1})+\mathbb{1}(z_{i_2,1},z_{i_4,1}<z_{i_1,1},z_{i_3,1})\hfill \\ & -\mathbb{1}(z_{i_1,1},z_{i_4,1}<z_{i_2,1},z_{i_3,1})-\mathbb{1}(z_{i_2,1},z_{i_3,1}<z_{i_1,1},z_{i_4,1})]\hfill \\ & \times [\mathbb{1}(z_{i_1,2},z_{i_3,2}<z_{i_2,2},z_{i_4,2})+\mathbb{1}(z_{i_2,2},z_{i_4,2}<z_{i_1,2},z_{i_3,2})\hfill \\ & -\mathbb{1}(z_{i_1,2},z_{i_4,2}<z_{i_2,2},z_{i_3,2})-\mathbb{1}(z_{i_2,2},z_{i_3,2}<z_{i_1,2},z_{i_4,2})].\hfill \end{array}$$

Here, $\mathbb{1}(y_1,y_2<y_3,y_4)$ denotes the indicator that both $y_1<y_3$ and $y_2<y_4$.

 Example 8 

(The Wilcoxon Statistic). We suppose $E\subset \mathbb{R}$ is symmetric around zero. As an estimator of the quantity

$${\int }_{(x,y)\in E^2}\left\{2\mathbb{1}(x+y>0)-1\right\}dF\left(x\right)dF\left(y\right),$$

we consider the Wilcoxon statistic,

$$W_n={\displaystyle \frac{2}{n(n-1)}}\sum _{1\le i<j\le n}\left\{2·\mathbb{1}(X_i+X_j>0)-1\right\},$$

which is useful for testing whether the mean μ is located at zero.

 Example 9 

(The Takens Estimator). We let $\parallel ·\parallel$ denote the Euclidean norm on ${\mathbb{R}}^d$. In [189], the following estimator of the correlation integral,

$$C_F\left(r\right)=\int {\mathbb{I}}_{\left\{\parallel x-x^{\prime }\parallel \le r\right\}}dF\left(x\right)dF\left(x^{\prime }\right),r>0,$$

is given by

$$C_n\left(r\right)={\displaystyle \frac{1}{n(n-1)}}\sum _{1\le i\ne j\le n}{\mathbb{I}}_{\left\{\parallel X_i-X_j\parallel \le r\right\}}.$$

When a scaling law holds, i.e., $C_F\left(r\right)=c·r^{-\alpha }$ for some $(\alpha ,r_0,c)$, the U-statistic

$$T_n={\displaystyle \frac{1}{n(n-1)}}\sum _{1\le i\ne j\le n}log\left({\displaystyle \frac{\parallel X_i-X_j\parallel }{r_0}}\right),$$

provides the Takens estimator ${\widehat{\alpha }}_n=-T_n^{-1}$ of the correlation dimension α.

 Example 10. 

We let $\widehat{Y_1{Y}_2}$ denote the oriented angle between $Y_1$ and $Y_2\in T$, where T is the unit circle in ${\mathbb{R}}^2$. We define

$$h_t(Y_1,Y_2)=\mathbb{1}\{\widehat{Y_1{Y}_2}\le t\}-t/\pi ,fort\in [0,\pi ).$$

Ref. [190] used this kernel to propose a U-process for testing uniformity on the circle.

 Example 11. 

For $m=3$, we let

$$\phi (Y_1,Y_2,Y_3)=\mathbb{1}\{Y_1-Y_2-Y_3>0\}.$$

We have

$$r^{\left(3\right)}(\phi ,t_1,t_2,t_3)=\mathbb{P}(Y_1>Y_2+Y_3\mid X_1=X_2=X_3=t),$$

and the corresponding conditional U-statistic can be considered a conditional analog of the Hollander–Proschan test statistic [191], which tests whether the conditional distribution of $Y_1$ given $X_1=t$ is exponential or of the New-Better-than-Used type.

 Example 12 

(The Gini Mean Difference). The Gini index, another popular measure of dispersion, corresponds to the case where $E\subset \mathbb{R}$ and $h(x,y)=|x-y|$. The Gini statistic is given by

$$G_n={\displaystyle \frac{2}{n(n-1)}}\sum _{1\le i<j\le n}|X_i-X_j|.$$

In the next section, we provide more details on how several neighborhood selection methodologies from the literature can be combined with our results.

6. The Bandwidth Selection Criterion

Numerous methodologies have been proposed for constructing asymptotically optimal bandwidth selection rules, particularly in the context of the Nadaraya–Watson regression estimator. Prominent references in this area include [42,51,52,192,193,194]. The accurate bandwidth selection, whether in a finite- or infinite-dimensional setting, is critical to achieving optimal performance in practical applications. However, to the best of our knowledge, no studies have yet addressed the general functional conditional U-statistic. Unlike the real-valued case, where the selection of the number k has been thoroughly explored in [195], we propose an extension of the leave-one-out cross-validation procedure. For any fixed $\mathbf{i}=(i_1,\dots ,i_m)\in I(m,n)$, we define

$${\widehat{r}}_{n,\mathbf{i}}^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))={\displaystyle \frac{{\displaystyle \sum _{\mathbf{j}\in I_n^m\left(\mathbf{i}\right)}\phi (Y_{j_1},\dots ,Y_{j_m})K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{j_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{j_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}{{\displaystyle \sum _{\mathbf{j}\in I_n^m\left(\mathbf{i}\right)}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{j_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{j_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}},$$

(67)

where $I_n^m\left(\mathbf{i}\right):=I(m,n)\setminus \left\{\mathbf{i}\right\}$. This expression defines the leave-one-out-$({\mathbf{X}}_{\mathbf{i}},{\mathbf{Y}}_{\mathbf{i}})$ estimator for functional regression and serves as a predictor for $\phi \left({\mathbf{Y}}_{\mathbf{i}}\right)$. To minimize the quadratic loss function, we introduce the following criterion, where $\mathcal{W}(·)$ represents a known non-negative weight function:

$$C{V}^{*}\left(\phi ,\mathit{\theta },k\right):=\sum _{\mathbf{i}\in I(m,n)}{\left(\phi \left({\mathbf{Y}}_{\mathbf{i}}\right)-{\widehat{r}}_{n,\mathbf{i}}^{*\left(m\right)}(\phi ,{\mathbf{X}}_{\mathbf{i}},\mathit{\theta },{\mathbf{h}}_{n,k}\left({\mathbf{X}}_{\mathbf{i}}\right))\right)}^2\tilde{\mathcal{W}}\left({\mathbf{X}}_{\mathbf{i}}\right),$$

(68)

where $\tilde{\mathcal{W}}\left(\mathbf{t}\right):={\prod }_{i=1}^m\mathcal{W}\left(t_i\right)$. Similarly, we define

$$CV\left(\phi ,\mathit{\theta },h\right):=\sum _{\mathbf{i}\in I(m,n)}{\left(\phi \left({\mathbf{Y}}_{\mathbf{i}}\right)-{\widehat{r}}_{n,\mathbf{i}}^{\left(m\right)}(\phi ,{\mathbf{X}}_{\mathbf{i}},\mathit{\theta },h)\right)}^2\tilde{\mathcal{W}}\left({\mathbf{X}}_{\mathbf{i}}\right).$$

(69)

Following the ideas in [81,90,194], an intuitive approach to bandwidth selection is to minimize this criterion. We let

$${\widehat{\mathit{\theta }}}_{n,k}^{*}=\underset{\mathit{\theta }\in {\mathbf{\Theta }}_n^m}{argmin}\underset{\phi \in {\mathcal{F}}_m}{sup}C{V}^{*}\left(\phi ,\mathit{\theta },k\right),$$

and

$${\widehat{\mathit{\theta }}}_{n,h}=\underset{\mathit{\theta }\in {\mathbf{\Theta }}_n^m}{argmin}\underset{\phi \in {\mathcal{F}}_m}{sup}CV\left(\phi ,\mathit{\theta },h\right).$$

We let $\widehat{k}\in [k_{1,n},k_{2,n}]$ minimize over $k\in [k_{1,n},k_{2,n}]$,

$$\underset{\phi \in {\mathcal{F}}_m}{sup}CV\left(\phi ,{\widehat{\mathit{\theta }}}_{n,k}^{*},k\right),$$

and let $\widehat{\mathbf{h}}\in {\mathfrak{H}}_n^{\left(m\right)}$ minimize over $\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}$,

$$\underset{\phi \in {\mathcal{F}}_m}{sup}C{V}^{*}\left(\phi ,{\widehat{\mathit{\theta }}}_{n,h},\mathbf{h}\right).$$

We now state the following corollary:

Corollary 9. 

• Under the assumptions of Corollary 5, as $n\to \infty$,

  $$\begin{array}{c}{\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\left|{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},{\widehat{\mathit{\theta }}}_{n,h};\widehat{\mathbf{h}})-r^{\left(m\right)}(\phi ,\mathbf{t},{\mathit{\theta }}_0)\right|}\hfill \\ \hspace{1em}\hspace{1em}=O\left({\widetilde{h^{\prime }}}_n^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left({\tilde{h}}_n\right)}}}}\right).\hfill \end{array}$$

  (70)
• Under the assumptions of Corollary 6, as $n\to \infty$,

  $$\begin{array}{c}{\displaystyle \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\left|{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},{\widehat{\mathit{\theta }}}_{n,k}^{*};{\mathbf{h}}_{n,\widehat{k}}\left(\mathbf{t}\right))-r^{\left(m\right)}(\phi ,\mathbf{t},{\mathit{\theta }}_0)\right|}\hfill \\ \hspace{1em}\hspace{1em}=O\left({\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{{\rho }_n^{*}n}}\right)}^{\gamma }\right)+O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left({\mu }^{*}{\varphi }^{-1}\left(\frac{{\rho }_n^{*}{k}_{1,n}}{n}\right)\right)}}}}\right).\hfill \end{array}$$

  (71)

A key advantage of our results lies in deriving asymptotics for data-driven parameters. We let $K^{\prime }(·)$ be a density function in ${\mathbb{R}}^d$ and let $H_{n,k^{\prime }}^{\prime }\left(t_i\right)$ denote the number of neighborhoods associated with the $Y_i$ values. The conditional density ${\mathbf{f}}^{\left(m\right)}(y_1,\dots ,y_m\mid \mathbf{t},\mathit{\theta })$ can be estimated as follows:

$$\begin{array}{c}{\displaystyle {\widehat{\mathbf{f}}}_n^{\left(m\right)}(\mathbf{y},\mathbf{t},\mathit{\theta })}\hfill \\ :={\displaystyle \frac{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}{K}^{\prime }\left(\frac{t_1-Y_{i_1}}{H_{n,k^{\prime }}^{\prime }\left(t_1\right)}\right)\cdots K^{\prime }\left(\frac{t_m-Y_{i_m}}{H_{n,k^{\prime }}\left(t_m\right)}\right)K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}\left(t_1\right)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{i_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{i_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}}.\hfill \end{array}$$

The leave-one-out estimator is given by

$$\begin{array}{c}{\displaystyle {\widehat{\mathbf{f}}}_{n,\mathbf{i}}^{\left(m\right)}(\mathbf{y},\mathbf{t},\mathit{\theta })}\hfill \\ :={\displaystyle \frac{{\displaystyle \sum _{(j_1,\dots ,j_m)\in I_n^m\left(\mathbf{i}\right)}{K}^{\prime }\left(\frac{t_1-Y_{j_1}}{H_{n,k^{\prime }}^{\prime }\left(t_1\right)}\right)\cdots K^{\prime }\left(\frac{t_m-Y_{j_m}}{H_{n,k^{\prime }}\left(t_m\right)}\right)K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{j_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{j_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}{{\displaystyle \sum _{(i_1,\dots ,i_m)\in I(m,n)}K\left(\frac{d_{{\mathit{\theta }}_1}(t_1,X_{j_1})}{H_{n,k}(t_1,{\mathit{\theta }}_1)}\right)\cdots K\left(\frac{d_{{\mathit{\theta }}_m}(t_m,X_{j_m})}{H_{n,k}(t_m,{\mathit{\theta }}_m)}\right)}}}.\hfill \end{array}$$

Although cross-validation procedures aim to approximate the quadratic estimation errors, alternative methods for selecting smoothing parameters can be introduced to optimize predictive power. The criterion is given by

$$(\breve{k},{\breve{k}}^{\prime })=arg\underset{k_{1,n}\le k\le k_{2,n},k_{1,n}^{\prime }\le k^{\prime }\le k_{2,n}^{\prime }}{min}\sum _{\mathbf{i}\in I(m,n)}{\left(\phi \left({\mathbf{Y}}_{\mathbf{i}}\right)-arg\underset{\mathbf{y}\in \mathbf{J}}{max}{\widehat{\mathbf{f}}}_{n,\mathbf{i}}^{\left(m\right)}(\mathbf{y},{\mathbf{X}}_{\mathbf{i}},\mathit{\theta })\right)}^2.$$

 Remark 12. 

As emphasized in [90], the range of values for parameter h, denoted as $h\in [a_n,b_n]$, is typically chosen by minimizing a criterion function, such as the cross-validation (CV) function, over a broad interval. This ensures that any reasonable values of h satisfying the technical conditions are included. While the automatic determination of the interval $[a_n,b_n]$ remains unresolved in one-dimensional nonparametric statistics, it is often of lesser concern since the criterion function typically exhibits flat behavior around its minimum. Early works, such as [193,196], suggest selecting an interval allowing bandwidths to capture up to 95% of the sample. As discussed in [90], this remains effective in the functional framework. Similarly, for parameter k, the set of values $\{k_{1,n},k_{1,n}+1,\dots ,k_{2,n}\}$ is determined by similar principles, with global kernel estimate considerations applying equally to local bandwidth estimates (see [197] for early developments). The same recommendation applies to the selection of $\{k_{1,n},k_{1,n}+1,\dots ,k_{2,n}\}$.

In the following section, we provide a comprehensive summary of our main findings, highlighting the key results. Additionally, we outline potential directions for future work, suggesting areas where further investigation could be beneficial.

7. Concluding Remarks

This paper is centered on the k-Nearest Neighbor (kNN) kernel-type estimator for single-index conditional U-statistics, with the single-index Nadaraya–Watson estimator serving as a special case within a functional framework applied to regular datasets. Our results are predicated on specific regularity conditions related to conditional U-statistics and conditional moments, along with decay rates governing the probability distribution of variables within diminishing open balls and favorable decreasing rates for mixing coefficients. The conditional moment assumption plays a critical role, as it allows us to work with unbounded function classes. The proof of weak convergence adheres to a classical methodology, which entails establishing finite-dimensional convergence and rigorously controlling the equicontinuity of conditional U-processes. Finite-dimensional convergence is achieved through a block decomposition technique that ensures independence, after which a central limit theorem for independent variables is proven. However, the challenge of controlling equicontinuity necessitates a more sophisticated approach, particularly given the generality and complexity of the framework we consider. Full details of these proofs are provided in the subsequent sections. It is important to note that while we assume mixing—a form of asymptotic independence—primarily for simplicity, this assumption may not fully capture scenarios involving strong data dependence. As highlighted in [198], $\beta$-mixing is regarded as the minimal assumption that allows for a “complete” empirical process theory, incorporating maximal inequalities and uniform central limit theorems. Explicit upper bounds for $\beta$-mixing coefficients are available in the context of Markov chains (cf. [199]) and for V-geometric mixing coefficients (cf. [200]). Various stationary time series models, including linear processes (see [201] for $\alpha$-mixing), ARMA models (cf. [202]), and nonlinear autoregressive models (cf. [203]), have been extensively studied under similar conditions. A common assumption across these works is the continuity of either the observed process or the innovations associated with it. The application of nonparametric functional methods to general dependence structures remains a relatively unexplored domain; see [204,205,206,207,208,209,209]. Notably, our study employs an ergodic framework that circumvents the need for strong mixing conditions and their variations, thus avoiding the intricate probabilistic computations these conditions typically entail. While extending our results to encompass functional ergodic data would undoubtedly be a valuable direction for future research, it would necessitate significant mathematical advancements and falls beyond the scope of this paper. The primary challenge lies in the development of new probabilistic tools analogous to those employed in this study, but specifically adapted for $\beta$-mixing samples. Finally, it would be of great interest to investigate the potential connection between our results and change-point detection, a method widely utilized for identifying abrupt shifts in stochastic systems caused by external perturbations. This technique has found applications across diverse scientific disciplines (see [210]), and exploring how our findings may relate to change-point problems presents a promising avenue for future inquiry. It will be of interest to find some links of the present with papers [211,212].

8. Mathematical Developments

This section is dedicated to the detailed presentation of the proofs underlying our results, maintaining consistency with the notation established earlier. Our proof techniques are inspired by those developed in [155], which we adapt and extend to the single-index framework. Moreover, we incorporate several of the more intricate arguments from [109], as used in prior works such as [110,112].

8.1. Proofs of Uniform Consistency Results

8.1.1. Proof of Theorem 1

To derive the convergence rates, it is essential to introduce the following notation. For every $t\in S_{\mathcal{H}}$, we define

$$\begin{array}{ccc}\hfill {\Delta }_i(t,\mathit{\theta };h_K\left(t\right))& :=& K\left(d_{\mathit{\theta }}(X_i,t)/h_K\left(t\right)\right),\hfill \\ \hfill {\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))& :=& {\displaystyle \frac{1}{n\mathbb{E}\left({\Delta }_1(t,\mathit{\theta };h_K\left(t\right))\right)}}\sum _{i=1}^n\phi \left(Y_i\right){\Delta }_i(t,\mathit{\theta };h_K\left(t\right)),\hfill \\ \hfill {\widehat{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩;h_K\left(t\right))& :=& {\displaystyle \frac{1}{n\mathbb{E}\left({\Delta }_1(t,\mathit{\theta };h_K\left(t\right))\right)}}\sum _{i=1}^n{\Delta }_i(t,\mathit{\theta };h_K\left(t\right)).\hfill \end{array}$$

This allows us to write

$${\widehat{r}}_n^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))={\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))/{\widehat{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩;h_K\left(t\right)).$$

Now, let us consider the following decomposition:

$$\begin{array}{cc}{\displaystyle {\widehat{r}}_n^{\left(1\right)}}& (\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\hfill \\ =& {\displaystyle \frac{1}{{\widehat{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))}}\left\{{\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))-\mathbb{E}\left[{\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))\right]\right\}\hfill \\ & +{\displaystyle \frac{1}{{\widehat{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))}}\left\{\mathbb{E}\left[{\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))\right]-r^{\left(1\right)}(\phi ,t)\right\}\hfill \\ & +{\displaystyle \frac{r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)}{{\widehat{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))}}\left\{1-{\widehat{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))\right\}.\hfill \end{array}$$

Therefore, the proof of (A2) is based on Lemmas A2, A3 and A9.

8.1.2. Proof of Theorem 2

In a manner analogous to [51], verifying the conditions of Lemma A1 for the case of $m=1$ is crucial for establishing Theorem 2. To achieve this, we first introduce the following designations: $S_{\Omega }=S_{\mathcal{H}}$, $A_i=X_i$, and $\phi \left(B_i\right)=\phi \left(Y_i\right)$. Subsequently, we define

$$\begin{array}{ccc}\hfill G(H,(⟨t,\mathit{\theta }⟩,A_i))& =& K\left(H^{-1}{d}_{\mathit{\theta }}(t,X_i)\right),\hfill \\ \hfill D_{n,k}\left(⟨t,\mathit{\theta }⟩\right)& =& H_{n,k}(t,\mathit{\theta }),\hfill \\ \hfill {\mathfrak{M}}_n^{\left(1\right)}(\phi ,t,\mathit{\theta };H_{n,k}(t,\mathit{\theta }))& =& {\widehat{r}}_n^{*\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta })),\hfill \\ \hfill \mathfrak{M}(\phi ,t,\mathit{\theta })& =& r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩).\hfill \end{array}$$

We then define $D_{n,k}^{-}({\mathit{\xi }}_n,⟨t,\mathit{\theta }⟩)$ and $D_{n,k}^{+}({\mathit{\xi }}_n,⟨t,\mathit{\theta }⟩)$ such that

$${\varphi }_{t,\mathit{\theta }}\left(D_{n,k}^{-}({\mathit{\xi }}_n,⟨t,\mathit{\theta }⟩)\right)={\displaystyle \frac{\sqrt{{\mathit{\xi }}_n}k}{n}},$$

(72)

$${\varphi }_{t,\mathit{\theta }}\left(D_{n,k}^{+}({\mathit{\xi }}_n,⟨t,\mathit{\theta }⟩)\right)={\displaystyle \frac{k}{n\sqrt{{\mathit{\xi }}_n}}}.$$

(73)

We denote

$$h^{-}(t,\mathit{\theta })=D_{n,k}^{-}({\mathit{\xi }}_n,⟨t,\mathit{\theta }⟩),h^{+}(t,\mathit{\theta })=D_{n,k}^{+}({\mathit{\xi }}_n,⟨t,\mathit{\theta }⟩),$$

and define

$${\mathcal{U}}_n={\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{\sqrt{{\mathit{\xi }}_n}n}}\right)}^{\gamma }+\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}{n\varphi \left(\mu {\varphi }^{-1}\left(\frac{{\rho }_n{k}_{1,n}}{n}\right)\right)}}},$$

for any increasing sequence ${\mathit{\xi }}_n\in (0,1)$ such that ${\mathit{\xi }}_n-1=O\left({\mathcal{U}}_n\right)$. It is important to observe that for all ${\mathit{\xi }}_n\in (0,1)$, $t\in S_{\mathcal{H}}$, and $k_{1,n}\le k\le k_{2,n}$, we have

$${\varphi }_{t,\mathit{\theta }}^{-1}\left({\displaystyle \frac{\sqrt{{\mathit{\xi }}_n}{k}_{1,n}}{n}}\right)\le h^{-}(t,\mathit{\theta })\le {\varphi }_{t,\mathit{\theta }}^{-1}\left({\displaystyle \frac{\sqrt{{\mathit{\xi }}_n}{k}_{2,n}}{n}}\right),$$

$${\varphi }_{t,\mathit{\theta }}^{-1}\left({\displaystyle \frac{k_{1,n}}{n\sqrt{{\mathit{\xi }}_n}}}\right)\le h^{+}(t,\mathit{\theta })\le {\varphi }_{t,\mathit{\theta }}^{-1}\left({\displaystyle \frac{k_{2,n}}{n\sqrt{{\mathit{\xi }}_n}}}\right).$$

Using Condition (8), it follows that the bandwidths $h^{-}(t,\mathit{\theta })$ and $h^{+}(t,\mathit{\theta })$ both belong to the interval

$$\left[h_{n,1},h_{n,2}\right]=\left[\mu {\varphi }^{-1}\left({\displaystyle \frac{{\rho }_n{k}_{1,n}}{n}}\right),\nu {\varphi }^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n}n}}\right)\right].$$

The remainder of the proof follows the same steps as those in [155] and is therefore omitted. □

8.2. Preliminaries of the Proofs

This section is primarily devoted to the analysis of functional conditional U-statistics. As in the case of $m=1$, where $S_{\mathcal{H}}$ is covered by

$$\bigcup _{\ell =1}^{N_{\epsilon }\left(S_{\mathcal{H}}\right)}B(t_{\ell },\epsilon ),$$

for some radius $\epsilon$, it follows that for each $\mathbf{t}\in S_{\mathcal{H}}^m$, there exists $\ell \left(\mathit{t}\right)=(\ell \left(t_1\right),\dots ,\ell \left(t_m\right))$, where $\forall 1\le i\le m,1\le \ell \left(t_i\right)\le N_{\epsilon }\left(S_{\mathcal{H}}\right)$, such that

$$\mathbf{t}\in \prod _{i=1}^m{B}_{{\mathit{\theta }}_i}(t_{\ell \left(t_i\right)},\epsilon )\mathrm{and}d(t_i,t_{\ell \left(t_i\right)})=arg\underset{1\le \ell \left(t_i\right)\le N_{\epsilon }\left(S_{\mathcal{H}}\right)}{min}{d}_{{\mathit{\theta }}_i}(t_i,t_{\ell }).$$

Thus, for each $\mathbf{t}\in S_{\mathcal{H}}^m$, the closest center is ${\mathbf{t}}_{\ell \left(\mathbf{t}\right)}$, and the corresponding ball with the closest center is

$$\prod _{i=1}^m{B}_{{\mathit{\theta }}_i}(t_{\ell \left(t_i\right)},\epsilon ):=B_{\mathit{\theta }}({\mathbf{t}}_{\ell \left(\mathbf{t}\right)},\epsilon ).$$

The consistency proofs for the uniform integrated bandwidth (UIB) concerning the multivariate bandwidths follow the same approach as the UIB consistency proofs for the univariate smoothing parameter in [49,51]. Furthermore, analogous to the proof of Theorem 1, we partition the sequence $\left\{({\mathbf{X}}_i,{\mathbf{Y}}_i)\right\}$ into ${\upsilon }_n$ alternating blocks, where the block sizes $a_n$ and $b_n$ differ, satisfying

$$b_n\ll a_n,({\upsilon }_n-1)(a_n+b_n)<n\le {\upsilon }_n(a_n+b_n),$$

(74)

and for $1\le j\le {\upsilon }_n-1$, we set

$$\begin{array}{ccc}\hfill H_j^{\left(U\right)}& =& \left\{i:(j-1)(a_n+b_n)+1\le i\le (j-1)(a_n+b_n)+a_n\right\},\hfill \\ \hfill T_j^{\left(U\right)}& =& \left\{i:(j-1)(a_n+b_n)+a_n+1\le i\le (j-1)(a_n+b_n)+a_n+b_n\right\},\hfill \\ \hfill H_{{\upsilon }_n}^{\left(U\right)}& =& \left\{i:({\upsilon }_n-1)(a_n+b_n)+1\le i\le ({\upsilon }_n-1)(a_n+b_n)+a_n\right\},\hfill \\ \hfill T_{{\upsilon }_n}^{\left(U\right)}& =& \left\{i:({\upsilon }_n-1)(a_n+b_n)+a_n+1\le i\le n\right\}.\hfill \end{array}$$

8.2.1. Proof of Theorem 3

In this section, we consider a bandwidth $\mathbf{h}=(h_1,\dots ,h_m)\in {\mathfrak{H}}_n^{\left(m\right)}$. To establish Theorem 3, we represent the U-statistic for each $\mathbf{t}\in S_{\mathcal{H}}^m$ as follows:

$$\begin{array}{c}{\displaystyle \left|u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-\mathbb{E}\left[u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right]\right|}\\ \le \left|u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-u_n(\phi ,{\mathbf{t}}_{\ell \left(\mathbf{t}\right)},\mathit{\theta };\mathbf{h})\right|\hfill \\ +\left|\mathbb{E}\left[u_n(\phi ,{\mathbf{t}}_{\ell \left(\mathbf{t}\right)},\mathit{\theta };\mathbf{h})\right]-\mathbb{E}\left[u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right]\right|\hfill \\ +\left|u_n(\phi ,{\mathbf{t}}_{\ell \left(\mathbf{t}\right)},\mathit{\theta };\mathbf{h})-\mathbb{E}\left[u_n(\phi ,{\mathbf{t}}_{\ell \left(\mathbf{t}\right)},\mathit{\theta };\mathbf{h})\right]\right|\hfill \end{array}$$

(75)

$$\begin{array}{cc}\le & \left|u_{n,11}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|+\left|u_{n,12}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|+\left|u_{n,13}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|.\hfill \end{array}$$

(76)

Let us begin with the term $\left|u_{n,11}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|$. We have

$$\left|u_{n,11}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|\le {\displaystyle \frac{\mathbb{C}(n-m)!}{n!\tilde{\varphi }\left(\mathbf{h}\right)}}\sum _{i\in I(m,n)}\left|\phi (Y_{i_1},\dots ,Y_{i_m})\left\{\prod _{j=1}^{m}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}}\right)-\prod _{j=1}^{m}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_i}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}}\right)\right\}\right|.$$

By applying the Telescoping binomial, we obtain

$$\begin{gathered} {\displaystyle \prod _{j=1}^{m}K\left(\frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}\right)-\prod _{j=1}^{m}K\left(\frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}\right)} \\ =\sum _{j=1}^m\left[\left\{K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}}\right)-K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}}\right)\right\}\right] \end{gathered}$$

(77)

$$\begin{array}{c}\times \prod _{q=1}^{j-1}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_q}(X_{iq},t_q)}}{h_q}}\right)\prod _{p=j+1}^{m}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_p}(X_{ip},t_{\ell \left(t_p\right)})}}{h_p}}\right).\hfill \end{array}$$

(78)

From Condition (C.4.1), we could claim that

$$\prod _{q=1}^{j-1}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_q}(X_{iq},t_q)}}{h_q}}\right)\le {\kappa }_2^{j-1}\prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right).$$

Similarly, we have

$$\prod _{p=j+1}^{m}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_p}(X_{ip},t_{\ell \left(t_p\right)})}}{h_p}}\right)\le {\kappa }_2^{m-j}\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right).$$

So, (77) satisfies

$$\begin{array}{ccc}\hfill {\displaystyle \prod _{j=1}^m}& K\hfill & \left(\frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}\right)-\prod _{j=1}^{m}K\left(\frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}\right)\hfill \\ & & \left.\prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right]\hfill \\ & \hfill \le & {\kappa }_2^{m-1}\sum _{j=1}^m\left[\left\{K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}}\right)-K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}}\right)\right\}{\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\right.\hfill \\ & & \left.\prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_j}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_j}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right]\hfill \\ & =:& \sum _{j=1}^m{\mathbf{K}}_{i_j,h_j}(t_j,{\mathit{\theta }}_j,t_{\ell \left(t_j\right)}),\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill {\mathbf{K}}_{i_j,h_j}^{(\ell )}(t_j,{\mathit{\theta }}_j,t_{\ell \left(t_j\right)})& =& {\kappa }_2^{m-1}\left\{K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}}\right)-K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}}\right)\right\}{\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\hfill \\ & & \times \prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right).\hfill \end{array}$$

Therefore, we infer that

$$\begin{array}{ccc}\hfill \left|u_{n,11}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|& \le & {\displaystyle \frac{\mathbb{C}(n-m)!}{n!\tilde{\varphi }\left(\mathbf{h}\right)}}{\kappa }_2^{m-1}\sum _{i\in I(m,n)}\left|\phi (Y_{i_1},\dots ,Y_{i_m})\sum _{j=1}^m{\mathbf{K}}_{i_j,h_j}^{(\ell )}(t_j,{\mathit{\theta }}_j,t_{\ell \left(t_j\right)})\right|\hfill \\ \hfill & \le & {\displaystyle \frac{\mathbb{C}(n-m)!}{n!\tilde{\varphi }\left(\mathbf{h}\right)}}M{\kappa }_2^{m-1}\sum _{i\in I(m,n)}\sum _{j=1}^m\left|\left\{K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}}\right)-K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}}\right)\right\}\right.\hfill \\ & & \left.\times {\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right|\hfill \\ \hfill & \le & {\displaystyle \frac{(n-m)!}{n!}}\mathbb{C}mM{\kappa }_2^{m-1}\sum _{i\in I(m,n)}{\displaystyle \frac{1}{m}}\sum _{j=1}^m\left[{\displaystyle \frac{{\epsilon }_n}{\tilde{\varphi }\left(\mathbf{h}\right)h_{n,j}}}{\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\right.\hfill \end{array}$$

(79)

$$\begin{array}{ccc}& & \left.\times \prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right].\hfill \end{array}$$

(80)

The transition from Equations (79) to (80) is enabled by the Lipschitz continuity of the kernel function $K(·)$. Uniformly for all $\mathbf{t}\in S_{\mathcal{H}}^m$ and $\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}$, we derive the following:

$$\begin{array}{c}{\displaystyle \underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\left|u_{n,11}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|}\hfill \\ \le \underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}{\displaystyle \frac{(n-m)!}{n!}}\sum _{i\in I(m,n)}{\displaystyle \frac{1}{m}}\sum _{j=1}^m\left[{\displaystyle \frac{\mathbb{C}mM{\kappa }_2^{m-1}{\epsilon }_n}{\tilde{\varphi }\left({\mathbf{H}}_n\right)h_{n,j}}}{\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\right.\hfill \\ \left.\times \prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right]\hfill \\ \le \underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}{\displaystyle \frac{(n-m)!}{n!}}\sum _{i\in I(m,n)}{\displaystyle \frac{1}{m}}\sum _{j=1}^m\left[{\displaystyle \frac{\mathbb{C}mM{\kappa }_2^{m-1}{\epsilon }_n}{\tilde{\varphi }\left({\tilde{\mathbf{H}}}_{\mathbf{n}}\right){\tilde{h}}_n}}{\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\right.\hfill \\ \left.\times \prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right]\hfill \\ \le \underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}{\displaystyle \frac{(n-m)!}{n!}}\sum _{i\in I(m,n)}{\displaystyle \frac{1}{m}}\sum _{j=1}^m\left[{\displaystyle \frac{\mathbb{C}mM{\kappa }_2^{m-1}{\epsilon }_n}{C_1^{\prime }\varphi \left({\tilde{h}}_n\right){\tilde{h}}_n}}{\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\right.\hfill \\ \left.\times \prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right]\hfill \end{array}$$

by (14), where ${\mathbf{H}}_n=(h_{n,1},\dots ,h_{n,m})$ and ${\tilde{\mathbf{H}}}_n=({\tilde{h}}_n,\dots ,{\tilde{h}}_n)$, with ${\tilde{\mathbf{H}}}_n\le {\mathbf{H}}_n$ component by component. The idea is to apply Lemma A21 on function

$$f_{\mathbf{t},\mathbf{h},\mathit{\theta }}\left({\mathbf{X}}_{\mathbf{i}}\right)={\displaystyle \frac{1}{m}}\sum _{j=1}^m\left[{\displaystyle \frac{\mathbb{C}mM{\kappa }_2^{m-1}{\epsilon }_n}{C_1^{\prime }\varphi \left({\tilde{h}}_n\right)h_{n,j}}}{\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right],$$

which satisfies, for all $\mathbf{t}\in S_{\mathcal{H}}^m$ and $\mathit{\theta }\in {\mathbf{\Theta }}^m$,

$$0\le \underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|f_{\mathbf{t},\mathbf{h}}\left(\mathbf{X}\right)\right|\le {\displaystyle \frac{\mathbb{C}mM{\kappa }_2^{m-1}{\epsilon }_n}{C_1^{\prime }\varphi \left({\tilde{h}}_n\right){\tilde{h}}_n}}\le Cte=C_7.$$

It is important to observe that Constant $C_7$ appearing on the final right-hand side of the preceding inequality is derived from Condition (7). At this stage, we can invoke Lemma A21 with ${\displaystyle x=\sqrt{\frac{{\psi }_{S_{\mathcal{H}}}\left({\epsilon }_n\right)+{\psi }_{\mathbf{\Theta }}\left({\epsilon }_n\right)}{n\varphi \left({\tilde{h}}_n\right)}}-\frac{{\epsilon }_n}{{\tilde{h}}_n}}$, yielding

$$\begin{array}{cc}\hfill \mathbb{P}& \left\{\underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|u_{n,11}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|\ge \sqrt{\frac{{\psi }_{S_{\mathcal{H}}}\left({\epsilon }_n\right)+{\psi }_{\mathbf{\Theta }}\left({\epsilon }_n\right)}{n\varphi \left({\tilde{h}}_n\right)}}\right\}\hfill \\ \hfill & \le \mathbb{P}\left\{\left|u_n^{\left(m\right)}\left(f\right)-\mathit{\theta }\right|\ge \sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left({\epsilon }_n\right)+{\psi }_{\mathbf{\Theta }}\left({\epsilon }_n\right)}{n\varphi \left({\tilde{h}}_n\right)}}}-{\displaystyle \frac{{\epsilon }_n}{{\tilde{h}}_n}}+C_{\gamma ,m}/\sqrt{n}\right\}\hfill \end{array}$$

(81)

$$\begin{array}{ccc}& \le & 2exp\left\{-{\displaystyle \frac{C_{\gamma ,m}^{\prime }{x}^{2}n}{C_7^2+C_{7}x(logn)\left({log}_2\left(n\right)\right)}}\right\}\le n^{-c{ϵ}_0},\hfill \end{array}$$

(82)

such that $c{ϵ}_0>1$. By rigorously performing the calculations while adhering to the specified conditions, particularly (C.6) and (C.7), we derive the following result:

$$\sum _{n\ge 1}\mathbb{P}\left\{\underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|u_{n,11}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|\ge \sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left({\epsilon }_n\right)+{\psi }_{\mathbf{\Theta }}\left({\epsilon }_n\right)}{n\varphi \left({\tilde{h}}_n\right)}}}\right\}<\infty .$$

(83)

The study of term $u_{n,12}$ is deduced from the previous one. In fact,

$$\begin{gathered} {\displaystyle \left|u_{n,12}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|} \\ =\left|\mathbb{E}\left[u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right]\right| \end{gathered}$$

$$\begin{array}{ccc}& & \le {\displaystyle \frac{\mathbb{C}(n-m)!}{n!\tilde{\varphi }\left(\mathbf{h}\right)}}\left|\sum _{i\in I(m,n)}\mathbb{E}\left(\phi (Y_{i_1},\dots ,Y_{i_m})\left\{\prod _{j=1}^{m}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}}\right)-\prod _{j=1}^{m}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}}\right)\right\}\right)\right|\hfill \end{array}$$

(84)

$$\begin{array}{ccc}& & \le {\displaystyle \frac{\mathbb{C}(n-m)!}{n!\tilde{\varphi }\left(\mathbf{h}\right)}}\sum _{i\in I(m,n)}\mathbb{E}\left|\phi (Y_{i_1},\dots ,Y_{i_m})\left\{\prod _{j=1}^{m}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}}\right)-\prod _{j=1}^{m}K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}}\right)\right\}\right|.\hfill \end{array}$$

(85)

To facilitate the transition from (84) to (85), we apply Jensen’s inequality, leveraging specific properties of the absolute value function. By proceeding along the same steps as previously outlined, we obtain

$$\left|u_{n,12}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|\le \mathbb{C}\left|u_{n,12}^{\prime }(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|,$$

where

$$\left|u_{n,12}^{\prime }(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|:={\displaystyle \frac{(n-m)!}{n!\tilde{\varphi }\left(\mathbf{h}\right)}}\sum _{i\in I(m,n)}\mathbb{E}\left|\phi (Y_{i_1},\dots ,Y_{i_m})\sum _{j=1}^m{\mathbf{K}}_{i_j,h_j}^{(\ell )}(t_j,{\mathit{\theta }}_j,t_{\ell \left(t_j\right)})\right|.$$

Notice that we have

$$\begin{array}{ccc}\hfill \left|u_{n,12}^{\prime }(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|& \le & {\displaystyle \frac{(n-m)!}{n!\tilde{\varphi }\left(\mathbf{h}\right)}}M{\kappa }_2^{m-1}\sum _{i\in I(m,n)}\sum _{j=1}^m\mathbb{E}\left|\left\{K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_j)}}{h_j}}\right)-K\left({\displaystyle \frac{{\displaystyle d_{{\mathit{\theta }}_j}(X_{ij},t_{\ell \left(t_j\right)})}}{h_j}}\right)\right\}\right.\hfill \\ & & \left.\times {\mathbb{1}}_{B(t_j,h_j)\cup B(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right|\hfill \\ \hfill & \le & {\displaystyle \frac{(n-m)!}{n!}}mM{\kappa }_2^{m-1}\sum _{i\in I(m,n)}{\displaystyle \frac{1}{m}}\sum _{j=1}^m\left[{\displaystyle \frac{{\epsilon }_n}{\tilde{\varphi }\left({\mathbf{H}}_n\right)h_{n,j}}}\mathbb{E}\left({\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\right.\right.\hfill \\ & & \left.\left.\times \prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right)\right]\hfill \\ \hfill & \le & {\displaystyle \frac{(n-m)!}{n!}}mM{\kappa }_2^{m-1}\sum _{i\in I(m,n)}{\displaystyle \frac{1}{m}}\sum _{j=1}^m\left[{\displaystyle \frac{{\epsilon }_n}{C_1^{\prime }\tilde{\varphi }\left({\tilde{h}}_n\right)h_{n,j}}}\mathbb{E}\left({\mathbb{1}}_{B_{{\mathit{\theta }}_j}(t_j,h_j)\cup B_{{\mathit{\theta }}_j}(t_{\ell \left(t_j\right)},h_j)}\left(X_{ij}\right)\right.\right.\hfill \\ & & \left.\left.\times \prod _{q=1}^{j-1}{\mathbb{1}}_{B_{{\mathit{\theta }}_q}\left(t_q,h_q\right)}\left(X_{iq}\right)\prod _{p=j+1}^m{\mathbb{1}}_{B_{{\mathit{\theta }}_p}\left(t_{\ell \left(t_p\right)},h_p\right)}\left(X_{ip}\right)\right)\right].\hfill \end{array}$$

That implies

$$\begin{array}{ccc}\hfill \underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|u_{n,12}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|& \le & mM{\kappa }_2^{m-1}{\displaystyle \frac{log\left(n\right)}{n{\tilde{h}}_n}}\le C_7^{\prime }{\displaystyle \frac{log\left(n\right)}{n\varphi \left({\tilde{h}}_n\right)}}\hfill \\ & =& O\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left({\epsilon }_n\right)+{\psi }_{\mathbf{\Theta }}\left({\epsilon }_n\right)}{n\varphi \left({\tilde{h}}_n\right)}}}\right).\hfill \end{array}$$

This offers

$$\underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|u_{n,12}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right|=O\left(\sqrt{{\displaystyle \frac{{\psi }_{S_{\mathcal{H}}}\left({\epsilon }_n\right)+{\psi }_{\mathbf{\Theta }}\left({\epsilon }_n\right)}{n\varphi \left({\tilde{h}}_n\right)}}}\right).$$

(86)

Continuing, now with $u_{n,13}$,

$$\left|u_{n,13}(\phi ,{\mathbf{t}}_{\ell },\mathit{\theta };\mathbf{h})\right|=\left|u_n(\phi ,{\mathbf{t}}_{\ell };\mathbf{h})-\mathbb{E}\left[u_n(\phi ,{\mathbf{t}}_{\ell };\mathbf{h})\right]\right|.$$

Assuming symmetry of the kernel function $G_{\phi ,{\mathbf{t}}_{\ell },\mathit{\theta },\mathbf{h}}(·)$, it becomes necessary to decompose the U-statistic using the [1] decomposition, leading to the following expression:

$$\begin{array}{ccc}\hfill u_{n,13}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})& :=& u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-\mathbb{E}\left[u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right]\hfill \\ \hfill & =& \sum _{p=1}^m{\displaystyle \frac{m!}{(m-p)!}}{u}_n^{\left(p\right)}\left({\pi }_{p,m}\left(G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}\right)\right)\hfill \\ & =& m{u}_n^{\left(1\right)}\left({\pi }_{1,m}\left(G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}\right)\right)+\sum _{p=2}^m{\displaystyle \frac{m!}{(m-p)!}}{u}_n^{\left(p\right)}\left({\pi }_{p,m}\left(G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}\right)\right).\hfill \end{array}$$

(87)

We define new classes of functions for $\mathbf{h}=(h_1,\dots ,h_m)\in {\mathfrak{H}}_n^{\left(m\right)}$, $\ell \in {\left\{1,\dots ,N_{{\epsilon }_n}\left(S_{\mathcal{H}}\right)\right\}}^m$, $\overline{\ell }\in {\left\{1,\dots ,N_{{\epsilon }_n}\left(\mathbf{\Theta }\right)\right\}}^m$ and $1\le p\le m:$

$${\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}^{\left(p\right)}:=\left\{{\varphi }_{{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }}}\left(\mathbf{h}\right){\pi }_{p,m}\left(G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right)(·,·)for\mathbf{h}\in {\mathcal{H}}_0^{\left(m\right)}\mathrm{and}\phi \in {\mathcal{F}}_m\right\}.$$

These classes are VC-type classes of functions with the same characteristics and the envelope function $F_p$ satisfying

$$F_p\le 2^p{\kappa }_2^m{\parallel F\parallel }_{\infty }.$$

Let us start with the linear term of (87), which is

$$m{u}_n^{\left(1\right)}\left({\pi }_{1,m}\left(G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right)\right)={\displaystyle \frac{m}{n}}\sum _{j=1}^m{\pi }_{1,m}\left(G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right)(X_i,Y_i).$$

From Hoeffding’s projection, we have

$$\begin{array}{ccc}\hfill {\pi }_{1,m}\left(G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right)(x,y)& =& \left\{\mathbb{E}\left[G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((x,X_2,\dots ,X_m),(y,Y_2,\dots ,Y_m)\right)\right]-\mathbb{E}\left[G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(\mathbf{X},\mathbf{Y}\right)\right]\right\}\hfill \\ & =& \left\{\mathbb{E}\left[G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(\mathbf{X},\mathbf{Y}\right)\right|(X_1,Y_1)=(x,y)]-\mathbb{E}\left[G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(\mathbf{X},\mathbf{Y}\right)\right]\right\}.\hfill \end{array}$$

One can see that

$$m{u}_n^{\left(1\right)}\left({\pi }_{1,m}\left(G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right)\right)=:{\displaystyle \frac{1}{\sqrt{n}}}{\alpha }_n\left(S_{1,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right),$$

is an empirical process based on a VC-type class of functions contained in $m{\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}^{\left(1\right)}$ with the same characteristics, and the elements are defined by

$$S_{1,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}(x,y)=m{\varphi }_{{\mathbf{t}}_{\ell }\left(\mathbf{h}\right)}\mathbb{E}\left[G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(\mathbf{X},\mathbf{Y}\right)\right|(X_1,Y_1)=(x,y)].$$

Hence, the proof of this part is similar to that of the Lemma A2, and then

$$\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\underset{{\tilde{h}}_n\le h_{\ell }\le b_0}{sup}\underset{\phi \in {\mathcal{F}}_m}{sup}\left|u_n^{\left(1\right)}\left({\pi }_{1,m}\left(G_{\phi ,{\mathbf{t}}_{\ell },\mathit{\theta },\mathbf{h}}\right)\right)\right|=O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{{\displaystyle n}}\right)}}{{\displaystyle n\varphi \left({\tilde{h}}_n\right)}}}}\right).$$

Now, let us proceed to the nonlinear terms. The objective is to demonstrate that, for $2\le p\le m,$

$$\underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathfrak{H}}_n^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}{\displaystyle \frac{{\displaystyle \left(\genfrac{}{}{0pt}{}{m}{p}\right)\sqrt{n\varphi \left({\tilde{h}}_n\right)}\left|u_n^{\left(p\right)}\left({\pi }_{p,m}\left(G_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}\right)\right)\right|}}{{\displaystyle \sqrt{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{{\displaystyle n}}\right)}}}}=O_{a.co}\left(1\right).$$

(88)

To achieve that, we need to decompose the interval ${\displaystyle \prod _{j=1}^m(h_{n,j},h_{n,j}^{\prime })}$ into smaller intervals. First, let us consider the intervals ${\displaystyle \left(h_{n,j},b_j\right)}$ for all $j=1,\dots ,m\mathrm{and}{b}_j\in (0,1)$. We note

$${\mathcal{H}}_{i_j}=\left[h_{i_j},h_{i_j}^{\prime }\right],$$

where $h_{i_j}=2^{i-1}{h}_{n,j}$ and $h_{i_j}^{\prime }=2^i{h}_{n,j}$, and we set $L_j\left(n\right)=max\left\{i:h_{i_j}^{\prime }\le 2b_j\right\},$ and ${\mathcal{I}}_j=\left\{i_j:1\le i_j\le L_j\left(n\right)\right\}.$ We can observe that

$${\displaystyle \prod _{j=1}^m\left(h_{n,j},b_j\right)\subseteq \bigcup _{(i_1,\dots ,i_m)\in {\mathcal{I}}_1\times \dots \times {\mathcal{I}}_m}\prod _{j=1}^m{\mathcal{H}}_{i_j},}$$

and

$$\begin{array}{c}\hfill L_j\left(n\right)\sim {\displaystyle \frac{log(b_j/h_{n,j})}{log\left(2\right)}}\le L\left(n\right)=:max{L}_j\left(n\right)\mathrm{for}1\le j\le m.\end{array}$$

(89)

Now, we set the following new classes for $\mathbf{h}=(h_1,\dots ,h_m)\in {\mathfrak{H}}_n^{\left(m\right)}$, $\ell \in {\left\{1,\dots ,N_{{\epsilon }_n}\left(S_{\mathcal{H}}\right)\right\}}^m$, and $\overline{\ell }\in {\left\{1,\dots ,N_{{\epsilon }_n}\left(\mathbf{\Theta }\right)\right\}}^m$, $1\le j\le m$, $1\le i\le L\left(n\right)$ and $2\le p\le m:$

$$\begin{array}{ccc}\hfill {\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}_{i_j,\ell ,\overline{\ell }}& :=& \left\{{\varphi }_{{\mathbf{t}}_{\ell }}\left(\mathbf{h}\right)G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}(·,·)\mathrm{where}\phi \in {\mathcal{F}}_m\mathrm{and}\mathbf{h}\in \prod _{j=1}^m{\mathcal{H}}_{i_j}\right\},\hfill \\ \hfill {\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}_{i_j,\ell ,\overline{\ell }}^{\left(p\right)}& :=& \left\{{\varphi }_{{\mathbf{t}}_{\ell }}\left(\mathbf{h}\right){\pi }_{p,m}\left(G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right)(·,·)\mathrm{where}\phi \in {\mathcal{F}}_m\mathrm{and}\mathbf{h}\in \prod _{j=1}^m{\mathcal{H}}_{i_j}\right\}.\hfill \end{array}$$

Thus, to prove (88), we need to prove that for $1\le j\le m$, $\ell =\left({\ell }_1,\dots ,{\ell }_m\right)$ and $\overline{\ell }=\left({\overline{\ell }}_1,\dots ,{\overline{\ell }}_m\right)$,

$$\underset{1\le {\ell }_j\le N_{{\epsilon }_n}\left(S_{\mathcal{H}}\right)}{max}\underset{1\le {\overline{\ell }}_j\le N_{{\epsilon }_n}\left(\mathbf{\Theta }\right)}{max}\underset{1\le i\le L\left(n\right)}{max}\underset{\mathbf{h}\in {\prod }_{j=1}^m{\mathcal{H}}_{i_j}}{sup}\underset{\phi K\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}{\displaystyle \frac{{\displaystyle \left(\genfrac{}{}{0pt}{}{m}{p}\right)\sqrt{n\varphi \left({\tilde{h}}_n\right)}\left|u_n^{\left(p\right)}\left({\pi }_{p,m}(G_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right)\right|}}{{\displaystyle \sqrt{{\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{{\displaystyle n}}\right)}}}}=O_{a.co}\left(1\right).$$

Notice that for each ${ϵ}_0,$$1\le j\le m$, $\ell =\left({\ell }_1,\dots ,{\ell }_m\right)$ and $\overline{\ell }=\left({\overline{\ell }}_1,\dots ,{\overline{\ell }}_m\right)$, we have

$$\begin{array}{cc}\hfill \mathbb{P}& \left\{\underset{\mathbf{h}\in {\mathcal{H}}_0^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\frac{\left(\genfrac{}{}{0pt}{}{m}{p}\right)\sqrt{n\varphi \left({\tilde{h}}_n\right)}\left|u_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}\right)\right|}{\sqrt{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{{\displaystyle n}}\right)}}}\ge {ϵ}_0\right\}\hfill \\ & \le \sum _{1\le {\ell }_j\le N_{{\epsilon }_n}\left(S_{\mathcal{H}}\right)}\sum _{1\le {\overline{\ell }}_j\le N_{{\epsilon }_n}\left(\mathbf{\Theta }\right)}\sum _{1\le i\le L\left(n\right)}\mathbb{P}\left\{\underset{\mathbf{h}\in {\prod }_{j=1}^m{\mathcal{H}}_{i_j}}{sup}\underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}{\displaystyle \frac{\left(\genfrac{}{}{0pt}{}{m}{p}\right)\sqrt{n\varphi \left({\tilde{h}}_n\right)}\left|u_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right)\right|}{\sqrt{{\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{{\displaystyle n}}\right)}}}\ge {ϵ}_0\right\}\hfill \\ & \le L\left(n\right)N_{{\epsilon }_n}^m\left(S_{\mathcal{H}}\right)N_{{\epsilon }_n}^m\left(\mathbf{\Theta }\right)\underset{1\le {\ell }_j\le N_{{\epsilon }_n}\left(S_{\mathcal{H}}\right)}{max}\underset{1\le {\overline{\ell }}_j\le N_{{\epsilon }_n}\left(\mathbf{\Theta }\right)}{max}\underset{1\le i\le L\left(n\right)}{max}\hfill \\ & \times \mathbb{P}\left\{{∥\sum _{p=2}^m\left({\displaystyle \genfrac{}{}{0pt}{}{m}{p}}\right)u_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,{\mathbf{t}}_{\ell \left(\mathbf{t}\right)},\mathit{\theta },\mathbf{h}}\right)∥}_{{\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}_{i_j,\ell ,\overline{\ell }}}\ge {ϵ}_0\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{{\displaystyle n}}\right)}}{n\varphi \left({\tilde{h}}_n\right)}}}\right\}.\hfill \end{array}$$

At this stage, we focus on analyzing the aforementioned equation for the case of $m=2$ to simplify the proof (the same reasoning applies for $m>2$). Thus, we obtain

$$\begin{array}{cc}\hfill & u_n^{\left(2\right)}\left({\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\right)={\displaystyle \frac{1}{n\left(n-1\right)}}\sum _{1⩽<i<j⩽n}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,Y_i),(X_j,Y_j)\right),\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill \sum _{i\ne j}^n& {\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)\hfill \\ & =\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)\hfill \\ & +\sum _{p=1}^{{\upsilon }_n}\sum _{i\ne ji,j\in H_p^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)\hfill \\ & +2\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)\hfill \\ & +2\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩽1}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)\hfill \\ & +\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in T_p^{\left(U\right)}}\sum _{j\in T_q^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)\hfill \\ & +\sum _{p=1}^{{\upsilon }_n}\sum _{i\ne ji,j\in T_p^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)\hfill \\ & =:\mathbb{I}+\mathbb{II}+\mathbb{III}+\mathbb{IV}+\mathbb{V}+\mathbb{VI}.\hfill \end{array}$$

(90)

We begin by analyzing the term $\mathbb{I}$. Assuming that the sequence of independent blocks ${\left\{{\mathit{\xi }}_i=({\varsigma }_i,{\zeta }_i)\right\}}_{i\in {\mathbb{N}}^{*}}$ has a size of $a_n$, an application of [119] demonstrates that

$$\begin{array}{cc}\hfill \mathbb{P}& \left\{{∥\frac{1}{n\left(n-1\right)}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)∥}_{{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}}\right.\hfill \\ & \left.\ge {ϵ}_0\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}{n\varphi \left({\tilde{h}}_n\right)}}}\right\}\hfill \\ & \le \mathbb{P}\left\{{∥{\displaystyle \frac{1}{n\left(n-1\right)}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)∥}_{{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}}\right.\hfill \\ & \left.\ge {ϵ}_0\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}{n\varphi \left({\tilde{h}}_n\right)}}}\right\}+2{\upsilon }_n{\beta }_{b_n}.\hfill \end{array}$$

We retain the choice of ${\upsilon }_n$ and $b_n$ such that ${\upsilon }_n{b_n}^r\le 1$, which implies that $2{\upsilon }_n{\beta }_{b_n}\to 0$ as $n\to \infty$. Consequently, the primary term under consideration is the second summand. The central idea is to employ Lemma A19. It is clear that the class of functions ${\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}_{i_j,\ell ,\overline{\ell }}$ is uniformly bounded, i.e.,

$$\underset{\phi \tilde{K}\in {\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}}{sup}\left|\phi (·)\tilde{K}(·)\right|\le M{\kappa }_2^2.$$

Furthermore, by applying Proposition 2.6 from [10], we obtain the following for every $(x_i,y_i)\in S_{\mathcal{H}}\times \mathcal{Y}$ and for Rademacher variables ${ϵ}_i$:

$$\begin{array}{cc}\hfill \mathbb{E}& {∥n^{-1}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)∥}_{{\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}_{i_j,\ell ,\overline{\ell }}}\hfill \\ & ⩽c_2\mathbb{E}{∥n^{-1}\sum _{p\ne q}^{{\upsilon }_n}{ϵ}_p{ϵ}_q\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)∥}_{{\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}_{i_j,\ell ,\overline{\ell }}}\hfill \\ & ⩽c_2\mathbb{E}{\int }_0^{D_{2,n}}logN\left(ϵ,{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }},d_{2,n}\right)dϵ,\hfill \end{array}$$

(91)

where

$$D_{2,n}:={∥{\mathbb{E}}_{ϵ}^{1/2}\left\{n^{-2}\sum _{p\ne q}^{{\upsilon }_n}{\left({ϵ}_p{ϵ}_q\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)\right)}^2\right\}∥}_{{\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}_{i_j,\ell ,\overline{\ell }}}$$

and

$$d_{2,n}:={\mathbb{E}}_{ϵ}^{1/2}\left\{n^{-2}\sum _{p\ne q}^{{\upsilon }_n}{\left({ϵ}_p{ϵ}_q\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)\right)}^2\right\}.$$

We see that

$$D_{2,n}\le n^{-1}{\displaystyle \frac{n!}{(n-2)!}}M{\kappa }_2^2\le nM{\kappa }_2^2.$$

Given that ${\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2$ is a VC-type class of functions that satisfies (C.4.4), it follows that the class ${\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}$ is also a VC-type class of functions with similar characteristics to ${\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2$. Therefore,

$$\begin{array}{cc}\hfill \mathbb{E}& {∥n^{-1}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)∥}_{{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}}\hfill \\ & \le c_2\mathbb{E}{\int }_0^{D_{2,n}}logN\left(ϵ,{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }},d_{2,n}\right)dϵ\hfill \\ & \le c_{2}nM{C}^2.\hfill \end{array}$$

(92)

All the conditions of Lemma A19 are satisfied; thus, a direct application yields, for each ${ϵ}_0$,

$$\begin{array}{cc}\hfill \mathbb{P}& \left\{{∥\frac{1}{n\left(n-1\right)}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)∥}_{{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}}\right.\hfill \\ & \left.\ge {ϵ}_0\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}{n\varphi \left({\tilde{h}}_n\right)}}}\right\}\hfill \\ & \le 2exp\left(-{\displaystyle \frac{{ϵ}_0\sqrt{\frac{{\displaystyle 2\left({\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)\right)}}{{\displaystyle n\varphi \left({\tilde{h}}_n\right)}}}}{2^7{2}^3{\displaystyle c_2{M}^2{\kappa }_2^4}}}\right)\hfill \\ & \le n^{-{\epsilon }_0{C}_2^{\prime }},\hfill \end{array}$$

(93)

where

$${\epsilon }_n={\displaystyle \frac{logn}{n}},C_2^{\prime }>0.$$

Next, let us study the same blocks $\mathbb{II}$. We have

$$\begin{array}{cc}\hfill \mathbb{P}& \left\{{∥\frac{1}{n\left(n-1\right)}\sum _{p=1}^{{\upsilon }_n}\sum _{i\ne ji,j\in H_p^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)∥}_{{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}}\right.\hfill \\ & \left.\ge {ϵ}_0\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}{n\varphi \left({\tilde{h}}_n\right)}}}\right\}\hfill \\ & \le \mathbb{P}\left\{{∥{\displaystyle \frac{1}{n\left(n-1\right)}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\ne ji,j\in H_p^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)∥}_{{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}}\right.\hfill \\ & \left.\ge {ϵ}_0\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}{n\varphi \left({\tilde{h}}_n\right)}}}\right\}+2{\upsilon }_n{\beta }_{b_n}.\hfill \end{array}$$

Following the same argument as in blocks $\mathbb{I}$, we obtain

$$\begin{array}{cc}\hfill \mathbb{E}& {∥n^{-1}\sum _{p=1}^{{\upsilon }_n}\sum _{i\ne ji,j\in H_p^{\left(U\right)}}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)∥}_{{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}}\hfill \\ & ⩽c_2\mathbb{E}{∥n^{-1}\sum _{p=1}^{{\upsilon }_n}{ϵ}_p\sum _{i\ne ji,j\in H_p^{\left(U\right)}}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)∥}_{{\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}_{i_j,\ell ,\overline{\ell }}}\hfill \\ & \le c_2\mathbb{E}{\int }_0^{D_{2,n}^{\left(2\right)}}logN\left(ϵ,{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }},d_{2,n}^{\left(2\right)}\right)dϵ\hfill \\ & \le c_{2}nM{C}^{\prime 2},\hfill \end{array}$$

(94)

where

$$D_{2,n}^{\left(2\right)}:={∥{\mathbb{E}}_{ϵ}^{1/2}\left\{n^{-2}{\left(\sum _{p=1}^{{\upsilon }_n}{ϵ}_p\sum _{i\ne ji,j\in H_p^{\left(U\right)}}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)\right)}^2\right\}∥}_{{\left({\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m\right)}_{i_j,\ell ,\overline{\ell }}}$$

and

$$d_{2,n}^{\left(2\right)}:={\mathbb{E}}_{ϵ}^{1/2}\left\{n^{-2}{\left(\sum _{p=1}^{{\upsilon }_n}{ϵ}_p\sum _{i\ne ji,j\in H_p^{\left(U\right)}}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)\right)}^2\right\}.$$

Again, using Lemma A19, we readily obtain

$$\begin{array}{cc}\hfill {\displaystyle \mathbb{P}}& \left\{{∥\frac{1}{n\left(n-1\right)}\sum _{p=1}^{{\upsilon }_n}\sum _{i\ne ji,j\in H_p^{\left(U\right)}}{\pi }_{2,m}{G}_{\phi ,{\mathbf{t}}_{\ell },{\mathit{\theta }}_{\overline{\ell }},\mathbf{h}}\left((X_i,X_j),(Y_i,Y_j)\right)∥}_{{\left({\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2\right)}_{i_j,\ell ,\overline{\ell }}}\right.\hfill \\ & \left.\ge {ϵ}_0\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}{n\varphi \left({\tilde{h}}_n\right)}}}\right\}\hfill \\ & \le 2exp\left(-{\displaystyle \frac{{ϵ}_0\sqrt{\frac{{\displaystyle 2{\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}{{\displaystyle n\varphi \left({\tilde{h}}_n\right)}}}}{2^7{2}^3{\displaystyle c_2{M}^2{\kappa }_2^4}}}\right)\hfill \end{array}$$

(95)

$$\begin{array}{ccc}& \le & n^{-{\epsilon }_0{C}_3^{\prime }}.\hfill \end{array}$$

(96)

The results for the remaining blocks can be derived by following the same approach as described above. Consequently, we obtain

$$\begin{array}{cc}\hfill & \sum _{n\ge 1}\mathbb{P}\left\{\underset{\mathbf{h}\in {\mathcal{H}}_0^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}{\displaystyle \frac{\left(\genfrac{}{}{0pt}{}{m}{p}\right)\sqrt{n\varphi \left({\tilde{h}}_n\right)}\left|u_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,{\mathbf{t}}_{\ell \left(\mathbf{t}\right)},\mathit{\theta },\mathbf{h}}\right)\right|}{\sqrt{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}}}\ge {ϵ}_0\right\}\hfill \\ \hfill & \le \sum _{n\ge 1}L\left(n\right)n^{-{\epsilon }_0{C}_m^{\prime }}\hfill \\ \hfill & \le \infty ,\hfill \end{array}$$

where

$${\epsilon }_0={\displaystyle \frac{logn}{n}},C_m^{\prime }>0.$$

With this, the proof of the theorem is concluded. □

8.2.2. Proof of Theorem 5

We observe that

$$\begin{array}{c}{\displaystyle \left|{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-\widehat{\mathbb{E}}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)\right|}\hfill \\ =\left|{\displaystyle \frac{u_n(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h})}{u_n(1,\mathbf{t},\mathit{\theta },\mathbf{h})}}-{\displaystyle \frac{\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h})\right)}{\mathbb{E}\left(u_n(1,\mathbf{t},\mathit{\theta },\mathbf{h})\right)}}\right|\hfill \\ \le {\displaystyle \frac{\left|u_n(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h})-\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h})\right)\right|}{\left|u_n(1,\mathbf{t},\mathit{\theta },\mathbf{h})\right|}}\hfill \\ +{\displaystyle \frac{\left|\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h})\right)\right|·\left|u_n(1,\mathbf{t},\mathit{\theta },\mathbf{h})-\mathbb{E}\left(u_n(1,\mathbf{t},\mathit{\theta },\mathbf{h})\right)\right|}{\left|u_n(1,\mathbf{t},\mathit{\theta },\mathbf{h})\right|·\left|\mathbb{E}\left(u_n(1,\mathbf{t},\mathit{\theta },\mathbf{h})\right)\right|}}\hfill \\ =:\mathbf{I}+\mathbf{II}.\hfill \end{array}$$

Given the imposed assumptions and the results derived earlier, and for certain constants $c_1^{\prime }$ and $c_2^{\prime }$, we obtain that

$$\begin{array}{ccc}\hfill \underset{\mathbf{h}\in {\mathcal{H}}_0^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\left|u_n(1,\mathbf{t},\mathit{\theta },\mathbf{h})\right|& =& c_1^{\prime }\mathrm{a}.\mathrm{co}\hfill \\ \hfill \underset{\mathbf{h}\in {\mathcal{H}}_0^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\left|\mathbb{E}\left(u_n(1,\mathbf{t},\mathit{\theta },\mathbf{h})\right)\right|& =& c_2^{\prime }\hfill \\ \hfill \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathcal{H}}_0^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}\left|\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h})\right)\right|& =& O\left(1\right).\hfill \end{array}$$

Hence, we can now apply Theorem 3 to handle $\mathbf{II}$ and Theorems 3 and 4 to handle $\mathbf{I}$ depending on whether class ${\mathcal{F}}_m$ satisfies (C.4.1) or (C.4.2). As a result, we obtain, for some $c^{″}>0$, with probability 1,

$$\begin{array}{cc}\hfill \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathcal{H}}_0^{\left(m\right)}}{sup}& \underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}{\displaystyle \frac{\sqrt{n\varphi \left({\tilde{h}}_n\right)}\left|{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})-\widehat{\mathbb{E}}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right)\right|}{\sqrt{{\psi }_{S_{\mathcal{H}}}\left({\epsilon }_n\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}}\hfill \\ \hfill & \le \underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\in {\mathcal{H}}_0^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}{\displaystyle \frac{\sqrt{n\varphi \left({\tilde{h}}_n\right)}\left(\mathbf{I}\right)}{\sqrt{{\psi }_{S_{\mathcal{H}}}\left({\epsilon }_n\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}}\hfill \\ & +\underset{\phi \tilde{K}\in {\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}{sup}\underset{\mathbf{h}\left(\mathbf{t}\right)\in {\mathcal{H}}_0^{\left(m\right)}}{sup}\underset{\mathbf{t}\in S_{\mathcal{H}}^m}{sup}{\displaystyle \frac{\sqrt{n\varphi \left({\tilde{h}}_n\right)}\left(\mathbf{II}\right)}{\sqrt{{\psi }_{S_{\mathcal{H}}}\left({\epsilon }_n\right){\psi }_{\mathbf{\Theta }}\left(\frac{{\displaystyle logn}}{{\displaystyle n}}\right)}}}\hfill \\ \hfill & \le c^{″}.\hfill \end{array}$$

Thus, the proof is concluded. □

8.2.3. Proof of Theorem 6

Given Conditions (C.3.1), we have

$$\begin{array}{c}{\displaystyle \left|\frac{{\displaystyle \mathbb{E}\left[u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right]}}{{\displaystyle \mathbb{E}\left[u_n(1,\mathbf{t};\mathbf{h})\right]}}-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|}\hfill \\ \le \left|{\kappa }_1^{-m}{C}_1^{-1}\mathbb{E}\left[u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})\right]-r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|\hfill \\ \le \left|{\kappa }_1^{-m}{C}_1^{-1}\mathbb{E}\left({\displaystyle \frac{(n-m)!}{n!}}\sum _{i\in I(m,n)}{\displaystyle \frac{1}{{\varphi }_{\mathbf{t},\mathit{\theta }}\left(\mathbf{h}\right)}}\phi (Y_{i_1},\dots ,Y_{i_m})\prod _{j=1}^{m}K\left({\displaystyle \frac{d_{{\mathit{\theta }}_j}(X_{i_j},t_j)}{h_j}}\right)\right)\right.\hfill \\ \left.\phantom{\mathbb{E}\left({\displaystyle \frac{(n-m)!}{n!}}\sum _{i\in I(m,n)}{\displaystyle \frac{1}{{\varphi }_{\mathbf{t},\mathit{\theta }}\left(\mathbf{h}\right)}}\phi (Y_1,\dots ,Y_m)\prod _{i=1}^{m}K\left({\displaystyle \frac{d_{{\mathit{\theta }}_i}(X_i,t_i)}{h_i}}\right)\right)}-r^{\left(m\right)}\left(\phi ,\mathbf{t},\mathit{\theta }\right)\right|\hfill \\ \le \left|{\kappa }_1^{-m}{C}_1^{-1}{\displaystyle \frac{1}{{\varphi }_{\mathbf{t},\mathit{\theta }}\left(\mathbf{h}\right)}}\mathbb{E}\left(\phi (Y_1,\dots ,Y_m)\prod _{i=1}^{m}K\left({\displaystyle \frac{d_{{\mathit{\theta }}_i}(X_i,t_i)}{h_i}}\right)\right)-r^{\left(m\right)}\left(\phi ,\mathbf{t},\mathit{\theta }\right)\right|\hfill \\ \le {\displaystyle \frac{1}{{\varphi }_{\mathbf{t},\mathit{\theta }}\left(\mathbf{h}\right){\kappa }_1^m{C}_1}}\mathbb{E}\left(\prod _{i=1}^{m}K\left({\displaystyle \frac{d_{{\mathit{\theta }}_i}(X_i,t_i)}{h_i}}\right)\left|r^{\left(m\right)}\left(\phi ,\mathbf{X}\right)-r^{\left(m\right)}\left(\phi ,\mathbf{t},\mathit{\theta }\right)\right|\right).\hfill \end{array}$$

Considering Hypotheses (H.1), (C.2.1), (C.3.1), and (C.6.), we have ${\displaystyle \forall \mathbf{h}=(h_1,\dots ,h_m)\in \prod _{j=1}^m(h_{n,j},b_j)}$ and $\forall \mathbf{t}\in S_{\mathcal{X}}^m:$

$$\begin{array}{c}{\displaystyle \left|{\kappa }_1^{-m}{C}_1^{-1}\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h})\right)-r^{\left(m\right)}\left(\phi ,\mathbf{t},\mathit{\theta }\right)\right|}\hfill \\ \le {\displaystyle \frac{C_3}{{\varphi }_{\mathbf{t},\mathit{\theta }}\left(\mathbf{h}\right){\kappa }_1^m{C}_1}}\mathbb{E}\left(\prod _{i=1}^{m}K\left({\displaystyle \frac{d_{{\mathit{\theta }}_i}(X_i,t_i)}{h_i}}\right)d_{{\mathcal{X}}^m}^{\gamma }\left(\mathbf{X},\mathbf{t}\right)\right)\hfill \\ \le {\displaystyle \frac{C_3}{m}}{\left(d_{{\mathit{\theta }}_1}\left(X_1,t_1\right)+\cdots +d_{{\mathit{\theta }}_m}\left(X_m,t_m\right)\right)}^{\gamma }\hfill \\ \le {\displaystyle \frac{C_3}{m}}{\left(h_1+\cdots +h_m\right)}^{\gamma }\hfill \\ ={\displaystyle \frac{C_3}{m}}{(b_1+\dots +b_m)}^{\gamma }\le C_3^{\prime }{\tilde{b}}_0^{\gamma },\hfill \end{array}$$

where

$${\displaystyle {\tilde{b}}_0:=\underset{1\le j\le m}{max}{b}_j.}$$

With this, the proof of the theorem is concluded. □

8.2.4. Proof of Corollary 6

In this section, we establish Corollary 6 by applying Lemma A1. Following the same reasoning as in the case of functional regression, we adopt the following notation: $S_{\Omega }=S_{\mathcal{H}}$, ${\mathbf{A}}_i={\mathbf{X}}_i$, $\phi \left({\mathbf{B}}_i\right)=\phi \left({\mathbf{Y}}_i\right)$,

$$\begin{array}{ccc}\hfill \prod _{i=1}^{m}G(H_i,(⟨t_i,{\mathit{\theta }}_i⟩,A_i))& =& \prod _{i=1}^{m}K\left(H_i^{-1}{d}_{{\mathit{\theta }}_i}(t_i,A_i)\right),\hfill \\ \hfill D_{n,k}\left(⟨t_j,{\mathit{\theta }}_j⟩\right)& =& H_{n,k}(t_j,{\mathit{\theta }}_j),\forall \mathbf{t}=(t_1,\dots ,t_m)\in S_{\mathcal{H}}^m,\mathrm{and}j=1,\dots ,m,\hfill \\ \hfill {\mathfrak{M}}_n^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}(\mathbf{t},\mathit{\theta }))& =& {\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}(\mathbf{t},\mathit{\theta })),\hfill \\ \hfill \mathfrak{M}(\phi ,\mathbf{t},\mathit{\theta })& =& r^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta }).\hfill \end{array}$$

We choose $D_{n,k}^{-}({\mathit{\xi }}_n,⟨t,\mathit{\theta }⟩)$ and $D_{n,k}^{+}({\mathit{\xi }}_n,⟨t,\mathit{\theta }⟩)$ such that for all $j=1,\dots ,m,$

$${\varphi }_{t_j}\left(D_{n,k}^{-}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩)\right)={\displaystyle \frac{\sqrt{{\mathit{\xi }}_{n,j}}k}{n}},$$

$${\varphi }_{t_j}\left(D_{n,k}^{+}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩)\right)={\displaystyle \frac{k}{n\sqrt{{\mathit{\xi }}_{n,j}}}},$$

where ${\mathit{\xi }}_{n,j}$ are increasing sequences that belong to $(0,1)$ and

$${\mathit{\xi }}_n=\prod _{j=1}^m{\mathit{\xi }}_{n,j}.$$

We denote ${\mathbf{h}}^{-}(\mathbf{t},\mathit{\theta })=\left(h_1^{-}(t_1,{\mathit{\theta }}_1),\dots ,h_m^{-}(t_m,{\mathit{\theta }}_m)\right)\mathrm{and}{\mathbf{h}}^{+}(\mathbf{t},\mathit{\theta })=\left(h_1^{+}(t_1,{\mathit{\theta }}_1),\dots ,h_m^{+}(t_m,{\mathit{\theta }}_m)\right),$ where

$$h_j^{-}(t_j,{\mathit{\theta }}_j)=D_{n,k}^{-}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩)\mathrm{and}{h}^{+}(t_j,{\mathit{\theta }}_j)=D_{n,k}^{+}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩),\mathrm{for} \mathrm{all}j=1,\dots ,m.$$

We can easily see that, for all $j=1,\dots ,m,$

$$\begin{array}{ccc}\hfill {\varphi }_{t_j}^{-1}\left({\displaystyle \frac{\sqrt{{\mathit{\xi }}_{n,j}}{k}_{1,n}}{n}}\right)\le h^{-}(t_j,{\mathit{\theta }}_j)& \le & {\varphi }_{t_j}^{-1}\left({\displaystyle \frac{\sqrt{{\mathit{\xi }}_{n,j}}{k}_{2,n}}{n}}\right),\hfill \end{array}$$

(97)

$$\begin{array}{ccc}\hfill {\varphi }_{t_j}^{-1}\left({\displaystyle \frac{k_{1,n}}{n\sqrt{{\mathit{\xi }}_{n,j}}}}\right)\le h^{+}(t_j,{\mathit{\theta }}_j)& \le & {\varphi }_{t_j}^{-1}\left({\displaystyle \frac{k_{2,n}}{n\sqrt{{\mathit{\xi }}_{n,j}}}}\right).\hfill \end{array}$$

(98)

Using Condition $\left(8\right)$ one deetrmines that, for all $j=1,\dots ,m,$ there exist constants $0<{\mu }_j\le {\nu }_j<\infty$ such that

$${\mu }_j{\varphi }^{-1}\left({\displaystyle \frac{{\rho }_{n,j}{k}_{1,n}}{n}}\right)\le {\varphi }_{t_j,{\mathit{\theta }}_j}^{-1}\left({\displaystyle \frac{{\rho }_{n,j}{k}_{1,n}}{n}}\right)\mathrm{and}{\varphi }_{t_j,{\mathit{\theta }}_j}^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n,j}n}}\right)\le {\nu }_j{\varphi }^{-1}\left({\displaystyle \frac{k_{2,n}}{{\rho }_{n,j}n}}\right).$$

We put ${\rho }_{n,j}=\sqrt{{\mathit{\xi }}_{n,j}}$, ${\displaystyle h_{n,j}={\mu }_j{\varphi }^{-1}\left(\frac{{\rho }_{n,j}{k}_{1,n}}{n}\right)}$ and ${\displaystyle h_{n,j}^{\prime }={\nu }_j{\varphi }^{-1}\left(\frac{k_{2,n}}{{\rho }_{n,j}n}\right);}$ thus, ${\mathbf{h}}^{-}\left(\mathbf{t}\right)$ and ${\mathbf{h}}^{+}\left(\mathbf{t}\right)$ belong to the interval

$${\displaystyle {\mathcal{H}}_n^{\left(m\right)}:=\prod _{j=1}^m\left(h_{n,j},h_{n,j}^{\prime }\right).}$$

We denote ${\displaystyle {\tilde{h}}_n=\underset{1\le j\le m}{min}{h}_{n,j}}$ and ${\displaystyle {\widetilde{h^{\prime }}}_n=\underset{1\le j\le m}{max}{h}_{n,j}^{\prime }}$; therefore,

$$h_j\left(t_j\right)\in \left({\tilde{h}}_n,{\widetilde{h^{\prime }}}_n\right),\forall j=1,\dots ,m.$$

We also note, for all $\mathbf{b}=(b_1,\dots ,b_m)\in {(0,1)}^m,$

$${\displaystyle {\mathcal{H}}_0^{\left(m\right)}:=\prod _{j=1}^m\left(h_{n,j},b_j\right),}$$

and

$${\displaystyle {\tilde{b}}_0:=\underset{1\le j\le m}{max}{b}_j.}$$

Ultimately, we can choose constants $0<{\mu }^{*}<{\nu }^{*}<\infty$ and a sequence $\left\{{\rho }_n^{*}\right\}\in (0,1)$ while satisfying Condition (C.8) such that ${\displaystyle {\tilde{h}}_n={\mu }^{*}{\varphi }^{-1}\left(\frac{{\rho }_n^{*}{k}_{1,n}}{n}\right)}$, ${\displaystyle {\widetilde{h^{\prime }}}_n={\nu }^{*}{\varphi }^{-1}\left(\frac{k_{2,n}}{{\rho }_n^{*}n}\right)}$, and

$$\begin{array}{c}\hfill {\mathcal{U}}_n={\varphi }^{-1}{\left({\displaystyle \frac{k_{2,n}}{{\rho }_n^{*}n}}\right)}^{\gamma }+\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left({\mu }^{*}{\varphi }^{-1}\left(\frac{{\rho }_n^{*}{k}_{1,n}}{n}\right)\right)}}}}.\end{array}$$

It is evident that $\left(L_0\right)$ is fulfilled due to Condition (C.3.1), and from (97) and (98), we can readily verify that the construction of ${\mathbf{h}}^{-}(\mathbf{t},\mathit{\theta })$ and ${\mathbf{h}}^{+}(\mathbf{t},\mathit{\theta })$ satisfies Condition $\left(L_1\right)$. □

8.3. Proofs of Weak Convergence Results

8.3.1. Preliminaries of the Proofs

As previously noted, a direct approach proves ineffective when dealing with random bandwidths. Consequently, we often rely on general lemmas (see, for instance, [51]) to facilitate the application of results established for non-random bandwidths. In this section, we introduce key results from [111,112], which were obtained for a positive bandwidth $h_K$. These results are instrumental in the proofs that follow. We denote the bias term and the centered variate as the following quantities:

$$\begin{array}{ccc}\hfill B_n(⟨t,\mathit{\theta }⟩;h_K)& :=& {\displaystyle \frac{\mathbb{E}\left({\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K)\right)-r^{\left(1\right)}(\phi ,t)\mathbb{E}\left({\widehat{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩;h_K)\right)}{\mathbb{E}\left({\widehat{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩;h_K)\right)}},\hfill \\ \hfill Q_n(⟨t,\mathit{\theta }⟩;h_K)& :=& \left({\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K)-\mathbb{E}\left({\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K)\right)\right)\hfill \\ & & -r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\left({\widehat{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩;h_K)-\mathbb{E}\left({\widehat{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩;h_K)\right)\right).\hfill \end{array}$$

(99)

Decomposition (99) plays a pivotal role in our proof. Following the methodology employed in [111], we show that convergence in the quadratic mean to one is achieved and that the bias satisfies

$$B_n(⟨t,\mathit{\theta }⟩;h_K)=o\left(1\right)asn\to \infty .$$

Proof of Theorem 11

By applying the Cramér–Wold device, it is sufficient to demonstrate the convergence of the one-dimensional distribution to establish Theorem 11. In fact, leveraging the linearity of ${\nu }_n(\psi ;k_K\mid t)$ is enough to show that

$${\nu }_n(\Phi ;H_{n,k}(t,\mathit{\theta })\mid ⟨t,\mathit{\theta }⟩)\to N(0,{\sigma }^2(\Phi ,t,\mathit{\theta }))$$

for all $\Phi$ of the form

$$\Phi =\sum _{p=1}^L{c}_p{\mathit{\psi }}_p,c_1,\dots ,c_L\in \mathbb{R},{\mathit{\psi }}_1,\dots ,{\mathit{\psi }}_L\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}.$$

Hence, we solely illustrate the convergence in a single dimension. Recall that we are dealing with

$$\begin{array}{ccc}\hfill {\nu }_n(\psi ;H_{n,k}(t,\mathit{\theta })\mid ⟨t,\mathit{\theta }⟩)& =& \sqrt{k}({\widehat{r}}_n^{\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩))\hfill \\ & & =\sqrt{k}\left({\displaystyle \frac{{\sum }_{i=1}^n\phi \left(Y_i\right)K\left(d_{\mathit{\theta }}(X_i,t)/H_{n,k}(t,\mathit{\theta })\right)}{{\sum }_{i=1}^{n}K\left(d_{\mathit{\theta }}(X_i,t)/H_{n,k}(t,\mathit{\theta })\right)}}-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right).\hfill \end{array}$$

(100)

We set

$$\begin{array}{ccc}\hfill {\phi }_M\left(Y\right)& :=& \phi \left(Y\right){\mathbb{1}}_{\left\{F\left(Y\right)<M\right\}}.\hfill \end{array}$$

To obtain the desired result, we write

$$\begin{array}{ccc}\hfill {\widehat{r}}_n^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)& =& {\displaystyle \frac{1}{{\widehat{r}}_{n,1}^{\left(1\right)}(1,t;H_{n,k}(t,\mathit{\theta }))}}\left[I_1(⟨t,\mathit{\theta }⟩;H_{n,k})+I_2(t;H_{n,k})\right],\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill I_1(⟨t,\mathit{\theta }⟩;H_{n,k})& =& \underset{S_1(⟨t,\mathit{\theta }⟩;H_{n,k})}{\underbrace{\left({\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-{\widehat{r}}_{n,2}^{\left(1\right)}({\phi }_M,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))\right)}}-\mathbb{E}\left(S_1(⟨t,\mathit{\theta }⟩;H_{n,k})\right)\hfill \\ & & +\underset{S_2(t;H_{n,k})}{\underbrace{\mathbb{E}\left({\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,t;H_{n,k}(⟨t,\mathit{\theta }⟩,\mathit{\theta }))\right)-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)}},\hfill \end{array}$$

(101)

and

$$\begin{array}{ccc}\hfill I_2(⟨t,\mathit{\theta }⟩;H_{n,k})& =& \left({\widehat{r}}_{n,2}^{\left(1\right)}({\phi }_M,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-\mathbb{E}\left({\widehat{r}}_{n,2}^{\left(1\right)}({\phi }_M,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))\right)\right.\hfill \\ & & -\left.r^{\left(1\right)}(\phi ,t)\left({\widehat{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-1\right)\right).\hfill \end{array}$$

(102)

To derive the desired results, we adopt the strategy of [213].

Proof of Theorem 12

In this section, we utilize the same methodology as in [112] and previously in [109]. This approach involves applying the blocking method, which partitions a strictly stationary sequence $(X_1,\dots ,X_n)$ into $2{\upsilon }_n$ equal-sized blocks, each of length $n-2{\upsilon }_n{a}_n$. It is important to refer to the notation outlined in the proof of Lemma A2. The primary objective is to establish the asymptotic equicontinuity of the conditional empirical process

$$\left\{{\nu }_n(\psi \mid ⟨t,\mathit{\theta }⟩)=\sqrt{k}\left({\widehat{r}}_n^{\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right),\psi \in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}},\phi \in \mathcal{F}\right\}.$$

Let us introduce, for any $\phi K\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}$ and $t\in \mathcal{X},$

$$\begin{array}{ccc}\hfill {\mathcal{W}}_n(⟨t,\mathit{\theta }⟩,\phi ;H_{n,k}(t,\mathit{\theta }))& :=& \sum _{i=1}^n\phi \left(Y_i\right)K\left({\displaystyle \frac{d_{\mathit{\theta }}\left(X_i,t\right)}{H_{n,k}(t,\mathit{\theta })}}\right)-n\mathbb{E}\left\{\phi \left(Y_1\right)K\left({\displaystyle \frac{d_{\mathit{\theta }}\left(X_1,t\right)}{H_{n,k}(t,\mathit{\theta })}}\right)\right\},\hfill \end{array}$$

(103)

$$\begin{array}{ccc}\hfill {\nu }_n(\phi \mid ⟨t,\mathit{\theta }⟩)& =& \sqrt{k}\left({\widehat{r}}_n^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right)\hfill \\ & :=& \sqrt{k}\left({\widehat{r}}_n^{\left(1\right)}(\psi ,⟨t,\mathit{\theta }⟩,H_{n,k}(t,\mathit{\theta }))-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right).\hfill \end{array}$$

(104)

Then, we have

$$\begin{array}{ccc}\hfill {\nu }_n(\psi \mid ⟨t,\mathit{\theta }⟩)& =& \sqrt{k}({\widehat{r}}_n^{\left(1\right)}(\psi ,t,H_{n,k}(t,\mathit{\theta }))-r^{\left(1\right)}(\phi ,t))\hfill \\ & =& \sqrt{k}\left({\displaystyle \frac{{\displaystyle \sum _{i=1}^n\phi \left(Y_i\right)K\left(d_{\mathit{\theta }}(X_i,t)/H_{n,k}(t,\mathit{\theta })\right)}}{{\displaystyle \sum _{i=1}^{n}K\left(d_{\mathit{\theta }}(X_i,t)/H_{n,k}(t,\mathit{\theta })\right)}}}-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right)\hfill \\ & =& {\displaystyle \frac{{\displaystyle 1}}{{\displaystyle {\breve{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))}}}{\displaystyle \frac{{\displaystyle 1}}{{\displaystyle \sqrt{k}}}}{\mathcal{W}}_n(⟨t,\mathit{\theta }⟩,\phi ;H_{n,k}(t,\mathit{\theta }))\hfill \\ & & -{\displaystyle \frac{{\displaystyle \mathbb{E}\left({\breve{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))\right)}}{{\displaystyle {\breve{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\mathbb{E}\left({\breve{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta }))\right)}}}{\displaystyle \frac{{\displaystyle 1}}{{\displaystyle \sqrt{k}}}}{\mathcal{W}}_n(⟨t,\mathit{\theta }⟩,1;H_{n,k}(t,\mathit{\theta }))\hfill \\ & & -\sqrt{k}{B}_n(⟨t,\mathit{\theta }⟩;H_{n,k}(t,\mathit{\theta })),\hfill \end{array}$$

(105)

where for $h_K>0$, we have

$$\begin{array}{cc}\hfill {\breve{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,h_K)& :={\displaystyle \frac{1}{n\varphi \left(h_K\left(t\right)\right)}}\sum _{i=1}^n\phi \left(Y_i\right){\Delta }_i(t,h_K),\hfill \\ \hfill {\breve{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩,h_K)& :={\displaystyle \frac{1}{n\varphi \left(h_K\left(t\right)\right)}}\sum _{i=1}^n{\Delta }_i(t,\mathit{\theta },h_K).\hfill \end{array}$$

We investigate the asymptotic equicontinuity of each of the preceding terms. For a given class of functions $\mathcal{G}$, we let ${\alpha }_n(·)$ denote an empirical process based on $(X_1,Y_1),\dots ,(X_n,Y_n)$, indexed by $\mathcal{G}$:

$${\alpha }_n\left(g\right)={\displaystyle \frac{1}{\sqrt{n}}}\sum _{i=1}^n\left\{g(X_i,Y_i)-\mathbb{E}\left(g(X_i,Y_i)\right)\right\},\mathrm{with}{∥{\alpha }_n\left(g\right)∥}_{\mathcal{G}}=\underset{g\in \mathcal{G}}{sup}\left|{\alpha }_n\left(g\right)\right|,$$

and for a measurable function $\phi (·)$ and $t\in \mathcal{H}$, $\mathit{\theta }\in \mathbf{\Theta }$, we set

$${\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}(u,v,h_K)=\phi \left(v\right)K\left({\displaystyle \frac{d_{\mathit{\theta }}\left(u,t\right)}{h_K}}\right),\mathrm{for}u,v\in \mathcal{X}.$$

That implies

$${\displaystyle \frac{1}{\sqrt{n\varphi \left(h_K\right)}}}{\mathcal{W}}_n(⟨t,\mathit{\theta }⟩,\phi ,h_K)={\displaystyle \frac{1}{\sqrt{\varphi \left(h_K\right)}}}{\alpha }_n\left({\mathit{\eta }}_{n,t,\phi ,K}\right).$$

Again, keeping in mind that ${\mathbb{1}}_{\left\{D_n^{-}\le H_{n,k}(t,\mathit{\theta })\le D_n^{+}\right\}}\stackrel{a.co}{⟶}1$ when ${\displaystyle \frac{k}{n}\to 0}$, we establish the asymptotic equi-continuity of

$$\left\{\sqrt{{\displaystyle \frac{n}{k}}}{\alpha }_n\left({\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}\right):{\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right\},$$

which means, for every $\epsilon >0$, that

$$\underset{b\to 0}{lim}\underset{n\to \infty }{lim\; sup}\mathbb{P}\left\{\sqrt{{\displaystyle \frac{n}{k}}}{∥{\alpha }_n\left({\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}\right)∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{(b,\parallel ·{\parallel }_p)}}>\epsilon \right\}=0,$$

where

$$\begin{array}{ccc}\hfill {\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{(b,\parallel ·{\parallel }_p)}& =& \left\{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}:{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p<b,{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1},{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right\}.\hfill \end{array}$$

This approach entails working with the independent block sequence ${\left\{{\mathit{\xi }}_j=({\zeta }_j,{\varsigma }_j)\right\}}_{j=1}^{\infty }$ in place of the dependent sequence, facilitated by the results in [119]. Accordingly, we obtain

$$\begin{array}{cc}\hfill & \mathbb{P}\left\{{∥k^{-1/2}\sum _{j=1}^n\left(\phi \left(Y_j\right)K\left({\displaystyle \frac{d_{\mathit{\theta }}\left(X_j,t\right)}{H_{n,k}(t,\mathit{\theta })}}\right)-\mathbb{P}\left({\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}\left(H_{n,k}(t,\mathit{\theta })\right)\right)\right)∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{(b,\parallel ·{\parallel }_p)}}>\delta \right\}\hfill \\ \hfill & \le 2\mathbb{P}\left\{{∥k^{-1/2}\sum _{j=1}^{{\upsilon }_n}\sum _{i\in H_j}\left(\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{d_{\mathit{\theta }}\left({\varsigma }_i,t\right)}{H_{n,k}(t,\varsigma )}}\right)-\mathbb{P}\left({\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}\left(H_{n,k}(t,\varsigma )\right)\right)\right)∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{(b,\parallel ·{\parallel }_p)}}>{\delta }^{\prime }\right\}\hfill \\ \hfill & +2({\upsilon }_n-1){\beta }_{a_n},\hfill \end{array}$$

(106)

where $H_{n,k}(t,\varsigma )$ is defined by

$$H_{n,k}(t,\varsigma )=min\left\{h\in {\mathbb{R}}^{+}:\sum _{i=1}^n{\mathbb{1}}_{B_{\mathit{\theta }}(t,h)}\left({\varsigma }_i\right)=k\right\},$$

(107)

We choose

$$a_n=\left[{(logn)}^{-1}{\left(n^{p-2}{\varphi }^p\left(h_K\right)\right)}^{1/2(p-1)}\right]\mathrm{and}{\upsilon }_n=\left[{\displaystyle \frac{n}{2a_n}}\right]-1.$$

Note that in our context, $a_n$ is given by the following expression:

$$a_n={(logn)}^{-1}{\left(n^{-2}{k}^p\right)}^{1/2(p-1)}.$$

Utilizing Condition (C.5.1), we have $({\upsilon }_n-1){\beta }_{a_n}\to 0$ as $n\to \infty$. Consequently, we focus on the right-hand side term of (106). We begin by assuming that the blocks are independent. Symmetrization is performed using a sequence ${\left\{{ϵ}_j\right\}}_{j\in {\mathbb{N}}^{*}}$ of i.i.d. Rademacher variables, i.e., random variables satisfying $\mathbb{P}({ϵ}_j=1)=\mathbb{P}({ϵ}_j=-1)=1/2$. It is important to note that the sequence ${\left\{{ϵ}_j\right\}}_{j\in {\mathbb{N}}^{*}}$ is independent of the sequence ${\left\{{\mathit{\xi }}_i=({\varsigma }_i,{\zeta }_i)\right\}}_{i\in {\mathbb{N}}^{*}}$. Thus, we need to establish the following for all $\delta >0$:

$$\underset{b\to 0}{lim}\underset{n\to \infty }{lim\; sup}\mathbb{P}\left\{{∥k^{-1/2}\sum _{j=1}^{{\upsilon }_n}{ϵ}_j\sum _{i\in H_j}\left(\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{d\left({\varsigma }_i,t\right)}{H_{n,k}(t,\varsigma )}}\right)\right)∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{(b,\parallel ·{\parallel }_p)}}>\delta \right\}=0.$$

Again, using the fact that that ${\mathbb{1}}_{\left\{D_n^{-}\le H_{n,k}(t,\varsigma )\le D_n^{+}\right\}}\stackrel{a.co}{⟶}1$ when ${\displaystyle \frac{k}{n}\to 0}$ and [155], it suffices to show that

$$\underset{b\to 0}{lim}\underset{n\to \infty }{lim\; sup}\mathbb{P}\left\{{∥k^{-1/2}\sum _{j=1}^{{\upsilon }_n}{ϵ}_j\sum _{i\in H_j}\left(\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{d_{\mathit{\theta }}\left({\varsigma }_i,t\right)}{D_n^{-}}}\right)\right)∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{(b,\parallel ·{\parallel }_p)}}>\delta \right\}=0.$$

Since the $p^{th}$-conditional moment satisfies (C.4.2), we can truncate and obtain, for each $\lambda >0$, as $n\to \infty$,

$$\begin{array}{cc}\hfill {\displaystyle k^{-1/2}}& \sum _{j=1}^n\mathbb{E}\left({\kappa }_{2}F\left({\zeta }_i\right){\mathbb{1}}_{\{F\left({\zeta }_i\right)\ge \lambda {\left(M_n\right)}^{1/2(p-1)}\}}\right)\hfill \\ & =k^{-1/2}{\int }_0^{\infty }\mathbb{P}\left({\kappa }_{2}F{\mathbb{1}}_{\{F\ge \lambda {\left(M_n\right)}^{1/2(p-1)}\}}\ge t\right)dt\hfill \\ & =k^{-1/2}{\int }_0^{\lambda {\left(M_n\right)}^{1/2(p-1)}}\mathbb{P}\left(F\ge \lambda {\left(M_n\right)}^{1/2(p-1)}\right)dt\hfill \\ & +k^{-1/2}{\int }_{\lambda {\left(M_n\right)}^{1/2(p-1)}}^{\infty }\mathbb{P}\left(F\ge t\right)dt\hfill \end{array}$$

(108)

$$\begin{array}{ccc}& \underset{n\to \infty }{⟶}& 0.\hfill \end{array}$$

(109)

Hence,

$$\exists {\lambda }_n\underset{n\to \infty }{\to 0}:k^{-1/2}\mathbb{E}\left({\kappa }_{2}F{\mathbb{1}}_{\{F\ge {\lambda }_n{\left(M_n\right)}^{1/2(p-1)}\}}\right)\underset{n\to \infty }{⟶}0.$$

Then, it suffices to show

$$\begin{array}{c}\hfill \underset{b\to 0}{lim}\underset{n\to \infty }{lim\; sup}\mathbb{P}\left\{{∥k^{-1/2}\sum _{j=1}^{{\upsilon }_n}{ϵ}_j\sum _{i\in H_j}\left(\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{d\left({\varsigma }_i,t\right)}{D_n^{-}}}\right)\right){\mathbb{1}}_{\{{\kappa }_{2}F\left({\zeta }_i\right)\le {\lambda }_n{M}_n^{1/2(p-1)}\}}∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{(b,\parallel ·{\parallel }_p)}}>\delta \right\}=0.\end{array}$$

We have the following:

$${\nu }_n^{\left(2\right)}\left({\mathit{\eta }}_{n,t,\phi ,K}\right)=k^{-1/2}\sum _{j=1}^{{\upsilon }_n}{ϵ}_j\sum _{i\in H_j}\left(\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{d\left({\varsigma }_i,t\right)}{D_n^{-}}}\right)\right){\mathbb{1}}_{\{{\kappa }_{2}F\left({\zeta }_i\right)\le {\lambda }_n{M}_n^{1/2(p-1)}\}}.$$

This is achieved by using the chaining method. In [109], $b_q=b{2}^{-q}$, $q=0,\dots ,q_n$ was provided, where $q_n$

$$2^{-1}{\lambda }_n{(log\left(n\right))}^{-1}\le b_{q_n}^2\le {\lambda }_n{(log\left(n\right))}^{-1},$$

(110)

and we let class ${\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_q$ of measurable functions of $\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}$

$$\#{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_q=N_q:=N(b_q,\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}{,\parallel ·\parallel }_p)\underset{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\underset{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_q}{min}{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p\le b_q.$$

There is a map ${\pi }_q:\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}⟶{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_q$ that takes each ${\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}$ to its closest function in ${\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_q$ such that

$${∥{\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}-{\pi }_q\left({\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}\right)∥}_p\le b_q.$$

We apply the chaining method

$$\begin{array}{c}\hfill {\displaystyle \underset{\underset{{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p\le b}{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1},{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}}{sup}{\nu }_n^{\left(2\right)}({\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2})}\\ \le \underset{\underset{{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p\le b_{q_n}}{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1},{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}}{sup}{\nu }_n^{\left(2\right)}({\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2})\hfill \\ +2\sum _{q=1}^{q_n}\underset{\underset{{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p\le 3b_q}{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1},{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}\in {\left(\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right)}_{q-1}}}{sup}{\nu }_n^{\left(2\right)}({\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2})\hfill \\ +\underset{\underset{{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p\le 2b}{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1},{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}\in {\left(\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right)}_0}}{sup}{\nu }_n^{\left(2\right)}({\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}).\hfill \end{array}$$

(111)

We let ${\delta }_q$ be in such a way that

$${\delta }_q={\left(b_q\right)}^{1/2}\vee \left(3b_q{(8+c_{p,\beta }^2)}^{1/2}{(log{N}_q)}^{1/2}\right).$$

(112)

We let r be chosen so small in such a way that

$$2\sum _{q=1}^{+\infty }{\delta }_q\le \delta .$$

Therefore, from (111), we readily infer that

$$\begin{array}{cc}\hfill {\displaystyle \mathbb{P}}& \left\{\underset{\underset{{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p\le b}{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1},{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}}{sup}{\nu }_n^{\left(2\right)}({\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2})\ge 3\delta \right\}\hfill \\ & \le \mathbb{P}\left\{\underset{\underset{{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p\le b_{q_n}}{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1},{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}}{sup}{\nu }_n^{\left(2\right)}({\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2})\ge \delta \right\}\hfill \\ & +2\sum _{q=1}^{q_n}\mathbb{P}\left\{\underset{\underset{{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p\le 3b_q}{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1},{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}\in {\left(\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right)}_{q-1}}}{sup}{\nu }_n^{\left(2\right)}({\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2})\ge {\delta }_q\right\}\hfill \\ & +\mathbb{P}\left\{\underset{\underset{{∥{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}∥}_p\le 2b}{{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1},{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}\in {\left(\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}\right)}_0}}{sup}{\nu }_n^{\left(2\right)}({\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2})\ge \delta \right\}=:\mathbb{A}+\mathbb{B}+\mathbb{C}.\hfill \end{array}$$

By the fact that the terms composing ${\nu }_n^{\left(2\right)}\left({\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}\right)$ are bounded by $a_n{\lambda }_n{M}_n^{1/2(p-1)}$, and by applying the Bernstein inequality, we obtain

$$\begin{array}{c}\hfill \mathbb{B}\le 2\sum _{q=1}^{q_n}exp\left(2log{N}_q-{\displaystyle \frac{{\delta }_q^{2}k}{n{b}_q^2{c}_{p,\beta }^2+(4/3){\delta }_q{a}_n{\lambda }_n{n}^{p/2(p-1)}\varphi {\left(h_K\right)}^{(p-2)/2(p-1)}}}\right).\end{array}$$

By using (110), we have

$${\delta }_q{a}_n{\lambda }_n{n}^{p/2(p-1)}\varphi {\left(h_K\right)}^{(p-2)/2(p-1)}=(4/3){\delta }_q{\lambda }_{n}k{(log\left(n\right))}^{-1}\le (8/3)n{b}_q^2{\delta }_q\le 8n{b}_q^2,$$

that means

$$\begin{array}{ccc}\hfill \mathbb{B}& \le & 2\sum _{q=1}^{q_n}exp\left(2log{N}_q-{\displaystyle \frac{{\delta }_q^2}{(8+c_{p,\beta }^2)b_q^2}}\right)\le 2\sum _{q=1}^{q_n}exp\left(-{\displaystyle \frac{{\delta }_q^2}{2(8+c_{p,\beta }^2)b_q^2}}\right)\hfill \\ & \le & 2\sum _{q=1}^{\infty }exp\left(-{\displaystyle \frac{2^q}{2(8+c_{p,\beta }^2)b}}\right)⟶0asb\to 0.\hfill \end{array}$$

(113)

In view of (112), we assume that $\delta <3$. In a similar way, we have

$$\mathbb{C}\le 2exp\left(2log{N}_0-{\displaystyle \frac{{\delta }^2}{(8+c_{p,\beta }^2)b^2}}\right)⟶0asb\to 0.$$

Finally, by (110), it suffices to prove, for each $\delta >0$,

$$\underset{n\to \infty }{lim}\mathbb{P}\left\{\parallel {\nu }_n^{\left(2\right)}\left({\mathit{\eta }}_{n,t,\mathit{\theta },\phi ,K}\right){\parallel }_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{({\lambda }_n^{1/2}{(log\left(n\right))}^{-1/2},\parallel ·{\parallel }_p)}}\ge \delta \right\}=0.$$

Making use of the square root trick (Lemma 5.2 [214])—see also [215] in a similar way to that in [109]—we obtain

$$\begin{array}{c}\hfill {\displaystyle \mathbb{P}\left\{∥k^{-1/2}\sum _{j=1}^{{\upsilon }_n}{ϵ}_j\sum _{i\in H_j}\left(\phi \left({\zeta }_i\right)K\left(\frac{d_{\mathit{\theta }}\left({\varsigma }_i,t\right)}{D_n^{-}}\right)\right)\right.}\\ & \times {\mathbb{1}}_{\{{\kappa }_{2}F\left({\zeta }_i\right)\le {\lambda }_n{(n/\varphi \left(h_K\right))}^{1/2(p-1)}\}}{\parallel }_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{({\lambda }_n^{1/2}{(log\left(n\right))}^{-1/2},\parallel ·{\parallel }_p)}}\ge 2\delta \}\hfill \\ \le & \mathbb{P}\left\{∥k^{-1/2}\sum _{j=1}^{{\upsilon }_n}{ϵ}_j\sum _{i\in H_j}\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{d_{\mathit{\theta }}\left({\varsigma }_i,t\right)}{D_n^{-}}}\right)\right.\hfill \\ & \times {\mathbb{1}}_{\{{\kappa }_{2}F\left({\zeta }_i\right)\le {\lambda }_n{(n/\varphi \left(h_K\right))}^{1/2(p-1)}\}}{\parallel }_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{({\lambda }_n^{1/2}{(log\left(n\right))}^{-1/2},\parallel ·{\parallel }_p)}}\ge 2\delta ,\hfill \\ & {∥k^{-1}\sum _{j=1}^{{\upsilon }_n}{\left(\sum _{i\in H_j}\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{d_{\mathit{\theta }}\left({\varsigma }_i,t\right)}{D_n^{-}}}\right){\mathbb{1}}_{\{{\kappa }_{2}F\left({\zeta }_i\right)\le {\lambda }_n{(n/\varphi \left(h_K\right))}^{1/2(p-1)}\}}\right)}^2∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{({\lambda }_n^{1/2}{(log\left(n\right))}^{-1/2},\parallel ·{\parallel }_p)}}\hfill \\ & \le \left.\phantom{{∥k^{-1}\sum _{j=1}^{{\upsilon }_n}{\left(\sum _{i\in H_j}\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{{\varsigma }_i-\mathbf{t}}{D_n^{-}}}\right){\mathbb{1}}_{\{{\kappa }_{2}F\left({\zeta }_i\right)\le {\lambda }_n{\left(nh\right)}^{1/2(p-1)}\}}\right)}^2∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{({\lambda }_n^{1/2}{(log\left(n\right))}^{-1/2},\parallel ·{\parallel }_p)}}}64{\lambda }_n{c}_{p,\beta }^2{(log\left(n\right))}^{-1}\right\}\hfill \\ & +\mathbb{P}\left\{{∥k^{-1}\sum _{j=1}^{{\upsilon }_n}{\left(\sum _{i\in H_j}\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{d_{\mathit{\theta }}\left({\varsigma }_i,t\right)}{D_n^{-}}}\right){\mathbb{1}}_{\{{\kappa }_{2}F\left({\zeta }_i\right)\le {\lambda }_n{(n/\varphi \left(h_K\right))}^{1/2(p-1)}\}}\right)}^2∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{({\lambda }_n^{1/2}{(log\left(n\right))}^{-1/2},\parallel ·{\parallel }_p)}}\right.\hfill \\ & >\left.\phantom{{∥k^{-1}\sum _{j=1}^{{\upsilon }_n}{\left(\sum _{i\in H_j}\phi \left({\zeta }_i\right)K\left({\displaystyle \frac{{\varsigma }_i-t}{D_n^{-}}}\right){\mathbb{1}}_{\{{\kappa }_{2}F\left({\zeta }_i\right)\le {\lambda }_n{(n/h)}^{1/2(p-1)}\}}\right)}^2∥}_{{\mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}_{({\lambda }_n^{1/2}{(log\left(nh\right))}^{-1/2},\parallel ·{\parallel }_p)}}}64{\lambda }_n{c}_{p,\beta }^2{(log\left(n\right))}^{-1}\right\}\hfill \\ =:& \mathbb{P}\left({\mathbb{A}}_1\right)+\mathbb{P}\left({\mathbb{A}}_2\right).\hfill \end{array}$$

(114)

Let us introduce the semi-norm

$${\tilde{d}}_{n\varphi ,2}:={\left(k^{-1}\sum _{j=1}^{{\upsilon }_n}\sum _{i\in H_j}{\left|{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_1,K_1}({\varsigma }_i,{\zeta }_i)-{\mathit{\eta }}_{n,t,\mathit{\theta },{\phi }_2,K_2}({\varsigma }_i,{\zeta }_i)\right|}^2\right)}^{1/2}$$

and the covering number defined for any class of functions $\mathcal{E}$ by

$${\tilde{N}}_{n\varphi ,2}(u,\mathcal{E}):=N_{n\varphi ,2}(u,\mathcal{E},{\tilde{d}}_{n\varphi ,2}).$$

By utilizing the latter approach, we can bound $\mathbb{P}\left({\mathbb{A}}_1\right)$ (the detailed calculations can be found in [110]). Similarly, as in [110] and earlier in [109], leveraging the independence between the blocks and Condition (C.4.3), we apply Lemma 5.2 from [214] to obtain

$$\mathbb{P}\left({\mathbb{A}}_2\right)\to 0.$$

Consequently, the theorem is established. □

8.3.2. Proof of Theorem 14

As previously mentioned, the examination of the weak convergence of the conditional U-process is grounded in the analysis of two components: the truncated part and the remainder part.

 Lemma 1. 

Let ${\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m$ be a uniformly bounded class of measurable canonical functions from ${\mathcal{X}}^m\times {\mathcal{Y}}^m\to \mathbb{R}$, where $m\ge 2$. Assume that there exist finite constants $\mathbf{a}$ and $\mathbf{b}$ such that the covering number of ${\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m$ satisfies

$$N(ϵ,{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m{,\parallel ·\parallel }_{L_2\left(Q\right)})\le \mathbf{a}{ϵ}^{-\mathbf{b}},$$

(115)

for every $ϵ>0$ and every probability measure Q. If the mixing coefficient β of the stationary sequence ${\{Z_i=(X_i,Y_i)\}}_{i\in {\mathbb{N}}^{*}}$ fulfills

$${\beta }_s{s}^r⟶0,ass\to \infty$$

(116)

for some $r>1$, then

$${∥n^{{\displaystyle \frac{-3m+1}{2}}}\sqrt{k^m}\sum _{\mathbf{i}\in I_m^n}{G}_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{h}}_{n,k}}({\mathbf{X}}_i,{\mathbf{Y}}_i)∥}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}\stackrel{\mathbb{P}}{⟶}0.$$

8.3.3. Proof of Theorem 15

It is well established that the weak convergence of an empirical process can be derived from its finite-dimensional convergence and its asymptotic equicontinuity, provided certain conditions are satisfied. Theorem 14 establishes the finite-dimensional convergence of the conditional U-process ${\left\{{\mu }_n(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))\right\}}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}$. Thus, the remaining task is to demonstrate its asymptotic equicontinuity. We decompose the U-process ${\mu }_n(\phi ,\mathbf{t},\mathit{\theta })$ into two components: the truncated part and the remainder part.

$${\mu }_n(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))={\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))+{\mu }_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right)).$$

Following the same reasoning as in [155], we also have

$${\mu }_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{D}}_n^{+})\le {\mu }_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(t\right))\le {\mu }_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{D}}_n^{-}).$$

(117)

Therefore, it suffices to establish the weak convergence of ${\mu }_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{D}}_n^{-})$ and ${\mu }_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{D}}_n^{+})$ rather than directly studying ${\mu }_n(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(t\right))$. The steps of the proof closely follow those in [112], with the distinction that we now consider a multivariate bandwidth $\mathbf{h}\left(\mathbf{t}\right)$ instead of a univariate bandwidth $h_K$. In this section, we present the proof for ${\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{D}}_n^{-})$, with the same approach applicable to ${\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{D}}_n^{+})$. As shown earlier, the truncated part ${\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}\left(\mathbf{t}\right))$ is decomposed following Hoeffding’s decomposition:

$$\begin{array}{c}\hfill {\mu }_n^{\left(T\right)}(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-})=\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}\left\{m{u}_n^{\left(1\right)}\left({\pi }_{1,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(T\right)}\right)+\sum _{p=2}^m{\displaystyle \frac{m!}{(m-p)!}}{u}_n^{\left(p\right)}\left({\pi }_{p,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(T\right)}\right)\right\}.\end{array}$$

We first investigate the linear term $m\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}{u}_n^{\left(1\right)}\left({\pi }_{1,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(T\right)}\right)$. Notice that

$$m\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}{u}_n^{\left(1\right)}\left({\pi }_{1,m}{G}_{\phi ,\mathbf{t},\mathit{\theta }}^{\left(T\right)}\right)={\displaystyle \frac{m\sqrt{{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}{\sqrt{n}}}\sum _{i=1}^n{\pi }_{1,m}{G}_{\phi ,\mathbf{t},\mathit{\theta }}^{\left(T\right)}({\mathbf{X}}_i,{\mathbf{Y}}_i)$$

We can write

$$\begin{array}{cc}\hfill {\pi }_{1,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(T\right)}(x,y)& =\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(T\right)}(x,X_2,\dots ,X_m),(y,X_2,\dots ,X_m)\right]-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(T\right)}(\mathbf{X},\mathbf{Y})\right]\hfill \\ \hfill & =\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(T\right)}(\mathbf{X},\mathbf{Y})|(X_1,Y_1)=(x,y)\right]-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(T\right)}(\mathbf{X},\mathbf{Y})\right].\hfill \end{array}$$

We need to introduce a new function

$$\begin{array}{cc}\hfill S_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}:& \mathcal{X}\times \mathcal{Y}⟶\mathbb{R}\hfill \\ \hfill & (x,y)⟶m\mathbb{E}\left[\phi \left(y\right)\tilde{K}\left({\displaystyle \frac{d(\mathbf{t},\mathbf{x})}{\mathbf{h}\left(\mathbf{t}\right)}}\right)|(X_1,Y_1)=(x,y)\right].\hfill \end{array}$$

Hence,

$$m{\pi }_{1,m}{G}_{\phi ,\mathbf{t},\mathit{\theta }}^{\left(T\right)}(x,y)={\tilde{\varphi }}^{-1/m}\left({\mathbf{D}}_n^{-}\right)\left(S_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}(x,y)-\mathbb{E}\left[S_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}(x,y)\right]\right).$$

The linear term of the process is given by

$$\begin{array}{cc}\hfill {\displaystyle m}& \sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}{u}_n^{\left(1\right)}\left({\pi }_{1,m}{G}_{\phi ,\mathbf{t},\mathit{\theta }}^{\left(T\right)}\right)\hfill \\ & ={\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{i=1}^n\left\{S_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}(X_i,Y_i)-\mathbb{E}\left[S_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}(X_i,Y_i)\right]\right\}\hfill \\ & :={\alpha }_n\left(S_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}\right).\hfill \end{array}$$

Thus, the linear term of the U-process ${\left\{{\mu }_n(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-})\right\}}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}$ is an empirical process indexed by the class of functions $\mathcal{S}$, defined as follows:

$$\mathcal{S}=\left\{S_{\phi ,\mathbf{t},\mathit{\theta },\mathbf{h}}(·,·)\phi \in {\mathcal{F}}_m,\mathbf{t}=(t_1,\dots ,t_m)\in {\mathcal{X}}^m\right\}.$$

Hence, its weak convergence can be established in a manner similar to the proof of Theorem 14. It is evident that $\mathcal{S}\subset m{\mathcal{G}}^{\left(1\right)}$. Now, turning to the nonlinear part, we must demonstrate that

$${∥\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}{u}_n^{\left(p\right)}\left({\pi }_{k,m}{G}_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(T\right)}\right)∥}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}\stackrel{\mathbb{P}}{⟶}0,\mathrm{for}2\le p\le m.$$

This is a consequence of Lemma 1. It is important to note that the selection of the number and size of the blocks must be performed in such a way that the terms $\mathrm{I}-\mathrm{VI}$ converge to 0. We need to prove that

$$\mathbb{P}\left\{{∥{\mu }_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-})∥}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}>\lambda \right\}\to 0\mathrm{as}n\to \infty .$$

Again, we confine our discussion to the case $m=2$ for clarity. We have

$$\begin{array}{c}\hfill {\displaystyle {\mu }_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-})}\\ & & =\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}\left\{u_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-})-\mathbb{E}\left(u_n^{\left(R\right)}(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-})\right)\right\}\hfill \\ & & ={\displaystyle \frac{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}{n(n-1)}}\sum _{i\ne j}^n\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left((X_i,X_j),(Y_i,Y_j)\right)\right]\right\}\hfill \\ & & \le {\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left((X_i,X_j),(Y_i,Y_j)\right)\right]\right\}\hfill \\ & & +{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\ne ji,j\in H_p^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left((X_i,X_j),(Y_i,Y_j)\right)\right]\right\}\hfill \\ & & +2{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\hfill \\ & & \times \left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left((X_i,X_j),(Y_i,Y_j)\right)\right]\right\}\hfill \\ \hfill \\ & & +2{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩽1}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)\hfill \\ & & \times \left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left((X_i,X_j),(Y_i,Y_j)\right)\right]\right\}\hfill \\ \hfill \\ & & +{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in T_p^{\left(U\right)}}\sum _{j\in T_q^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left((X_i,X_j),(Y_i,Y_j)\right)\right]\right\}\hfill \\ & & +{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\ne ji,j\in T_p^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left((X_i,X_j),(Y_i,Y_j)\right)\right]\right\}\hfill \\ & & =:{\mathrm{I}}^{\prime }+{\mathrm{II}}^{\prime }+{\mathrm{III}}^{\prime }+{\mathrm{IV}}^{\prime }+{\mathrm{V}}^{\prime }+{\mathrm{VI}}^{\prime }.\hfill \end{array}$$

We employ blocking arguments to handle the resulting terms. We begin by examining the first term ${\mathrm{I}}^{\prime }$. We have

$$\begin{array}{ccc}& & \hfill {\displaystyle \mathbb{P}\left\{{∥\frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left((X_i,X_j),(Y_i,Y_j)\right)\right]\right\}∥}_{{\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2}\right.}\\ & & >\delta \}\hfill \\ & & \le \mathbb{P}\left\{{∥{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)\right]\right\}∥}_{{\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2}\right.\hfill \\ & & >\delta \}+2{\upsilon }_n{\beta }_{b_n}.\hfill \end{array}$$

Notice that (48) readily implies that ${\upsilon }_n{\beta }_{b_n}\to 0$ and recall that for all $\phi \in {\mathcal{F}}_m$,

$$\mathbf{x},\mathbf{t}\in {\mathcal{X}}^2,\mathbf{y}\in {\mathcal{Y}}^2:{\kappa }_2^2{\mathbb{1}}_{\left\{d\left(\mathbf{x},\mathbf{t}\right)⩽h_K\right\}}F\left(\mathbf{y}\right)⩾\phi \left(\mathbf{y}\right)\tilde{K}\left({\displaystyle \frac{d\left(\mathbf{x},\mathbf{t}\right)}{h_K}}\right).$$

By the symmetry of Function $F(·)$, it holds that

$$\begin{array}{ccc}\hfill & & {∥{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)\right]\right\}∥}_{{\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2}\hfill \\ & & ⩽\left|{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}\left\{{\kappa }^{2}F({\zeta }_i,{\zeta }_j){\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right.\right.\hfill \\ & & \left.\left.-\mathbb{E}\left[{\kappa }^{2}F({\zeta }_i,{\zeta }_j){\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right]\right\}\phantom{{\displaystyle \frac{1}{\sqrt{n\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}}\right|.\hfill \end{array}$$

(118)

Employing Chebyshev’s inequality and Hoeffding’s trick while maintaining order, we obtain

$$\begin{array}{ccc}& & \mathbb{P}\left\{\left|{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}\left\{{\kappa }^{2}F({\zeta }_i,{\zeta }_j){\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right.\right.\right.\hfill \\ & & \left.\left.\left.-\mathbb{E}\left[{\kappa }^{2}F({\zeta }_i,{\zeta }_j){\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right]\right\}\phantom{{\displaystyle \frac{1}{\sqrt{n\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}}\right|>\delta \right\}\hfill \\ & & ⩽{\delta }^{-2}{n}^{-1}{\tilde{\varphi }}^{-1/m}\left({\mathbf{D}}_n^{-}\right)Var\left(\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}{\kappa }^{2}F({\zeta }_i,{\zeta }_j){\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right)\hfill \\ & & ⩽c_2{\upsilon }_n{\delta }^{-2}{n}^{-1}{\tilde{\varphi }}^{-1/m}\left({\mathbf{D}}_n^{-}\right)Var\left(\sum _{p=1}^{{\upsilon }_n}\sum _{i,j\in H_p^{\left(U\right)}}{\kappa }^{2}F({\zeta }_i,{\zeta }_j^{\prime }){\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right)\hfill \\ & & ⩽2c_2{\upsilon }_n{\delta }^{-2}{n}^{-2}{\tilde{\varphi }}^{-1/m}\left({\mathbf{D}}_n^{-}\right)\mathbb{E}\left[{\left({\kappa }^{2}F({\zeta }_1,{\zeta }_2)\right)}^2{\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right].\hfill \end{array}$$

(119)

Under (C.6.) we have, for each $\lambda >0$,

$$\begin{array}{ccc}& & c_2{\upsilon }_n{\delta }^{-2}{n}^{-2}{\tilde{\varphi }}^{-1/m}\left({\mathbf{D}}_n^{-}\right)\mathbb{E}\left[{\left({\kappa }^{2}F({\zeta }_1,{\zeta }_2)\right)}^2{\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right]\hfill \\ & & =c_2{\upsilon }_n{\delta }^{-2}{n}^{-2}{\tilde{\varphi }}^{-1/m}\left({\mathbf{D}}_n^{-}\right){\int }_0^{\infty }\mathbb{P}\left\{{\left({\kappa }^{2}F({\zeta }_1,{\zeta }_2)\right)}^2{\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}⩾t\right\}dt\hfill \\ & & =c_2{\upsilon }_n{\delta }^{-2}{n}^{-2}{\tilde{\varphi }}^{-1/m}\left({\mathbf{D}}_n^{-}\right){\int }_0^{\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}}\mathbb{P}\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}dt\hfill \\ & & +c_2{\upsilon }_n{\delta }^{-2}{n}^{-2}{\tilde{\varphi }}^{-1/m}\left({\mathbf{D}}_n^{-}\right){\int }_{\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}}^{\infty }\mathbb{P}\left\{{\left({\kappa }^{2}F\right)}^2>t\right\}dt,\hfill \end{array}$$

which tends to 0 as $n\to \infty$. Terms ${\mathrm{II}}^{\prime },{\mathrm{V}}^{\prime }$, and ${\mathrm{VI}}^{\prime }$ are handled in the same manner as the first, with the exception that for ${\mathrm{II}}^{\prime },{\mathrm{VI}}^{\prime }$, we do not need to apply Hoeffding’s trick. This is because our variables ${\{{\zeta }_i,{\zeta }_j\}}_{i,j\in H_p^{\left(U\right)}}$ (or ${\{{\zeta }_i,{\zeta }_j\}}_{i,j\in T_p^{\left(U\right)}}$ for ${\mathrm{VI}}^{\prime }$) are in the same blocks. For term ${\mathrm{IV}}^{\prime }$, we deduce its study from those of ${\mathrm{I}}^{\prime }$ and ${\mathrm{III}}^{\prime }$. Let us consider term ${\mathrm{III}}^{\prime }$. As for the truncated part, we have

$$\begin{array}{ccc}& \mathbb{P}& \left\{∥{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left(X_i,X_j),(Y_i,Y_j\right)\right)\right.\right.\hfill \\ & & \left.{\left.-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left((X_i,X_j),(Y_i,Y_j)\right)\right]\right\}\phantom{{\displaystyle \frac{1}{\sqrt{n\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}}∥}_{{\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2}>\delta \right\}\hfill \\ & & ⩽\mathbb{P}\left\{∥{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j\right)\right)\right.\right.\hfill \\ & & \left.{\left.-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)\right]\right\}\phantom{{\displaystyle \frac{1}{\sqrt{n\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p=1}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}}∥}_{{\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2}>\delta \right\}\hfill \\ & & +{\displaystyle \frac{{\upsilon }_n^2{a}_n{b}_n{\beta }_{a_n}}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}.\hfill \end{array}$$

(120)

We also have

$$\begin{array}{ccc}& & \mathbb{P}\left\{{∥{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{i\in H_p^{\left(U\right)}}\sum _{q:|q-p|⩾2}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)\right]\right\}∥}_{{\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2}\right.\hfill \\ & & >\delta \}\hfill \\ & & ⩽\mathbb{P}\left\{{∥{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{i\in H_1^{\left(U\right)}}\sum _{q=3}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}\left\{G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(\left({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j\right)\right)-\mathbb{E}\left[G_{\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-}}^{\left(R\right)}\left(({\varsigma }_i,{\varsigma }_j),({\zeta }_i,{\zeta }_j)\right)\right]\right\}∥}_{{\mathcal{F}}_2{\mathfrak{K}}_{\mathbf{\Theta }}^2}\right.\hfill \\ & & >\delta \}.\hfill \end{array}$$

Since Equation (118) is still satisfied, the problem is reduced to

$$\begin{array}{ccc}& & \mathbb{P}\left\{\left|{\displaystyle \frac{1}{\sqrt{n{\tilde{\varphi }}^{1/m}\left({\mathbf{D}}_n^{-}\right)}}}\sum _{i\in H_1^{\left(U\right)}}\sum _{q=3}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}\left\{{\kappa }^{2}F({\zeta }_i,{\zeta }_j){\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right.\right.\right.\hfill \\ & & \left.\left.\left.-\mathbb{E}\left[{\kappa }^{2}F({\zeta }_i,{\zeta }_j){\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right]\right\}\phantom{{\displaystyle \frac{1}{\sqrt{n\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)}}}\sum _{p\ne q}^{{\upsilon }_n}\sum _{i\in H_p^{\left(U\right)}}\sum _{j\in H_q^{\left(U\right)}}}\right|>\delta \right\}\hfill \\ & & ⩽{\delta }^{-2}{n}^{-1/m}\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)Var\left(\sum _{i\in H_1^{\left(U\right)}}\sum _{q=3}^{{\upsilon }_n}\sum _{j\in T_q^{\left(U\right)}}{\kappa }^{2}F({\zeta }_i,{\zeta }_j){\mathbb{1}}_{\left\{{\kappa }^{2}F>\lambda {\left(n/\tilde{\varphi }\left({\mathbf{D}}_n^{-}\right)\right)}^{1/2(p-1)}\right\}}\right),\hfill \end{array}$$

and we follow the same procedure as in (119). The remaining terms have been shown to be asymptotically negligible. Consequently, process ${\left\{{\mu }_n(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{-})\right\}}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}$ converges in distribution to a Gaussian process, which admits a version with uniformly bounded and uniformly continuous paths with respect to the ${\parallel ·\parallel }_2$-norm. By applying the same reasoning, the same result holds for process ${\left\{{\mu }_n(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{D}}_n^{+})\right\}}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}$. Thus, according to (117), it follows that process ${\left\{{\mu }_n(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right\}}_{{\mathcal{F}}_m{\mathfrak{K}}_{\mathbf{\Theta }}^m}$ also converges in distribution to a Gaussian process, which has a version with uniformly bounded and uniformly continuous paths with respect to the ${\parallel ·\parallel }_2$-norm. Similarly, process ${\left\{{\mu }_n(1,\mathbf{t},\mathit{\theta },{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right\}}_{{\mathfrak{K}}_{\mathbf{\Theta }}^m}$ is treated in the same manner, and for $\mathbb{E}\left(u_n(\phi ,\mathbf{t},\mathit{\theta },{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right)$, $u_n(1,\mathbf{t},\mathit{\theta },{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))$, and $\mathbb{E}\left(u_n(1,\mathbf{t},\mathit{\theta },{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))\right)$, the analysis follows the same steps as in the proof of Theorem 14. □

Appendix A.1. General Lemma

Lemma A1 is presented in a broad context, as illustrated in [51], making it applicable beyond the specific scenario under consideration. This represents an extension of the result established in [118]. In a more general sense, this technical tool proves invaluable for addressing random bandwidths. Consider a sequence of random vectors, ${({\mathbf{A}}_i,{\mathbf{B}}_i)}_{n\ge 1}$, taking values in $({\Omega }^m\times {\Omega }^{\prime m},\mathcal{A}\times \mathcal{B})$, a general space. Let $S_{\Omega }$ be a fixed subset of $\Omega$. Additionally, define $G:\mathbb{R}\times (S_{\Omega }\times \Omega )\to {\mathbb{R}}^{+}$ as a function such that, for every $t\in S_{\Omega }$, $G(·,(t,·\left)\right)$ is measurable. Furthermore, for all $x,x^{\prime }\in \mathbb{R}$, the following condition holds:

• (L₀) If $x\le x^{\prime }$, then $G(⟨x,\mathit{\theta }⟩,\mathbf{z})\le G(⟨x^{\prime },\mathit{\theta }⟩,\mathbf{z})$, for all $\mathbf{z}\in S_{\Omega }\times \Omega$, $\mathit{\theta }\in \mathbf{\Theta }$.
• Define the pointwise measurable class of functions for $1\le m\le n$:

  $${\mathcal{G}}^m:=\left\{(⟨x_1,{\mathit{\theta }}_1⟩,\dots ,⟨x_m,{\mathit{\theta }}_m⟩)\mapsto \prod _{i=1}^{m}G(h_i,(⟨x_i,{\mathit{\theta }}_i⟩,t_i)),(h_1,\dots ,h_m)\in {\mathbb{R}}_{+}^m\setminus \left\{0\right\},(t_1,\dots ,t_m)\in S_{\Omega }^m\right\}.$$

  Next, consider a sequence ${\left({\mathbf{D}}_{n,k}(\mathbf{t},\mathit{\theta })\right)}_{n\in \mathbb{N}}$ consisting of random real vectors, where for all $\mathbf{t}=(t_1,\dots ,t_m)\in S_{\Omega }^m$ and $\mathit{\theta }\in {\mathbf{\Theta }}^m$, ${\mathbf{D}}_{n,k}(\mathbf{t},\mathit{\theta })=\left(D_{n,k}\left(⟨x_1,{\mathit{\theta }}_1⟩\right),\dots ,D_{n,k}\left(⟨x_m,{\mathit{\theta }}_m⟩\right)\right)$. Let $\phi :{\Omega }^{\prime m}\to \mathbb{R}$ be a measurable function belonging to a class of functions ${\mathcal{F}}_m$. Define ${\mathfrak{M}}^{\left(m\right)}:{\mathcal{F}}_m\times S_{\Omega }^m\times {\mathbf{\Theta }}^m\to \mathbb{R}$ as a non-random function satisfying

  $$\underset{\phi \in {\mathcal{F}}_m}{sup}\underset{\mathbf{t}\in S_{\Omega }^m}{sup}\underset{\mathit{\theta }\in {\mathbf{\Theta }}^m}{sup}\left|{\mathfrak{M}}^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|<\infty .$$

  For all $\mathbf{t}\in S_{\Omega }^m$, $\mathit{\theta }\in {\mathbf{\Theta }}^m$, $\phi \in {\mathcal{F}}_m$, and $n\in \mathbb{N}\setminus \left\{0\right\}$, define

  $${\mathfrak{M}}_n^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h})={\displaystyle \frac{{\displaystyle \sum _{i\in I(m,n)}\phi (B_{i_1},\dots ,B_{i_m})\prod _{j=1}^{m}G(H_j,(⟨t_j,{\mathit{\theta }}_j⟩,A_{i,j}))}}{{\displaystyle \sum _{i\in I(m,n)}\prod _{j=1}^{m}G(H_j,(⟨t_j,{\mathit{\theta }}_j⟩,A_{i,j}))}}}.$$

 Lemma A1. 

Let ${\left({\mathcal{U}}_n\right)}_{n\in \mathbb{N}}$ be a decreasing positive sequence such that ${\displaystyle \underset{n\to \infty }{lim}{\mathcal{U}}_n=0}$. If, for all increasing sequence ${\mathit{\xi }}_n\in \left(0,1\right)$ with ${\mathit{\xi }}_n-1=O\left({\mathcal{U}}_n\right)$, there exist two sequences of r.r.v. ${\left({\mathbf{D}}_{n,k}^{-}({\mathit{\xi }}_n,\mathbf{t},\mathit{\theta })\right)}_{n\in \mathbb{N}}$ and ${\left({\mathbf{D}}_{n,k}^{+}({\mathit{\xi }}_n,\mathbf{t},\mathit{\theta })\right)}_{n\in \mathbb{N}}$ such that

 (L₁) 

$$\forall n\in \mathbb{N}and\mathbf{t}\in S_{\Omega }^m,D_{n,k}^{-}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩)\le D_{n,k}^{+}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩),\forall j=1,\dots ,m,$$

 (L₂) 

$$\prod _{j=1}^m{\mathbb{1}}_{\left\{D_{n,k}^{-}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩)\le D_{n,k}\left(⟨t_j,{\mathit{\theta }}_j⟩\right)\le D_{n,k}^{+}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩)\right\}}\to 1,a.co,\forall \mathbf{t}\in S_{\Omega }^m,$$

 (L₃) 

$$\underset{\tilde{G}\in {\mathcal{G}}^m}{sup}\underset{\mathbf{t}\in S_{\Omega }^m}{sup}\left|{\displaystyle \frac{{\displaystyle \sum _{i\in I(m,n)}\prod _{j=1}^{m}G\left(D_{n,k}^{-}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩),(⟨t_j,{\mathit{\theta }}_j⟩,A_{i,j})\right)}}{{\displaystyle \sum _{i\in I(m,n)}\prod _{j=1}^{m}G\left(D_{n,k}^{+}({\mathit{\xi }}_n,⟨t_j,{\mathit{\theta }}_j⟩),(⟨t_j,{\mathit{\theta }}_j⟩,A_{i,j})\right)}}}-{\mathit{\xi }}_n\right|=O_{a.co}\left({\mathcal{U}}_n\right),$$

 (L₄) 

$$\underset{\phi \tilde{G}\in {\mathcal{F}}_m{\mathcal{G}}^m}{sup}\underset{\mathbf{t}\in S_{\Omega }^m}{sup}\left|{\mathfrak{M}}_n^{\left(m\right)}\left(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{D}}_{n,k}^{-}({\mathit{\xi }}_n,\mathbf{t},\mathit{\theta })\right)-{\mathfrak{M}}^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|=O_{a.co}\left({\mathcal{U}}_n\right),$$

 (L₅) 

$$\underset{\phi \tilde{G}\in {\mathcal{F}}_m{\mathcal{G}}^m}{sup}\underset{\mathbf{t}\in S_{\Omega }^m}{sup}\left|{\mathfrak{M}}_n^{\left(m\right)}\left(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{D}}_{n,k}^{+}({\mathit{\xi }}_n,\mathbf{t},\mathit{\theta })\right)-{\mathfrak{M}}^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|=O_{a.co}\left({\mathcal{U}}_n\right).$$

Then, as $n\to \infty$,

$$\underset{\phi \tilde{G}\in {\mathcal{F}}_m{\mathcal{G}}^m}{sup}\underset{\mathbf{t}\in S_{\Omega }^m}{sup}\left|{\mathfrak{M}}_n^{\left(m\right)}\left(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{D}}_{n,k}(\mathbf{t},\mathit{\theta })\right)-{\mathfrak{M}}^{\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta })\right|=O_{a.co}\left({\mathcal{U}}_n\right).$$

(A1)

Refer to [51] for proof of this lemma.

 Lemma A2. 

Under Assumptions  (C.1.1),  (C.3.1),  (C.4.1),  (C.5.1),  (C.5.2’),  (C.6.)   and  (C.7.), obtain, as $n\to \infty$,

$$\begin{array}{c}\hfill {\displaystyle \underset{\phi K\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\underset{h_{n,1}\le h_K\le h_{n,2}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\underset{\mathit{\theta }\in \mathbf{\Theta }}{sup}\left|{\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,h_K\left(t\right))-\mathbb{E}\left({\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,h_K\left(t\right))\right)\right|}\\ =O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left(h_{n,1}\right)}}}}\right),\hfill \end{array}$$

(A2)

and

$$\begin{gathered} {\displaystyle \underset{\phi K\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\underset{h_{n,1}\le h_K\le h_{n,2}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\underset{\mathit{\theta }\in \mathbf{\Theta }}{sup}\left|{\widehat{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,h_K\left(t\right))-1\right|} \\ =O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left(h_{n,1}\right)}}}}\right). \end{gathered}$$

(A3)

This lemma establishes the consistency rates for the stochastic component when the function class $\mathcal{F}$ is restricted in scope. The following lemma extends this result to the case where the function class is unbounded.

 Lemma A3. 

Under Assumptions  (C.1.1),  (C.3.1),  (C.4.2),  (C.5.1),  (C.5.2’),  (C.6.)   and  (C.7.), obtain, as $n\to \infty$,

$$\begin{array}{c}\hfill {\displaystyle \underset{\phi K\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\underset{h_{n,1}\le h_K\le h_{n,2}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\left|{\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,h_K\left(t\right))-\mathbb{E}\left({\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,h_K\left(t\right))\right)\right|}\\ =O_{a.co}\left(\sqrt{{\displaystyle \frac{{\displaystyle {\psi }_{S_{\mathcal{H}}}\left(\frac{logn}{n}\right)+{\psi }_{\mathbf{\Theta }}\left(\frac{logn}{n}\right)}}{{\displaystyle n\varphi \left(h_{n,1}\right)}}}}\right).\hfill \end{array}$$

(A4)

 Lemma A4. 

Under Conditions  (C.1.),  (C.3.1), and  (C.5.1)   (or equivalently  (C.1’.),  (C.3’.)   and  (C.5.1)), and if $n\varphi \left(h_K\right)\to \infty$ as $n\to \infty$, obtain, for each $t\in \mathcal{H}$,

$${\widehat{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩;h_K)\stackrel{p.s.}{⟶}1.$$

(A5)

Before presenting the next result, it is necessary to introduce the following notation:

$$\begin{array}{ccc}\hfill {\mu }_{n,1}\left(t,\mathit{\theta };h_K\right)& :=& \mathbb{E}\left[\left(\phi \left(Y_1\right)-r^{\left(1\right)}(\phi ,t)\right){\Delta }_1(t,\mathit{\theta };h_K)\right],\hfill \end{array}$$

(A6)

$$\begin{array}{ccc}\hfill Z_{n,i}\left(h_K\right)& :=& \left(\phi \left(Y_i\right)-r^{\left(1\right)}(\phi ,t;h_K)\right){\Delta }_i(t,\mathit{\theta };h_K)-{\mu }_{n,1}\left(⟨t,\mathit{\theta }⟩;h_K\right).\hfill \end{array}$$

(A7)

Set

$$Q_n(t,\mathit{\theta };h_K)={\displaystyle \frac{1}{n\mathbb{E}\left[{\Delta }_1(t,\mathit{\theta };h_K)\right]}}\sum _{i=1}^n{Z}_{n,i}\left(h_K\right).$$

and

$${\sigma }_{n,0}^2(t,\mathit{\theta };h_K):={\displaystyle \frac{1}{{\mathbb{E}}^2\left[{\Delta }_1(t,\mathit{\theta };h_K)\right]}}\mathrm{Var}\left[Z_{n,1}\left(h_K\right)\right].$$

 Lemma A5. 

Under Conditions  (C.1.),  (C.2.),  (C.3.1)   and  (C.5.1)   (with $p=\infty$ and $\delta >1$ for condition  (C.5.1)), obtain, for $n\varphi \left(h_K\right)\underset{n\to \infty }{⟶}\infty ,t\in \mathcal{X}$ and positive constants $C_4,C_5$,

$$C_4{\displaystyle \frac{g_2(t,\mathit{\theta },\phi )}{f_1\left(t\right)}}\le \varphi \left(h_K\right){\sigma }_{n,0}^2(t,\mathit{\theta })\le C_5{\displaystyle \frac{g_2(t,\mathit{\theta },\phi )}{f_1\left(t\right)}},$$

(A8)

whenever $f_1\left(t\right)>0.$ Obtain

$$\begin{array}{ccc}\hfill {\displaystyle \frac{1}{n{\mathbb{E}}^2\left[{\Delta }_1(t,\mathit{\theta };h_K)\right]}}\underset{i\ne j}{\sum _{i=1}^n\sum _{i=1}^n}\mathrm{Cov}\left[Z_{n,i},Z_{n,j}\right]& =& o\left({\sigma }_{n,0}^2(t,\mathit{\theta })\right).\hfill \end{array}$$

(A9)

$$\begin{array}{ccc}\hfill \mathrm{Var}\left[Q_n(t,\mathit{\theta })\right]& =& {\sigma }_{n,0}^2(t,\mathit{\theta })(1+o\left(1\right))/n.\hfill \end{array}$$

(A10)

 Lemma A6. 

Under Conditions  (C.1’.),  (C.2.),  (C.3’.)   and  (C.5.1)   (with $p=\infty$ and $\delta >1$ for condition  (C.5.1)), obtain, for $n\varphi \left(h_K\right)\to \infty$ as $n\to \infty$ and $t\in \mathcal{H}$,

$$\varphi \left(h_K\right){\sigma }_{n,0}^2(t,\mathit{\theta })⟶{\displaystyle \frac{{\kappa }_1^{\prime }}{{\kappa }_2^{\prime }}}{\displaystyle \frac{g_2(t,\mathit{\theta },\phi )}{f_1\left(t\right)}},$$

(A11)

whenever $f_1\left(t\right)>0,$ and ${\kappa }_1^{\prime },{\kappa }_2^{\prime }$ are constants specified previously.

$$\begin{array}{ccc}\hfill {\displaystyle \frac{1}{n{\mathbb{E}}^2\left[{\Delta }_1(t,\mathit{\theta };h_K)\right]}}\underset{i\ne j}{\sum _{i=1}^n\sum _{i=1}^n}\mathrm{Cov}\left[Z_{n,i},Z_{n,j}\right]& =& o\left({\sigma }_{n,0}^2\left(t\right)\right),\hfill \end{array}$$

(A12)

$$\begin{array}{ccc}\hfill n\varphi \left(h_K\right)\mathrm{Var}\left[Q_n(t,\mathit{\theta };h_K)\right]& ⟶& {\displaystyle \frac{{\kappa }_1^{\prime }}{{\kappa }_2^{\prime }}}{\displaystyle \frac{g_2(t,\mathit{\theta },\phi )}{f_1\left(t\right)}}.\hfill \end{array}$$

(A13)

To simplify the notation and for convenience, denote

$${\sigma }^2(t,\mathit{\theta }):=\left({\kappa }_1^{\prime }{g}_2(t,\mathit{\theta },\phi )\right)/\left({\kappa }_2^{\prime }{f}_1\left(t\right)\right).$$

 Lemma A7. 

Under Conditions  (C.1’.),  (C.2.),  (C.3’.),  (C.5.1)   and  (C.5.2), if $n\varphi \left(h_K\right)\to \infty$ as $n\to \infty ,$ obtain, for $t\in \mathcal{H}$, as $n\to \infty$,

$${\left(n\varphi \left(h_K\right)\right)}^{1/2}{Q}_n(t,\mathit{\theta },h_K)⟶\mathcal{N}\left(0,{\sigma }^2(t,\mathit{\theta })\right).$$

(A14)

 Lemma A8. 

Under Conditions  (C.1.),  (C.2.),  (C.3.1)   and  (C.5.1), and if

$$\left(n\varphi \left(h_K\right)/loglogn\right)\underset{n\to \infty }{⟶}\infty ,$$

we obtain, as $n\to \infty$,

$${\left({\displaystyle \frac{n\varphi \left(h_K\right)}{loglogn}}\right)}^{1/2}\left[{\widehat{r}}_n^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,\mathit{\theta })-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩,\mathit{\theta })-B_n\left(⟨t,\mathit{\theta }⟩\right)\right]\stackrel{\mathbb{P}}{⟶}0.$$

(A15)

The detailed proofs of the aforementioned lemmas can be found in [111,112,155].

The following result concerning the bias term is all that remains to be established. This lemma can be derived using a similar approach to that in [111], where more comprehensive details are provided.

 Lemma A9. 

Under Condition  (C.2.1), obtain, as $n\to \infty$,

$$\underset{\phi K\in \mathcal{F}{\mathfrak{K}}_{\mathbf{\Theta }}^{\mathcal{m}}}{sup}\underset{h_{n,1}\le h_K\le h_{n,2}}{sup}\underset{t\in S_{\mathcal{H}}}{sup}\left|\mathbb{E}\left[{\widehat{r}}_{n,2}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K\left(t\right))\right]-r^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩)\right|=O\left(h_{n,2}^{\gamma }\right).$$

(A16)

 Lemma A10. 

Under assumptions of Theorem 14, obtain

$$\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\left(\mathbf{t}\right)\right)}\left({\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))-\mathbb{E}\left[u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))\right]\right)\stackrel{d}{⟶}\mathcal{N}\left(0,m^2{\sigma }_{\mathbf{t},\mathit{\theta }}^2\left(\phi ,\phi \right)\right),$$

(A17)

and, if, in addition, Condition(C.8)is satisfied, then obtain

$$\sqrt{k}\left\{{\widehat{r}}_n^{*\left(m\right)}(\phi ,\mathbf{t},\mathit{\theta };{\mathbf{h}}_{n,k}\left(\mathbf{t}\right))-\mathbb{E}\left[u_n(\phi ,\mathbf{t},\mathit{\theta };\mathbf{h}\left(\mathbf{t}\right))\right]\right\}\stackrel{d}{⟶}\mathcal{N}\left(0,m^2{\sigma }_{\mathbf{t},\mathit{\theta }}^2\left(\phi ,\phi \right)\right).$$

(A18)

 Lemma A11. 

Under the assumptions of Theorem 14, obtain

$$\sqrt{n{\tilde{\varphi }}^{1/m}\left(\mathbf{h}\right)}\left(u_n({\phi }_1,\mathbf{t},\mathit{\theta },\mathbf{h})-\mathbb{E}\left(u_n({\phi }_1,\mathbf{t},\mathit{\theta },\mathbf{h}\right),u_n({\phi }_2,\mathbf{t},\mathit{\theta },\mathbf{h})-\mathbb{E}\left(u_n({\phi }_2,\mathbf{t},\mathit{\theta },\mathbf{h}\right)\right)\stackrel{d}{⟶}\mathcal{N}\left(0,\Sigma \right),$$

(A19)

where

$$\Sigma :={\left(m^2{\sigma }_{\mathbf{t},\mathit{\theta }}^2\left({\phi }_i,{\phi }_j\right)\right)}_{i,j=1,2}.$$

The proofs of the preceding lemmas can be found in [112,155]. Similarly, the following lemmas follow the same steps as in [155] and are thus omitted for brevity.

 Lemma A12. 

Under Assumptions  (C.2.1),  (C.2.2),  (C.3.2),  (C.4.2)   and  (C.8.), obtain

$$\sqrt{k}{I}_1\left(⟨t,\mathit{\theta }⟩,H_{n,k}\left(t\right)\right)\stackrel{\mathbb{P}}{\to }0,asn\to \infty$$

 Lemma A13. 

Under Assumptions  (C.1’.),  (C.2.1.),  (C.2.2.),  (C.3’.)   and  (C.5.)   and if

$$k{\left({\varphi }^{-1}\left({\displaystyle \frac{k}{n}}\right)\right)}^{2\gamma }\to 0asn\to \infty ,$$

obtain

$$\sqrt{k}{I}_2(\phi ,⟨t,\mathit{\theta }⟩,H_{n,k}(t,\mathit{\theta }))\to \mathcal{N}\left(0,{\sigma }^2(t,\mathit{\theta })\right),asn\to \infty .$$

 Lemma A14. 

Under Assumptions  (C.1),  (C.3.1), and  (C.5.1)   (or equivalently  (C.1’.),  (C.3’.)   and  (C.5.1)), and if

$$k{\left({\varphi }^{-1}\left({\displaystyle \frac{k}{n}}\right)\right)}^{2\gamma }\to 0asn\to \infty ,$$

obtain, for each $t\in \mathcal{H}$:

$${\widehat{r}}_{n,1}^{\left(1\right)}(1,⟨t,\mathit{\theta }⟩,H_{n,k}(t,\mathit{\theta }))-1\stackrel{\mathbb{P}}{\to }0asn\to \infty .$$

 Lemma A15. 

Under Conditions  (C.1.),  (C.3.1), and  (C.5.1)   (or equivalently  (C.1’.),  (C.3’.)   and  (C.5.1)), and if $n\varphi \left(h_K\right)\to \infty$ as $n\to \infty$, obtain, for each $t\in \mathcal{H}$,

$${\widehat{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K)\stackrel{p.s.}{⟶}\mathbb{E}\left({\widehat{r}}_{n,1}^{\left(1\right)}(\phi ,⟨t,\mathit{\theta }⟩;h_K)\right),$$

(A20)

as $n\to \infty .$

It is important to highlight that this lemma is broader in scope than Lemma A4. While the result obtained here is somewhat weaker than the uniform almost sure convergence rate established in Section 7, the conditions imposed in this lemma are less stringent, making it more generally applicable.