Wavelet Estimation of Partial Derivatives in Multivariate Regression Under Discrete-Time Stationary Ergodic Processes

Abstract

This study introduces a wavelet-based framework for estimating derivatives of a general regression function within discrete-time, stationary ergodic processes. The analysis focuses on deriving the integrated mean squared error (IMSE) over compact subsets of ${\mathbb{R}}^d$, while also establishing rates of uniform convergence and the asymptotic normality of the proposed estimators. To investigate their asymptotic behavior, we adopt a martingale-based approach specifically adapted to the ergodic nature of the data-generating process. Importantly, the framework imposes no structural assumptions beyond ergodicity, thereby circumventing restrictive dependence conditions. By establishing the limiting behavior of the wavelet estimators under these minimal assumptions, the results extend existing findings for independent data and highlight the flexibility of wavelet methods in more general stochastic settings.

1. Introduction

Nonparametric estimation has attracted sustained attention over many years, leading to a broad spectrum of methodological developments. Given its extensive range of applications and significant role within mathematical statistics, the estimation of both density and regression functions has become a central research theme. Among the most frequently employed approaches are kernel-type estimators, recognized for their flexibility and robust performance. For comprehensive treatments of these methods and their applications, the reader is referred to [1,2], and the references therein. Estimating derivatives of a function—whether a density or a regression function—serves as a powerful technique in statistical data analysis. Despite their importance, however, the nonparametric estimation of higher-order density derivatives has not been extensively explored. A key objective of this work is thus to investigate wavelet-based nonparametric estimators for the partial derivatives of multivariate densities. Derivative estimation arises in many disciplines, including economics and industry, where complex systems must often be modeled under limited prior knowledge. For instance, the second-order derivative of a density can support statistical tests for mode detection [3] and also guides bandwidth selection in kernel density estimation [4]. In nonparametric signal estimation, the logarithmic derivative of the density—defined by the ratio of the derivative to the density itself—is instrumental for optimal filtering and interpolation procedures [5], thereby making precise estimation crucial for accurate signal processing. Moreover, gradient estimation is essential for filament detection in point cloud data, a task of broad relevance in medical imaging, remote sensing, seismology, and cosmology [6]. Additional motivations and applications appear in regression analysis, Fisher information estimation, parameter estimation, and hypothesis testing [7]. Seminal investigations into density derivative estimation include [8,9,10], among others. Further context for related statistical challenges, such as regression, Fisher information estimation, parameter estimation, and hypothesis testing, may be found in [7].

It is likewise well documented that estimating first- and higher-order regression derivatives holds substantial practical importance. Examples include modeling human growth processes [11], assessing kidney function in lupus nephritis patients [12], and analyzing Raman spectra of bulk materials [13]. In nonparametric regression, derivative estimation is indispensable for constructing confidence intervals [14], selecting kernel bandwidths [15], and comparing regression curves [16]. In the setting of a homoscedastic regression model, ref. [17] proposed kernel M-estimators to estimate the first derivative of the regression function in a nonparametric fashion, and extended these ideas heuristically to higher-order derivatives. Derivative information also underpins modal regression, an alternative to traditional regression methods for investigating the relationship between a response variable $\mathbf{Y}$ and a predictor variable $\mathbf{X}$; see [18]. Foundational studies of regression function estimation can be found in [19,20]. In contrast, fewer works address the estimation of derivatives for stationary densities or regression functions, with most established results pertaining to independent and identically distributed data. Wavelet-based techniques have recently attracted growing interest in statistical estimation, largely owing to their adaptability to varying function regularity and their capacity to handle discontinuities effectively. Furthermore, wavelet methods often lead to computationally efficient algorithms that require relatively modest memory. For a detailed overview of wavelet methodologies in nonparametric functional estimation, see [21]. Examples of wavelet applications include estimating the integrated squared derivative of a univariate density for independent data [22] and for negatively or positively associated sequences [23]. The work of [24] extended these methods to partial derivatives of multivariate densities under independence, while [25,26] addressed the mixing scenario. More recently, ref. [27] studied wavelet estimators for the partial derivatives of multivariate densities under additive noise.

To the best of our knowledge, methods for wavelet-based estimation of partial derivatives of multivariate densities have not yet been extended to more general dependence structures beyond the strong mixing framework. Addressing this gap forms the central motivation of our work. In particular, we draw on a collection of martingale-based techniques that markedly differ from the tools typically employed under strong mixing conditions. Nevertheless, as the subsequent sections will illustrate, bridging this gap entails much more than merely combining pre-existing methodologies: it requires advanced mathematical developments tailored to wavelet-based estimation in an ergodic setting.

The remainder of this article is organised as follows. Section 2 reviews the necessary mathematical foundations and introduces the proposed class of linear wavelet estimators. Section 3 specifies the underlying assumptions and presents the principal theoretical results—uniform convergence rates and asymptotic normality established under weak-dependence conditions. Section 4 applies the methodology to regression-derivative estimation, whereas Section 5 explores its extension to mode regression. Concluding remarks and avenues for further investigation are given in Section 6. To maintain the flow of exposition, all proofs are collected in Section 7.

Notation

Unless otherwise specified, C denotes a positive constant whose value may change from line to line. We write ${\mathbb{1}}_A(·)$ for the indicator function of a set A. For two sequences of positive real numbers ${\left\{a_n\right\}}_{n\ge 1}$ and ${\left\{b_n\right\}}_{n\ge 1}$, we use the Landau notation

$$a_n=O\left(b_n\right)$$

when there exists a constant $C>0$ such that $a_n\le C{b}_n$ for all sufficiently large n, and

$$a_n=o\left(b_n\right)$$

when ${\displaystyle \underset{n\to \infty }{lim}\frac{a_n}{b_n}=0}$.

2. Mathematical Backgrounds

2.1. Besov Spaces

Throughout this study we adopt the Besov scale ${\mathbf{B}}_{s,p,q}$ with parameters $s>0$ and $1\le p,q\le \infty$. Meyer’s wavelet characterization [28] shows that, for $0<s<r$ and any $f\in L_p\left({\mathbb{R}}^d\right)$, the inclusion $f\in {\mathbf{B}}_{s,p,q}$ is equivalent to the finiteness of either of the two seminorms

(B.1)

$$J_{s,p,q}\left(f\right)={∥P_{V_0}f∥}_{L_p}+{\left(\sum _{j>0}{\left(2^{js}{\parallel P_{W_j}f\parallel }_{L_p}\right)}^q\right)}^{1/q}<\infty ,$$

(B.2)

$$J_{s,p,q}^{\prime }\left(f\right)={∥a_{0,·}∥}_{l_p}+{\left(\sum _{j>0}{(2^{j[s+d(\frac{1}{2}-\frac{1}{p})]}{∥b_{j,·}∥}_{l_p})}^q\right)}^{1/q}<\infty ,$$

where the scaling and wavelet coefficients are defined by

$$a_{0,\mathbf{k}}={\int }_{{\mathbb{R}}^d}f\left(\mathbf{u}\right){\varphi }_{0,\mathbf{k}}\left(\mathbf{u}\right)\mathrm{d}\mathbf{u},b_{i,j,\mathbf{k}}={\int }_{{\mathbb{R}}^d}f\left(\mathbf{u}\right){\psi }_{i,j,\mathbf{k}}\left(\mathbf{u}\right)\mathrm{d}\mathbf{u},$$

and

$$\parallel b_{j,·}{\parallel }_{{\ell }_p}={\left({\displaystyle \sum _{i=1}^d}\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\left|b_{i,j,\mathbf{k}}\right|}^p\right)}^{1/p}.$$

When $q=\infty$, the summations in $(\mathrm{B}.1)$ and $(\mathrm{B}.2)$ are replaced by the essential supremum. The Besov framework subsumes many classical smoothness classes that are ubiquitous in statistics and machine learning—for instance, the Sobolev (Hilbert) space $H^s$ corresponds to ${\mathbf{B}}_{s,2,2}$, while the (non-integer) Hölder–Lipschitz–Zygmund class $C^s$ coincides with ${\mathbf{B}}_{s,\infty ,\infty }$. Additional equivalent formulations, together with their advantages in approximation theory and statistical methodology, are detailed in [29,30,31,32] and in Appendix A.

2.2. Linear Wavelets Estimator

This section opens with a concise review of the essentials of wavelet theory, following the notation of [28]. Let ${\left\{V_j\right\}}_{j\in \mathbb{Z}}$ be a multiresolution analysis of the Hilbert space $L_2\left({\mathbb{R}}^d\right)$. Denote by $\varphi$ the scaling function and by $\psi$ the associated orthogonal wavelet, both assumed to be r-regular ($r\ge 1$) and compactly supported in the hyper-cube ${[-L,L]}^d$ for some $L>0$. For every integer j and every index vector $\mathbf{k}\in {\mathbb{Z}}^d$, define

$${\varphi }_{j,\mathbf{k}}\left(\mathbf{x}\right)=2^{jd/2}\varphi \left(2^j\mathbf{x}-\mathbf{k}\right).$$

The family ${\left\{{\varphi }_{j,\mathbf{k}}\right\}}_{\mathbf{k}\in {\mathbb{Z}}^d}$ forms an orthonormal basis of $V_j$; moreover, each partial derivative of $\varphi$ up to total order r is rapidly decreasing. Specifically, for every integer $i>0$ there exists a constant $A_i>0$ such that (see [28], p. 29, Thm.2)

$$\left|\left({\partial }^{\beta }\varphi \right)\left(\mathbf{x}\right)\right|\le {\displaystyle \frac{A_i}{{\left(1+\parallel \mathbf{x}\parallel \right)}^i}},\mathrm{for}\mathrm{all}\mid \beta \mid \le r.$$

(1)

In addition, there exist precisely $2^d-1$ companion wavelets ${\psi }_1,\dots ,{\psi }_{2^d-1}$. Together they generate the collection

$$\left\{{\psi }_{i,j,\mathbf{k}}\left(\mathbf{x}\right)=2^{jd/2}{\psi }_i\left(2^j\mathbf{x}-\mathbf{k}\right)|i=1,\dots ,2^d-1,\mathbf{k}\in {\mathbb{Z}}^d\right\},$$

which constitutes an orthonormal basis of $L_2\left({\mathbb{R}}^d\right)$.

We next recall the notions of strong mixing and ergodicity and the relationship between them. Let ${\left\{X_n\right\}}_{n\in \mathbb{Z}}$ be a strictly stationary sequence, and set

$${\mathcal{A}}_n=\sigma (X_k:k\le n)$$

and

$${\mathcal{B}}_m=\sigma (X_k:k\ge m).$$

The sequence is ($\alpha$-)strongly mixing if

$$\alpha \left(n\right)=\underset{A\in {\mathcal{A}}_0,B\in {\mathcal{B}}_n}{sup}\left|\mathbb{P}(A\cap B)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)\right|⟶0(n\to \infty ).$$

It is ergodic when

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}{\displaystyle \sum _{k=0}^{n-1}}\left|\mathbb{P}\left(A\cap {\tau }^{-k}B\right)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)\right|=0,$$

where $\tau$ denotes the left-shift operator. The foregoing definition of strong mixing is stricter than the one often used for measure-preserving transformations—namely, ${lim}_{n\to \infty }\mathbb{P}(A\cap {\tau }^{-n}B)=\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)$ for all measurable sets $A,B$ [33]. While strong mixing implies ergodicity, the converse need not hold (cf. Remark 2.6, p. 50, and Proposition 2.8, p. 51, of [34]). A number of authors have argued that an ergodic, rather than a strongly mixing, dependence structure is often preferable; see, for example, the discussion and illustrative examples in [35].

Finally, let $(\mathbf{X},\mathbf{Y})$ be a random vector with $\mathbf{X}=(X_1,\dots ,X_d)\in {\mathbb{R}}^d$ and $\mathbf{Y}=(Y_1,\dots ,Y_q)\in {\mathbb{R}}^q$. Define the joint distribution function (df) of $(\mathbf{X},\mathbf{Y})$ by

$$F(\mathbf{x},\mathbf{y}):=\mathbb{P}(\mathbf{X}\le \mathbf{x},\mathbf{Y}\le \mathbf{y}),\mathbf{x}\in {\mathbb{R}}^d,\mathbf{y}\in {\mathbb{R}}^q.$$

Henceforth, for vectors ${\mathbf{v}}^{\prime }=(v_1^{\prime },\dots ,v_r^{\prime })\in {\mathbb{R}}^r$ and ${\mathbf{v}}^{″}=(v_1^{″},\dots ,v_r^{″})\in {\mathbb{R}}^r$, we write ${\mathbf{v}}^{\prime }\le {\mathbf{v}}^{″}$ if and only if $v_j^{\prime }\le v_j^{″}$ for all $j=1,\dots ,r$. We let $\mathbf{I}$ and $\mathbf{J}$ be two fixed subsets of ${\mathbb{R}}^d$ such that

$$\mathbf{I}={\displaystyle \prod _{j=1}^d}[a_j,b_j]\subset \mathbf{J}={\displaystyle \prod _{j=1}^d}[c_j,d_j]\subset {\mathbb{R}}^d,$$

with

$$-\infty <c_j<a_j<b_j<d_j<\infty \mathrm{for}\mathrm{each}j=1,\dots ,d.$$

Suppose $(\mathbf{X},\mathbf{Y})$ has a joint density function

$$f_{\mathbf{X},\mathbf{Y}}(\mathbf{x},\mathbf{y}):={\displaystyle \frac{{\partial }^{d+q}}{\partial x_1\cdots \partial x_d\partial y_1\cdots \partial y_q}}F(\mathbf{x},\mathbf{y})\mathrm{on}\mathbf{J}\times {\mathbb{R}}^q,$$

with respect to the Lebesgue measure $d\mathbf{x}d\mathbf{y}$. Denote

$$f_{\mathbf{X}}\left(\mathbf{x}\right)={\int }_{{\mathbb{R}}^q}f(\mathbf{x},\mathbf{y})d\mathbf{y},\mathbf{x}\in \mathbf{J},$$

as the marginal density of $\mathbf{X}$ on $\mathbf{J}$. Let $\rho :{\mathbb{R}}^q\to \mathbb{R}$ be a measurable function. We are chiefly interested in estimating derivatives of the form

$$\begin{array}{cc}\hfill \left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})& :={\partial }^{\beta }\left[\mathbb{E}\left[\rho \right(\mathbf{Y})\mid \mathbf{X}=\mathbf{x}]\right]f_{\mathbf{X}}\left(\mathbf{x}\right)\hfill \\ \hfill & = {\int }_{{\mathbb{R}}^q}\rho \left(\mathbf{y}\right)\left({\partial }^{\beta }f\right)(\mathbf{x},\mathbf{y})d\mathbf{y},\hfill \end{array}$$

where

$$\left({\partial }^{\beta }f\right)(\mathbf{x},\mathbf{y})={\displaystyle \frac{{\partial }^{\left|\beta \right|}f(\mathbf{x},\mathbf{y})}{\partial x_1^{{\beta }_1}\cdots \partial x_d^{{\beta }_d}}},\beta =({\beta }_1,\dots ,{\beta }_d),\left|\beta \right|={\displaystyle \sum _{i=1}^d}{\beta }_i.$$

When the regression function

$$m_{\rho }\left(\mathbf{x}\right)=\mathbb{E}[\rho \left(\mathbf{Y}\right)\mid \mathbf{X}=\mathbf{x}]$$

is sufficiently smooth, we may also consider its derivative

$${\partial }_{\mathbf{x}}^{\beta }{m}_{\rho }\left(\mathbf{x}\right)={\displaystyle \frac{{\partial }^{\left|\beta \right|}{m}_{\rho }\left(\mathbf{x}\right)}{\partial x_1^{{\beta }_1}\cdots \partial x_d^{{\beta }_d}}}.$$

Assume now that $\left({\partial }^{\beta }r\right)(\rho ;·)\in L_2\left({\mathbb{R}}^d\right)$. According to [36], for any integer $\tau$, the function $\left({\partial }^{\beta }r\right)(\rho ;·)$ admits an expansion in the subspace $V_{\tau }$ of the multiresolution analysis:

$$\left({\partial }^{\beta }r\right)\left(\mathbf{x}\right)=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{a}_{\tau ,\mathbf{k}}{\varphi }_{\tau ,\mathbf{k}}\left(\mathbf{x}\right),$$

(2)

where each wavelet coefficient $a_{\tau ,\mathbf{k}}$ can be represented (via repeated integration by parts) as

$$\begin{array}{cc}\hfill a_{\tau ,\mathbf{k}}& ={\int }_{{\mathbb{R}}^d}\left({\partial }^{\beta }r\right)(\rho ;\mathbf{u}){\varphi }_{\tau ,\mathbf{k}}\left(\mathbf{u}\right)d\mathbf{u}\hfill \\ \hfill & = {(-1)}^{\left|\beta \right|}{\int }_{{\mathbb{R}}^d}r(\rho ;\mathbf{u})\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left(\mathbf{u}\right)d\mathbf{u}.\hfill \end{array}$$

Let ${\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}(·)$ denote the $\beta$-order partial derivative of ${\varphi }_{\tau ,\mathbf{k}}(·)$. We define the linear estimator of $\left({\partial }^{\beta }r\right)(\rho ;·)$ at the resolution level $\tau =m\left(n\right)\to \infty$ (the precise divergence rate of $m\left(n\right)$ will be specified below) by

$${\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{\tau ,\mathbf{k}}{\varphi }_{\tau ,\mathbf{k}}\left(\mathbf{x}\right),$$

(3)

where ${\widehat{a}}_{\tau ,\mathbf{k}}$ is the unbiased empirical estimator of $a_{\tau ,\mathbf{k}}$:

$${\widehat{a}}_{\tau ,\mathbf{k}}={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}\rho \left({\mathbf{Y}}_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right).$$

(4)

Remark 1.

Since $\varphi (·)$ and ${\psi }_i(·)$ are bounded and compactly supported—where the support grows in a controlled manner with increasing differentiability—it follows that, for any fixed point $\mathbf{x}$, only finitely many terms in the sums over $\mathbf{k}\in {\mathbb{Z}}^d$ contribute to the value of the wavelet expansions (see [36] for further details). This implies pointwise convergence of the expansions.

3. Assumptions and Main Results

To facilitate the presentation of our main results, we introduce the following notation. Let us denote by $g(·)$ the density function of the random variable Y. Let ${\mathcal{F}}_n$ be the $\sigma$-field generated by $\{({\mathbf{X}}_i,Y_i):1\le i\le n\}$, let $f_{(\mathbf{X},Y)}^{{\mathcal{F}}_{i-1}}(·)$ and $g^{{\mathcal{F}}_{i-1}}(·)$ and be the conditional densities of $(\mathbf{X},Y)$ and Y respectively, given the $\sigma -$ field ${\mathcal{F}}_{i-1}$. Define the $\sigma -$ field

$${\mathcal{G}}_n=\sigma \left(({\mathbf{X}}_k,Y_k);\left({\mathbf{X}}_{n+1}\right):1\le k\le n\right).$$

The following assumptions are imposed throughout the paper.

(C.1) 

For every $\mathbf{x}\in \mathbf{S}$, the sequence

$${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)$$

converges to $f\left(\mathbf{x}\right)$ as $n\to \infty$, both almost surely (a.s.) and in the $L^2$ sense.

(C.2) 

Moreover, for every $\mathbf{x}\in \mathbf{S}$,

$$\underset{n\to \infty }{lim}\underset{\mathbf{x}\in {\mathbb{R}}^d}{sup}\left|{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)-f\left(\mathbf{x}\right)\right|=0,$$

again in both the almost sure and $L^2$ senses.

(C.3) 

 

(i)

Assume that the multiresolution analysis is r-regular.

(ii)

The density $\left({\partial }^{\beta }r\right)(\rho ;·)\in B_{spq}$ for some $0<s<r,1⩽p,q⩽\infty$.

(C.4) 

 

(i)

The density $f(·)\in B_{spq}$ for some $0<s<r,1⩽p,q⩽\infty$.

(ii)

The conditional density $f^{{\mathcal{F}}_{i-1}}(·)\in B_{spq}$ for some $0<s<r,1⩽p,q⩽\infty$.

(N.0) 

$\mathbb{E}\left[|\rho \left(Y_1\right){|}^{\nu }\right]<\infty$ for some $\nu >2$.

(N.1) 

For any $y\in \mathbb{R}$

$$\underset{n\to \infty }{lim}\underset{y\in \mathbb{R}}{sup}\left|{\displaystyle \frac{1}{ng\left(y\right)}}{\displaystyle \sum _{i=1}^n}{g}^{{\mathcal{F}}_{i-1}}\left(y\right)-1\right|=0,\mathrm{in} \mathrm{the}a.s.\mathrm{and}{L}^2\mathrm{sense}.$$

We may refer to [37] for further details.

(N.2) 

 

(i)

The conditional mean of $Y_i$ given the $\sigma -$ field ${\mathcal{S}}_i$ depends only on ${\mathbf{X}}_i$, i.e., for any $i\ge 1$,

$$\mathbb{E}\left[\rho \left(Y_i\right)\mid {\mathcal{G}}_{i-1}\right]=\mathbb{E}\left[\rho \left(Y_i\right)\mid {\mathbf{X}}_i\right]={\mathbf{m}}_{\rho }\left(\mathbf{x}\right).$$

(ii)

for all $\eta \le 2$,

$$\mathbb{E}\left[{\left|\rho \left(Y_i\right)\right|}^{\eta }\mid {\mathcal{G}}_{i-1}\right]=\mathbb{E}\left[{\left|\rho \left(Y_i\right)\right|}^{\eta }\mid {\mathbf{X}}_i\right]={\mathbf{m}}_{\rho ,\eta }\left(\mathbf{x}\right).$$

and the function ${\mathbf{m}}_{\rho ,\eta }\left(\mathbf{x}\right)$ is continuous at $\mathbf{x}$

(N.3) 

 

(i)

We shall suppose that there exist two known constants $C_{\mathbf{m}}>0$ such that

$$\underset{\mathbf{x}\in \mathbf{S}}{sup}{\mathbf{m}}_{\rho }\left(\mathbf{x}\right)\le C_{\mathbf{m}}.$$

(ii)

The function regression satisfies a Hölder condition, that is, there exist constants $\beta >0$ and $c_3>0$ such that, for any $(\mathbf{u},\mathbf{v})\in {\mathbb{R}}^{2d}$

$$\left|{\mathbf{m}}_{\rho }\left(\mathbf{u}\right)-{\mathbf{m}}_{\rho }\left(\mathbf{v}\right)\right|\le c_3{\parallel \mathbf{u}-\mathbf{v}\parallel }^{\beta }.$$

(iii)

The function regression $m_{\rho ,\eta }$ satisfy a Hölder condition that is, there exist constants ${\beta }_1>0$ and $c_4>0$ such that, for any $(\mathbf{u},\mathbf{v})\in {\mathbb{R}}^{2d}$

$$\left|{\mathbf{m}}_{\rho ,\eta }\left(\mathbf{u}\right)-{\mathbf{m}}_{\rho ,\eta }\left(\mathbf{v}\right)\right|\le c_4{\parallel \mathbf{u}-\mathbf{v}\parallel }^{{\beta }_1}.$$

Lemma 1.

Under condition (C.3), for $s>d/p$, we have

$$\begin{array}{cc}\underset{\mathbf{x}\in {\mathbb{R}}^d}{sup}\left|\mathbb{E}\left[{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})\right]-\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})\right|\hfill & =\underset{\mathbf{x}\in {\mathbb{R}}^d}{sup}\left|\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})-\left(P_{V_m}\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})\right)\left(\mathbf{x}\right)\right|\hfill \\ & ⩽\mathit{const}.2^{-(s-d/p)m\left(n\right)}{J}_{spq}\left(\left({\partial }^{\beta }r\right)(\rho ;·)\right).\hfill \end{array}$$

Define the kernel $K(\mathbf{u},\mathbf{v})$ by

$$K(\mathbf{u},\mathbf{v}):=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}\varphi (\mathbf{u}-\mathbf{k})\varphi (\mathbf{v}-\mathbf{k})\mathrm{and}{h}_n=2^{-m\left(n\right)}.$$

(5)

consequently

$$K^{\left(\beta \right)}(\mathbf{u},\mathbf{v}):=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}\varphi (\mathbf{u}-\mathbf{k})\left({\partial }_{\mathbf{v}}^{\beta }\varphi \right)(\mathbf{v}-\mathbf{k}).$$

(6)

where $K^{\left(\beta \right)}(\mathbf{u},\mathbf{v}):=\left({\partial }_{\mathbf{v}}^{\beta }k\right)(\mathbf{u},\mathbf{v})$ denotes the $\beta$-th partial derivative of $K(\mathbf{u},\mathbf{v})$ with respect to $\mathbf{v}$. We infer that the derivative kernel function $K^{\left(\beta \right)}(·)$ defined in (6) converges uniformly and satisfies ([28], p. 33), i.e., for $\mid \alpha \mid \le r$, $\mid \beta \mid \le r$, and for some constant $C_m>0$ with $m\ge 1$, the following inequalities holds:

$$\left|\left({\partial }_{\mathbf{u}}^{\alpha }{\partial }_{\mathbf{v}}^{\beta }K\right)(\mathbf{u},\mathbf{v})\right|\le {\displaystyle \frac{C_m}{{(1+\parallel \mathbf{v}-\mathbf{u}\parallel }_2{)}^m}},$$

where, for any $\mathbf{x}=(x_1,\dots ,x_d)\in {\mathbb{R}}^d,$

$${\parallel \mathbf{x}\parallel }_2=\sqrt{{\displaystyle \sum _{i=1}^d}{x}_i^2}.$$

For $\alpha =0$ and $m=d+\left|\beta \right|+1$, we obtain

$$|K^{\left(\beta \right)}(\mathbf{u},\mathbf{v})|\le {\displaystyle \frac{C_{d+\left|\beta \right|+1}}{{(1+\parallel \mathbf{v}-\mathbf{u}\parallel }_2{)}^{d+\left|\beta \right|+1}}}.$$

(7)

That is, for any $j\ge 1$, we have

$${\int }_{{\mathbb{R}}^d}{\left|K^{\left(\beta \right)}(\mathbf{v},\mathbf{u})\right|}^{j}d\mathbf{v}\le G_j\left(d\right).$$

The function $G_j\left(d\right)$ is defined as:

$$G_j\left(d\right)=2{\pi }^{d/2}{\displaystyle \frac{\ell \left(d\right)\ell (j+d(j-1\left)\right)}{\ell (d/2)\ell \left(\right(d+1\left)j\right)}}{C}_{d+1}^j,$$

where $\ell \left(t\right)$ represents the ell function, given by

$$\ell \left(t\right):={\int }_0^{\infty }{y}^{t-1}{e}^{-y}dy.$$

Furthermore, by assuming $\left|\alpha \right|=1$ and $m=2$, we deduce that

$$\left|{\displaystyle \frac{{\partial }_{\mathbf{u}}{\partial }_{\mathbf{v}}^{\beta }K(\mathbf{u},\mathbf{y})}{\partial u_i}}\right|\le {\displaystyle \frac{C_m}{{(1+\parallel \mathbf{u}-\mathbf{y}\parallel }_2{)}^2}}\le C_2,i=1,\dots ,d.$$

(8)

This, in turn, implies that

$$\left|K^{\beta }(\mathbf{u},\mathbf{y})-K^{\beta }(\mathbf{v},\mathbf{y})\right|\le C_2{\displaystyle \sum _{i=1}^d}|u_i-v_i|\le d^{1/2}{C}_2{\parallel \mathbf{u}-\mathbf{v}\parallel }_2.$$

(9)

By incorporating Equations (4), (3), and (6), the estimator ${\widehat{\left({\partial }^{\beta }r\right)}}_n\left(\mathbf{x}\right)$ can be expressed in an extended kernel estimation framework as follows:

$${\widehat{\left({\partial }^{\beta }r\right)}}_n\left(\mathbf{x}\right)={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\rho \left(Y_i\right)K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right),\mathrm{where}{h}_n=2^{-m\left(n\right)}.$$

(10)

We refer the reader to [24] for further details.

Theorem 1.

Under the stated assumptions (C.1), (C.4)(i), (N.0) and (N.2)(i), let r be an element of the Besov space $B_{s,p,q}$ with $s>1/p$, and $1\le p,q\le \infty$. In this setting, the linear wavelet estimator ${\widehat{\left({\partial }^{\beta }r\right)}}_n$ satisfies

$$\mathbb{E}{\parallel {\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})\parallel }_2^2=O(n^{-\frac{2(s-|\beta |-d/4)}{2s+1}}).$$

Theorem 2.

Assume that assumptions (C.2), (N.1), (N.2), (N.3)(i) and

$$m\left(n\right)=m\to \infty and{\displaystyle \frac{2^{dm\left(n\right)}logn}{n}}\to 0asn\to \infty .$$

Define $T_n$ and assume that for $m\left(n\right)\to \infty$ slowly enough such that

$${\left({\displaystyle \frac{2^{m\left(n\right)d}logn}{n}}\right)}^{1/2}{T}_n={\left({\displaystyle \frac{logn}{n{h}_n^d}}\right)}^{1/2}{T}_n\to 0,\mathrm{as}n\to \infty .$$

(11)

For every compact subset $D\subset {\mathbb{R}}^d$, under assumptions (C.1) and (C.3), we have almost surely,

$$\underset{\mathbf{x}\in D}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-\mathbb{E}[{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})]\right|=O\left({\left({\displaystyle \frac{logn}{n{h}_n^{d+2\left|\beta \right|}}}\right)}^{1/2}\right)+O\left(h_n^{-1/2}\right).$$

Remark 2.

The factor $2^{-m\left(n\right)}$ that appears in the preceding theorems corresponds directly to the bandwidth $h_n$ in Parzen–Rosenblatt kernel density estimation. In practice, however, selecting the multiresolution level $m\left(n\right)$ within the wavelet framework is typically much simpler than choosing $h_n$. Because only a small, discrete set of candidate levels—usually three or four—needs to be examined, the procedure remains both conceptually transparent and computationally inexpensive. Adaptive choice of $m\left(n\right)$ most commonly relies on Stein’s unbiased risk estimate, the classical rule of thumb, and cross-validation. Comprehensive treatments of these methods, as well as their use in establishing asymptotically optimal data-driven bandwidth selection rules, are provided by [38] and [21]. For the univariate setting ($d=1$), the cross-validation criterion at resolution level j is given by

$$CV\left(j\right)=\sum _k\left[{\displaystyle \frac{2}{n(n-1)}}{\displaystyle \sum _{i=1}^n}{\left({\varphi }_{j,k}\left(X_i\right)\right)}^2-{\displaystyle \frac{n+1}{n^2(n-1)}}{\left({\displaystyle \sum _{i=1}^n}{\varphi }_{j,k}\left(X_i\right)\right)}^2\right].$$

The statistic $CV\left(j\right)$ depends solely on the observations $X_1,\dots ,X_n$ and the resolution index j. The optimal level is selected by

$$j_0=arg\underset{j}{min}CV\left(j\right).$$

Remark 3.

Kernel estimators suffer a well-documented decline in accuracy as the dimension of the covariate space grows—a manifestation of the curse of dimensionality [39]. In high dimensions, the volume of a local neighbourhood expands so rapidly that an impractically large sample is required to collect even a modest number of observations inside it. Unless the sample size is very large, one must therefore adopt bandwidths so wide that the resulting estimate is no longer truly local. A thorough exposition, complete with numerical illustrations, appears in [40], while more recent analyses are given in [41,42]. Although penalised splines enjoy considerable empirical popularity, their asymptotic behaviour is still not fully resolved even in classical non-parametric settings, and only a handful of theoretical investigations address the issue. In addition, most functional-regression techniques minimise an $L_2$ criterion and thus remain sensitive to outliers. Promising, if less conventional, alternatives include methods based on delta sequences and wavelet representations.

3.1. Asymptotic Normality Results

This section establishes a central limit theorem for the estimator defined in Equation (3). This problem was considered in several papers. In [10,43], the authors analyzed a non-parametric regression model with repeated measurements corrupted by mixing errors and, under mild moment and mixing-coefficient assumptions, established the r-th mean consistency, almost-sure consistency, complete consistency, optimal almost-sure convergence rates, and asymptotic normality of the associated wavelet estimator. For deconvolution density estimation with moderately ill-posed noise, ref. [44] proves the asymptotic normality of wavelet estimators when the target density lies in suitable Besov spaces. The study in [45] revisits non-parametric regression, deriving asymptotic normality of the wavelet estimator when the errors constitute an asymptotically negatively associated sequence. In a censored-data setting, ref. [46] treats density estimation where survival and censoring times form a stationary $\alpha$-mixing sequence, again demonstrating asymptotic normality of the wavelet estimator. A fixed-design non-parametric regression model driven by strongly mixing errors is considered in [47], which likewise confirms asymptotic normality for the wavelet estimator of the regression function. Wavelet methods for linear processes—Gaussian or otherwise—with long, short, or negative memory are examined in [48]; both the log-regression and Whittle wavelet estimators are shown to be asymptotically normal, and explicit expressions for the limiting variance are derived via a general result for the suitably centred and normalised scalogram. Finally, ref. [49] proposes a wavelet estimator for fixed-design non-parametric regression with strictly stationary, associated errors, proving pointwise weak consistency and uniform asymptotic normality. For additional treatments of asymptotic normality of wavelet estimators in alternative frameworks, see [10,50,51]. Our results are obtained under mild regularity conditions on the estimator and only minimal bandwidth assumptions. We write

$$Z_n\stackrel{\mathcal{D}}{\to }\mathcal{N}\left(0,{\Sigma }^2\left(\mathbf{x}\right)\right)$$

to indicate that the sequence of random variables ${\left(Z_n\right)}_{n\ge 1}$ converges in distribution to a mean-zero normal distribution with covariance matrix ${\Sigma }^2\left(\mathbf{x}\right)$.

Theorem 3.

Assume that the hypotheses (C.1)–(C.4), (N.0), (N.2)(ii) and (N.3)(iii) are satisfied. Additionally, suppose that

$$n{h}_n^{d+2\left|\beta \right|+2\delta }\to 0\mathrm{as}n\to \infty .$$

(12)

Then, the following convergence holds:

$$\sqrt{n{h}_n^{d+2\left|\beta \right|}}\left({\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})\right)\stackrel{\mathcal{D}}{\to }\mathcal{N}(0,{\Sigma }_{\left(\beta \right)}^2\left(\mathbf{x}\right)),$$

where

$${\Sigma }_{\left(\beta \right)}^2\left(\mathbf{x}\right):={\mathbf{m}}_2(\rho ,\mathbf{x})f_{\mathbf{X}}\left(\mathbf{x}\right){\int }_{{\mathbb{R}}^d}{\left(K^{\left(\beta \right)}\right)}^2(\mathbf{0},\mathbf{u})d\mathbf{u},$$

and

$${\mathbf{m}}_2(\rho ,\mathbf{x})=\mathbb{E}\left[{\rho }^2\left(\mathbf{Y}\right)\mid \mathbf{X}=\mathbf{x}\right]={\displaystyle \frac{1}{f_{\mathbf{X}}\left(\mathbf{x}\right)}}{\int }_{{\mathbb{R}}^q}{\rho }^2\left(\mathbf{y}\right)f_{\mathbf{X},\mathbf{Y}}(\mathbf{x},\mathbf{y})d\mathbf{y}.$$

The proof of Theorem 3 is detailed in Section 7.

Remark 4.

Invoking Theorem 1, we obtain a refined bound for the mean-squared error of the wavelet-based estimator of multivariate density derivatives. Under the usual smoothness and moment assumptions,

$$\mathbb{E}{∥{\widehat{\left({\partial }^{\beta }f\right)}}_n-{\partial }^{\beta }f∥}_2^2=O(n^{-{\displaystyle \frac{2(s-|\beta |)}{2s+1}}}).$$

A parallel analysis, this time relying on Theorem 3, yields an asymptotic normality result. Specifically, for any fixed $\mathbf{x}\in {\mathbb{R}}^d$,

$$\sqrt{n{h}_n^{d+2\left|\beta \right|}}\left({\widehat{\left({\partial }^{\beta }f\right)}}_n\left(\mathbf{x}\right)-{\partial }^{\beta }f\left(\mathbf{x}\right)\right)\stackrel{\mathcal{D}}{\to }\mathcal{N}(0,{\tilde{\Sigma }}_{\left(\beta \right)}^2\left(\mathbf{x}\right)),$$

with asymptotic variance

$${\tilde{\Sigma }}_{\left(\beta \right)}^2\left(\mathbf{x}\right)=f_{\mathbf{X}}\left(\mathbf{x}\right){\int }_{{\mathbb{R}}^d}{\left(K^{\left(\beta \right)}\right)}^2(\mathbf{0},\mathbf{u})d\mathbf{u}.$$

Both results are consistent with the theoretical developments in Theorem 4.8 of [52], further demonstrating the efficacy of wavelet methods for non-parametric estimation in multivariate contexts.

Remark 5.

A noteworthy case of $\left({\partial }^{\beta }r\right)(\rho ;·)$ arises when $\rho \left(\mathbf{y}\right)={\mathbb{1}}_{\{\mathbf{y}\le \mathbf{t}\}}$. In this setting, from (3) we obtain

$${\widehat{\left({\partial }^{\beta }r\right)}}_n\left({\mathbb{1}}_{\{·\le \mathbf{t}\}};\mathbf{x}\right)=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\breve{a}}_{\tau ,\mathbf{k}}{\varphi }_{\tau ,\mathbf{k}}\left(\mathbf{x}\right),$$

where ${\breve{a}}_{\tau ,\mathbf{k}}$ is the unbiased estimator given by

$${\breve{a}}_{\tau ,\mathbf{k}}={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}{\mathbb{1}}_{\{{\mathbf{Y}}_i\le \mathbf{t}\}}\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right).$$

Since this is a specific instance of the estimator in (3), Theorem 3 implies that

$$\sqrt{n{h}_n^{d+2\beta }}\left({\widehat{\left({\partial }^{\beta }r\right)}}_n\left({\mathbb{1}}_{\{·\le \mathbf{t}\}};\mathbf{x}\right)-{\partial }^{\beta }r\left({\mathbb{1}}_{\{·\le \mathbf{t}\}};\mathbf{x}\right)\right)\stackrel{\mathcal{D}}{\to }\mathcal{N}(0,{\breve{\Sigma }}^2\left(\mathbf{x}\right)),$$

where

$${\breve{\Sigma }}^2\left(\mathbf{x}\right):=m_2\left({\mathbb{1}}_{\{·\le \mathbf{t}\}},\mathbf{x}\right)f_{\mathbf{X}}\left(\mathbf{x}\right){\int }_{{\mathbb{R}}^d}{\left(K^{\left(\beta \right)}\right)}^2\left(\mathbf{0},\mathbf{u}\right)d\mathbf{u}.$$

This result is pivotal for determining the asymptotic behavior of wavelet estimators derived from the conditional distribution.

3.2. Confidence Interval

The asymptotic variance ${\Sigma }_{\left(\beta \right)}^2\left(\mathbf{x}\right)$ that features in the central limit theorem depends on two unknown components—the conditional variance of Y given $\mathbf{X}$ and the density $f_{\mathbf{X}}(·)$ of $\mathbf{X}$—and therefore must be estimated in practice. We obtain consistent estimates by employing a bounded, compactly supported wavelet basis drawn from the literature, such as the widely used Daubechies family [36]. Using a sufficiently large multiresolution level ${\tau }_n$ and an adaptively chosen initial level $j_0$, we recover the requisite nuisance parameters via wavelet-based estimation and subsequently insert them through a plug-in scheme. Using the estimators (3) we establish ${\Sigma }_{n,\left(\beta \right)}^2\left(\mathbf{x}\right)$ a consistent estimate of the variance ${\Sigma }_{\left(\beta \right)}^2\left(\mathbf{x}\right)$ given by

$${\Sigma }_{n,\left(\beta \right)}^2\left(\mathbf{x}\right):={\mathbf{m}}_{n,2}(\rho ,\mathbf{x})f_{n,\mathbf{X}}\left(\mathbf{x}\right){\int }_{{\mathbb{R}}^d}{\left(K^{\left(\beta \right)}\right)}^2(\mathbf{0},\mathbf{u})d\mathbf{u},$$

(13)

where $f_{n,\mathbf{X}}(·)$ is the wavelet estimator of $f_{\mathbf{X}}(·)$ and

$${\mathbf{m}}_{n,2}(\rho ,\mathbf{x})=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{j_0\mathbf{k}}^{″}{\varphi }_{j_0,\mathbf{k}}\left(\mathbf{x}\right)+{\displaystyle \sum _{j=j_0}^{\tau }}{\displaystyle \sum _{i=1}^N}\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{b}}_{ij\mathbf{k}}^{″}{\psi }_{i,j,\mathbf{k}}\left(\mathbf{x}\right).$$

(14)

The considered coefficients estimators are given, respectively, for any $j_0\le \tau$, by

$${\widehat{a}}_{\tau ,\mathbf{k}}^{″}={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\rho }^2\left(Y_i\right){\varphi }_{\tau ,\mathbf{k}}\left({\mathbf{X}}_i\right),$$

and

$${\widehat{b}}_{ij\mathbf{k}}^{″}={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\rho }^2\left(Y_i\right){\psi }_{i,j,\mathbf{k}}\left({\mathbf{X}}_i\right).$$

The approximate confidence intervals of $\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})$ can be obtained as

$$\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})\in \left[{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})\pm c_{\alpha }{\displaystyle \frac{{\Sigma }_{n,\left(\beta \right)}^2\left(\mathbf{x}\right)}{\sqrt{n{h}_n^{d+2\left|\beta \right|}}}}\right],$$

where $c_{\alpha }$ denotes the $(1-{\displaystyle \frac{\alpha }{2}})-$ quantile of the standard normal condition.

4. Application to the Regression Derivatives

In this section, we will follow the same notation as in [20]. We will consider especially the conditional expectation of $\rho \left(Y\right)$ given $X=x$, for $p=q=1$. Recall that

$$\begin{array}{cc}\hfill \mathbf{m}(x,\rho )& =\mathbb{E}(\rho \left(Y\right)\mid X=x)={\displaystyle \frac{1}{f_X\left(x\right)}}{\int }_{\mathbb{R}}\rho \left(y\right)f_{X,Y}(x,y)dy\hfill \\ \hfill & = {\displaystyle \frac{{\displaystyle r(\rho ,x)}}{{\displaystyle f_X\left(x\right)}}}.\hfill \end{array}$$

Recall the following derivatives

$${\mathbf{m}}_{\rho }^{\prime }\left(x\right)={\displaystyle \frac{{\displaystyle r^{\prime }(\rho ,x)}}{{\displaystyle f_X\left(x\right)}}}-{\displaystyle \frac{{\displaystyle r(\rho ,x)f_X^{\prime }\left(x\right)}}{{\displaystyle f_X^2\left(x\right)}}},$$

(15)

and

$$\begin{array}{ccc}{\mathbf{m}}_{\rho }^{″}\left(x\right)\hfill & =\hfill & {\displaystyle \frac{{\displaystyle r^{″}(\rho ,x)}}{{\displaystyle f_X\left(x\right)}}}-{\displaystyle \frac{{\displaystyle 2r^{\prime }(\psi ,x)f_X^{\prime }\left(x\right)}}{{\displaystyle f_X^2\left(x\right)}}}\hfill \\ & & +{\displaystyle \frac{{\displaystyle r(\rho ,x)\{2{\left(f_X^{\prime }\left(x\right)\right)}^2-f_X\left(x\right)f_X^{″}\left(x\right)\}}}{{\displaystyle f_X^3\left(x\right)}}}.\hfill \end{array}$$

(16)

In order to estimate the derivatives of ${\mathbf{m}}_{\rho }^{\prime }\left(x\right)$ in (15) and ${\mathbf{m}}_{\rho }^{″}\left(x\right)$ in (16) by replacing $f_X(·),$$f_X^{\prime }(·)$, $f_X^{″}(·)$, $r(\rho ;·)$, $r^{\prime }(\rho ;·)$ and $r^{″}(\rho ;·)$ by $f_{X;n}(·,h_n)$, $f_{X;n}^{\prime }(·,h_n)$, $f_{X;n}^{″}(·,h_n)$, $r_n(\rho ;·,h_n)$, $r_n^{\prime }(\rho ;·,h_n)$ and $r_n^{″}(\rho ;·,h_n)$. We so define ${\mathbf{m}}_{\rho ,n}^{\prime }(x;h_n)$ and ${\mathbf{m}}_{\rho ,n}^{″}(x;h_n)$ when $f_{X;n}(x,h_n)\ne 0$. The definition of ${\mathbf{m}}_{\rho ,n}^{\prime }(x;h_n)$ and $m_{\rho ,n}^{″}(x;h_n)$ is completed by setting ${\mathbf{m}}_{\rho ,n}^{\prime }(x;h_n)={\mathbf{m}}_{\rho ,n}^{″}(x;h_n)=0$ when $f_{X;n}(x,h_n)=0$.

The following theorem is more or less a straightforward consequence of Theorem 2.

Corollary 1.

Under the assumptions of Theorem 2, we have

$$\begin{array}{c}\underset{x\in J}{sup}\left|{\mathbf{m}}_{\rho ,n}^{\prime }(x;h_n)-{\mathbf{m}}_{\rho }^{\prime }\left(x\right)\right|=o_{\mathbb{P}}\left(1\right),\hfill \\ \underset{x\in J}{sup}\left|{\mathbf{m}}_{\rho ,n}^{″}(x;h_n)-{\mathbf{m}}_{\rho }^{″}\left(x\right)\right|=o_{\mathbb{P}}\left(1\right).\hfill \end{array}$$

5. Mode Regression

Now the location $\theta$ and size ${\mathbf{m}}_{\rho }\left(\theta \right)$ of the mode of ${\mathbf{m}}_{\rho }(·)$ are estimated from the respective functionals ${\widehat{\theta }}_n$ and ${\widehat{m}}_{\rho ,n}\left({\widehat{\theta }}_n\right)$ pertaining to ${\widehat{m}}_{\rho ,n}(·)$, i.e., ${\widehat{\theta }}_n$ is chosen through the equation

$$\begin{array}{c}{\widehat{m}}_{\rho ,n}\left({\widehat{\theta }}_n\right)=\underset{x\in I}{max}{\widehat{m}}_{\rho ,n}\left(x\right).\hfill \end{array}$$

(17)

Then, a natural estimator of $\theta$ is

$${\widehat{\theta }}_n:=arg\underset{x\in I}{max}{\widehat{m}}_{\rho ,n}\left(x\right).$$

(18)

Note that the estimate ${\widehat{\theta }}_n$ is not necessarily unique and our results are valid for any chosen value satisfying (18). We point out that we can specify our choice by taking

$$\begin{array}{c}{\widehat{\theta }}_n=inf\left\{y\in \mathbb{R}\mathrm{such} \mathrm{that}{\widehat{m}}_{\rho ,n}\left(y\right)=\underset{x\in \mathbb{R}}{sup}{\widehat{m}}_{\rho ,n}\left(x\right)\right\}.\hfill \end{array}$$

It is known that kernel estimators tend to produce some additional and superfluous modality. However, this has no bearing on the asymptotic theory; our results are valid for any choice of ${\widehat{\theta }}_n$ satisfying (18). Following the paper [53], we have the convergence result given in the following corollary.

Corollary 2.

Under the assumptions of Theorem 2, we have

$$\begin{array}{c}\left|{\widehat{\theta }}_n-\theta \right|=o_{\mathbb{P}}\left(1\right).\hfill \end{array}$$

Remark 6.

Using weak-convergence techniques in the space of continuous functions, ref. [54] strengthened Parzen’s distributional result under broadly weaker assumptions. He showed that an appropriately rescaled kernel-density estimator converges weakly to a randomly translated parabola that passes through the origin and possesses a fixed second derivative. In [55], the key insight is that, on a shrinking neighbourhood around the mode, the estimator f converges weakly—under very general conditions—to a Gaussian process whose mean function is a parabola through the origin. This mean depends on the kernel moments and on the unknown density and its derivatives evaluated at the mode. More precisely, define

$$Z_n\left(t\right)=h_n^{-2}\left[{\widehat{f}}_n(\breve{\theta }+h_{n}t)-{\widehat{f}}_n\left(\breve{\theta }\right)\right],t\in \mathbb{R},$$

where $\breve{\theta }$ denotes the mode of f. The process $Z_n$ is thus a normalised version of the estimator in an interval centred at the mode. The limiting Gaussian process $Z\left(t\right)$ is given by

$$\begin{array}{c}Z\left(t\right)=Y_{\alpha }\left(t\right)+{(-1)}^{p+1}{\displaystyle \frac{d}{cp!}}{f}^{(p+1)}\left(\breve{\theta }\right)B_{p}t+\frac{1}{2}{f}^{\left(2\right)}\left(\breve{\theta }\right)t^2,t\in \mathbb{R},\hfill \end{array}$$

where $c>0$, $d\ge 0$, $p\ge 2$ is fixed,

$$B_p=\int x^{p}K\left(x\right)dx,$$

and $Y_{\alpha }$ is a mean-zero Gaussian process with covariance

$$R(s,t)={\displaystyle \frac{f\left(\breve{\theta }\right)}{c^{2+\alpha }}}{V}_{\alpha }(s/t)s{t}^{1-\alpha },0\le \alpha \le 1,\left|s\right|\le \left|t\right|.$$

The function $V_{\alpha }$ is defined by

$$V_{\alpha }\left(\gamma \right)=\underset{\delta \to 0}{lim}{\delta }^{\alpha }\int \left[\frac{K(\gamma \delta -x)-K(-x)}{\gamma \delta }\right]\left[\frac{K(\delta -x)-K(-x)}{\delta }\right]dx.$$

Ref. [55] establishes weak convergence of $\left\{Z_n\right\}$ to Z for independent data. A central step in his Theorem 2.1 is to express $Z_n\left(t\right)$ as an average of i.i.d. variables, facilitating covariance computations and convergence of finite-dimensional distributions. Tightness is obtained via Theorem 15.7 of [56], which reduces the argument to evaluating the variance of independent variables (relation 2.9). The present study tackles the same problem via alternative methods, extending the framework of [18] to censored, dependent observations. Adapting Eddy’s approach to this setting would be an attractive direction for future work. Such an extension, however, requires new probabilistic results analogous to those in [55] but tailored to dependent—e.g., mixing—samples, a significant undertaking that we leave for subsequent research.

Remark 7.

We note that, when $\left|\mathbf{s}\right|\ge 2$, $m_{\rho ,n}^{\left(\right|\mathbf{s}\left|\right)}(x,h_n)=D^{\left|\mathbf{s}\right|}\left(m_{\rho ,n}(x,h_n)\right)$ may be obtained likewise through the usual Leibniz expansion of derivatives of products given by

$$m_{\rho ,n}^{\left(\right|\mathbf{s}\left|\right)}(x,h_n)={\displaystyle \sum _{\left|\mathbf{j}\right|=0}^{\left|\mathbf{s}\right|}}{C}_{\left|\mathbf{s}\right|}^{\left|\mathbf{j}\right|}{r}_n^{\left(\right|\mathbf{j}\left|\right)}(\rho ;x,h_n){\left\{f_{X;n}^{-1}(x,h_n)\right\}}^{\left(\right|\mathbf{s}|-|\mathbf{j}\left|\right)},f_{X;n}(x,h_n)\ne 0.$$

Remark 8.

It is well known that the accuracy of kernel estimators deteriorates as the dimension of the predictor space grows. This phenomenon—popularly termed the curse of dimensionality [39]—stems from the fact that, in high dimensions, an impractically large number of observations is needed within any local neighborhood to secure a dependable estimate. When the available sample size is modest, practitioners are forced to adopt a bandwidth so wide that the very notion of “local” averaging effectively collapses. A thorough treatment of these difficulties, complemented by illustrative numerical experiments, can be found in [40]; see also the more recent contributions [41,42,57] for additional insights. Despite their popularity, penalized splines remain theoretically under-explored: rigorous results describing their asymptotic behaviour are sparse even in classical non-parametric settings. Moreover, a large proportion of functional-regression techniques are predicated on minimising the $L_2$ loss, rendering them particularly susceptible to outlying observations. More robust alternatives—such as methods based on delta sequences or on wavelet decompositions—offer promising directions for mitigating this sensitivity.

Remark 9.

Let us recall some integral functions of the density function

$$T_1\left(F\right)={\int }_{\mathbb{R}}{\left(f^{\prime }\left(x\right)\right)}^2\mathrm{d}x,T_2\left(F\right)=-{\int }_{\mathbb{R}}a\left(F\left(x\right)\right)f^{\prime }\left(x\right)\mathrm{d}x$$

and

$$T_3\left(F\right)={\int }_{\mathbb{R}}{\left(f\left(x\right)\right)}^2\mathrm{d}x.$$

Notice that the functional $T_3\left(F\right)$ is a special case of $T_2\left(F\right)$. The functionals $T_1\left(F\right)$ and $T_3\left(F\right)$ appear in plug-in data-driven bandwidth selection procedures in density estimation (refer to [58] and the references therein) and the functional $T_2\left(F\right)$ arises as part of the variance in non-parametric location and regression estimation based on linear rank statistics (refer especially to [59]. Consider the following general class of integral functionals of the density:

$$T\left(F\right)={\int }_{\mathbb{R}}\rho \left(x,F\left(x\right),F^{\left(1\right)}\left(x\right),\dots ,F^{\left(r\right)}\left(x\right)\right)\mathrm{d}F\left(x\right),$$

where F is a cumulative distribution function on $\mathbb{R}$ with $r\ge 1$ derivatives $F^{\left(m\right)}$, refer to the [60,61] for more details. One can estimate $T\left(F\right)$ the plug-in methods making use of the wavelet estimate of the density function and its derivatives. The proof of such a statement, however, should require a different methodology than that used in the present paper, and we leave this problem open for future research.

Remark 10.

The nonlinear thresholding framework furnishes an alternative class of estimators for the unknown density $f_{\mathbf{X}}\left(\mathbf{x}\right)$. Specifically,

$${\widehat{f}}_{\mathbf{X}}\left(\mathbf{x}\right)=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{j_0,\mathbf{k}}{\varphi }_{j_0,\mathbf{k}}\left(\mathbf{x}\right)+{\displaystyle \sum _{j=j_0}^{\tau }}{\displaystyle \sum _{i=1}^N}\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{b}}_{ij\mathbf{k}}\mathbf{1}\left\{\left|{\widehat{b}}_{ij\mathbf{k}}\right|>{\delta }_{j,n}\right\}{\psi }_{i,j,\mathbf{k}}\left(\mathbf{x}\right),$$

(19)

where ${\delta }_{j,n}$ denotes an appropriately selected threshold. In the univariate case ($d=1$), this estimator was first proposed by [29]. Extending its theoretical guarantees and empirical performance to higher-dimensional settings remains an intriguing direction for future research.

Remark 11.

The sup-norm convergence rates proved in our theorems coincide with the optimal rates reported by [62,63,64]. The exact logarithmic factors depend on the resolution level, which is itself governed by the smoothness index s of the target function f in the Besov space $B_{s,p,q}$. Such a dependence on s is a hallmark of non-parametric estimation and is widely documented in the literature. By allowing f to possess general Besov regularity, our analysis relaxes the customary requirement of an integer-order derivative that underpins classical convolution-kernel methods, even though s is typically unknown in practice.

Remark 12.

To demonstrate that the assumptions stated in [65] are attainable, we provide three illustrative examples in which they are fulfilled:

• Long-memory discrete-time processes: Let ${\left({ϵ}_t\right)}_{t\in \mathbb{Z}}$ be white noise with variance ${\sigma }^2$; denote the identity and back-shift operators by I and B, respectively. According to [66] (Theorem 1, p. 55), the k-factor Gegenbauer process satisfies

  $$\prod _{i\le i\le k}{\left(I-2v_{i}B+B^2\right)}^{d_i}{X}_t={ϵ}_t,$$

  with $0<d_i<1/2$ when $|{\nu }_i|<1$ and $0<d_i<1/4$ when $|{\nu }_i|=1$ for $i=1,\dots ,k$. This specification yields a stationary, causal, invertible series that exhibits long-range dependence. Moreover, it admits the moving-average representation

  $$X_t=\sum _{j\ge 0}{\psi }_j(d,v){ϵ}_{t-j},$$

  and the condition

  $${\displaystyle \sum _{j=0}^{\infty }}{\psi }_j^2(d,v)<\infty$$

  guarantees asymptotic stability. Nevertheless, ref. [67] shows that if ${\left({ϵ}_t\right)}_{t\in \mathbb{Z}}$ is Gaussian, the process is not strongly mixing. Even so, the moving-average form secures stationarity, Gaussianity, and ergodicity, clarifying the subtle influence of mixing conditions and emphasizing the interpretive value of the moving-average representation for long-memory dynamics.
• Stationary solution of a linear Markov $\mathrm{AR}\left(1\right)$ process: Consider

  $$X_i={\displaystyle \frac{1}{2}}{X}_{i-1}+{ϵ}_i,$$

  where $\left({ϵ}_i\right)$ are independent symmetric Bernoulli variables taking the values $-1$ and 1. As shown in [68], this model is not α-mixing because of its dependence structure. It nevertheless remains stationary, Markov, and ergodic, illustrating that strong mixing is not necessary for either Markovianity or ergodicity—a point of direct relevance to statistical inference for time series and functional data.
• A stationary process with an $\mathrm{AR}\left(1\right)$ representation: Let $\left(u_i\right)$ be an independent and identically distributed sequence uniformly distributed on $\{1,\dots ,9\}$, and define

  $$X_t:={\displaystyle \sum _{i=0}^{\infty }}{10}^{-i-1}{u}_{t-i},$$

  so that $u_t,u_{t-1},\dots$ constitute the decimal expansion of $X_t$. The series is stationary and can be written in $\mathrm{AR}\left(1\right)$ form:

  $$X_t={\displaystyle \frac{1}{10}}{X}_{t-1}+{\displaystyle \frac{1}{10}}{u}_t={\displaystyle \frac{1}{10}}{X}_{t-1}+{\displaystyle \frac{1}{2}}+{ϵ}_t,$$

  where

  $${ϵ}_t={\displaystyle \frac{1}{10}}{u}_t-{\displaystyle \frac{1}{2}}$$

  is strong white noise. Although it fails the α-mixing criterion [69] (Example A.3, p. 349), the process is ergodic. This confirms that ergodicity may persist even in the absence of strong mixing, underscoring its suitability for non-parametric functional data analysis.

Remark 13.

The paper already includes several well-known examples from the literature of processes that are ergodic yet not mixing. Broadening this to encompass more general linear Markov models—such as ARMA$(p,q)$ or ARIMA$(p,d,q)$ with $q\ne 0$—would certainly be of considerable interest. Nonetheless, pursuing this extension would necessitate a meticulous and comprehensive study to rigorously delineate the circumstances under which such processes remain ergodic while failing to satisfy mixing conditions.

6. Concluding Remarks

This study tackles the estimation of partial derivatives of multivariate regression functions. We introduce a family of non-parametric estimators based on linear wavelet methods and provide a rigorous theoretical analysis. Specifically, we establish strong uniform consistency on compact subsets of ${\mathbb{R}}^d$ and derive the corresponding convergence rates. We also prove the estimators’ asymptotic normality and extend these results to ergodic processes, thereby broadening the scope of existing theory.

A key open problem is how to select smoothing parameters optimally so as to minimise the mean-squared error; addressing this question will be the focus of future work. Other promising directions include extending the methodology to functional ergodic data—a task that presents substantial mathematical challenges—and adapting it to settings with incomplete observations, such as data missing at random or subject to various censoring mechanisms, particularly in spatially dependent contexts.

Further progress could come from relaxing the stationarity assumption to accommodate locally stationary processes and from developing comparable results for stationary continuous-time models. Both extensions would require fundamentally different analytical tools.

Finally, extensive numerical experiments on simulated and real datasets would enhance the practical relevance of our procedures, and the development of weighted bootstrap techniques—building on recent contributions such as [70,71]—offers another fruitful avenue for investigation.

7. Proofs

We derive an upper bound for the partial sums of unbounded martingale differences—a key ingredient in analyzing the asymptotic properties of the wavelet estimator based on strictly stationary, ergodic observations. Throughout the paper, C denotes a generic positive constant that may change from one occurrence to the next. The necessary inequality is presented in the ensuing lemmas.

Lemma 2.

(Burkholder-Rosenthal inequality) Following Notation 1 in [72].

Let ${\left(X_i\right)}_{i\ge 1}$ be a stationary martingale adapted to the filtration ${\left({\mathcal{F}}_i\right)}_{i\ge 1}$, define ${\left(d_i\right)}_{i\ge 1}$ is the sequence of martingale differences adapted to ${\left({\mathcal{F}}_i\right)}_{i\ge 1}$ and

$$S_n={\displaystyle \sum _{i=1}^n}{d}_i,$$

then for any positive integer n,

$$\parallel \underset{1\le j\le n}{max}|S_j{\left|\right)\parallel }_p\ll n^{1/p}{\parallel d_1\parallel }_p+{∥{\displaystyle \sum _{k=1}^n}\mathbb{E}(d_k^2/{\mathcal{F}}_{k-1})∥}_{p/2}^{1/2},for anyp\ge 2;$$

(20)

where as usual the norm

$${\parallel ·\parallel }_p={\left(\mathbb{E}\left[\right|·{|}^p]\right)}^{1/p}.$$

Lemma 3.

Let ${\left(Z_n\right)}_{n\ge 1}$ be a sequence of real martingale differences with respect to the sequence of $\sigma -$ fields ${({\mathcal{F}}_n=\sigma (Z_1,\dots ,Z_n))}_{n\ge 1}$, where is the σ-field generated by the random variables $Z_1,\dots ,Z_n$. Set

$$S_n={\displaystyle \sum _{i=1}^n}{Z}_i.$$

For any $p\ge 2$ and any $n\ge 1$, assume that there exists some nonnegative constants C and $d_n$ such that

$$\mathbb{E}\left[Z_n^p|{\mathcal{F}}_{n-1}\right]\le C^{p-1}p!d_n^2,almost surely.$$

Then, for any $ϵ>0$, we have

$$\mathbb{P}\left(|S_n|>ϵ\right)\le 2exp\left\{-{\displaystyle \frac{{ϵ}^2}{2(D_n+Cϵ)}}\right\},$$

where

$$D_n={\displaystyle \sum _{i=1}^n}{d}_i^2.$$

Proof of Lemma 3.

The proof follows as a particular case of Theorem 8.2.2 due to [73]. □

To prove Theorem 1, we utilize the following two lemmas. Define the conditional expectation of $a_{\tau ,\mathbf{k}}$

$${\tilde{a}}_{\tau ,\mathbf{k}}={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\rho \left({\mathbf{Y}}_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)|{\mathcal{F}}_{i-1}\right].$$

(21)

Lemma 4.

For any $k\in {\mathbb{Z}}^d$, under the assumptions (C.1) and (N.2)(i), the following holds

$${\tilde{a}}_{\tau ,\mathbf{k}}=a_{\tau ,\mathbf{k}},\mathrm{as}n\to \infty .$$

(22)

Proof of Lemma 4.

Observe that by invoking assumptions (N.2)(i) and (C.1), and the fact that ${\mathcal{F}}_{i-1}\subset {\mathcal{G}}_{i-1}$, we can further expand ${\tilde{a}}_{\tau ,\mathbf{k}}$ as follows:

$$\begin{array}{cc}\hfill {\tilde{a}}_{\tau ,\mathbf{k}}& ={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\mathbb{E}\left[\rho \left({\mathbf{Y}}_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)|{\mathcal{G}}_{i-1}\right]|{\mathcal{F}}_{i-1}\right]\hfill \\ \hfill & ={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\mathbb{E}\left[\rho \left({\mathbf{Y}}_i\right)|{\mathbf{X}}_i\right]\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ \hfill & ={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}{\int }_{{\mathbb{R}}^d}{m}_{\rho }\left(\mathbf{x}\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left(\mathbf{x}\right)f^{{\mathcal{F}}_{j-1}}\left(\mathbf{x}\right)d\mathbf{x}\hfill \\ \hfill & ={(-1)}^{\left|\beta \right|}{\int }_{{\mathbb{R}}^d}{m}_{\rho }\left(\mathbf{x}\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left(\mathbf{x}\right)\left({\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\right)d\mathbf{x}\hfill \\ \hfill & ={(-1)}^{\left|\beta \right|}{\int }_{{\mathbb{R}}^d}{m}_{\rho }\left(\mathbf{x}\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left(\mathbf{x}\right)\left(f\left(\mathbf{x}\right)+o\left(1\right)\right)d\mathbf{x}\hfill \\ \hfill & ={(-1)}^{\left|\beta \right|}{\int }_{{\mathbb{R}}^d}r(\rho ,\mathbf{x})\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left(\mathbf{x}\right)d\mathbf{x}+o\left(1\right)\hfill \\ \hfill & =a_{\tau ,\mathbf{k}}+o\left(1\right).\hfill \end{array}$$

(23)

Consequently,

$${\tilde{a}}_{\tau ,\mathbf{k}}=a_{\tau ,\mathbf{k}}\mathrm{as}n\to \infty ,$$

(24)

□

Lemma 5.

For any $k\in {\mathbb{Z}}^d$, under the assumption (N.2)(i),the following holds:

$$\mathbb{E}\left[{\widehat{a}}_{\tau ,\mathbf{k}}\right]=a_{\tau ,\mathbf{k}},\mathrm{as}n\to \infty \mathrm{a}.\mathrm{s}.$$

(25)

Proof of Lemma 5.

Following a similar argument as for (23) and using the property $\mathbb{E}\left[\mathbb{E}\right[·\left|\mathcal{F}\right]]=\mathbb{E}[·]$ where $\mathcal{F}$ is a sigma field, under the assumption (N.2)(i), we have

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\widehat{a}}_{\tau ,\mathbf{k}}\right]& ={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\rho \left({\mathbf{Y}}_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)\right]\hfill \\ \hfill & ={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\mathbb{E}\left[\rho \left({\mathbf{Y}}_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)|{\mathbf{X}}_i\right]\right]\hfill \\ \hfill & ={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[m_{\rho }\left({\mathbf{X}}_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)\right]\hfill \\ \hfill & = {\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}{\int }_{{\mathbb{R}}^d}{m}_{\rho }\left(\mathbf{x}\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)(\mathbf{x},y)f\left(\mathbf{x}\right)d\mathbf{x}dy\hfill \\ \hfill & = {(-1)}^{\left|\beta \right|}{\int }_{{\mathbb{R}}^d\times \mathbb{R}}{\displaystyle \frac{r(\rho ,\mathbf{x})}{f\left(\mathbf{x}\right)}}\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left(\mathbf{x}\right)f\left(\mathbf{x}\right)d\mathbf{x}\hfill \end{array}$$

(26)

$$\begin{array}{cc} \hfill & = {(-1)}^{\left|\beta \right|}{\int }_{{\mathbb{R}}^d\times \mathbb{R}}r(\rho ,\mathbf{x})\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left(\mathbf{x}\right)d\mathbf{x}\hfill \end{array}$$

(27)

which implies

$$\mathbb{E}\left[{\widehat{a}}_{\tau ,\mathbf{k}}\right]=a_{\tau ,\mathbf{k}},\mathrm{as}n\to \infty .\mathrm{a}.\mathrm{s}.$$

(28)

This completes the proof of (25). □

Lemma 6.

For any $k\in {\mathbb{Z}}^d$, under the assumptions (C.1), (C.4)(i), (N.0), and (N.2)(i), the following holds:

$$\mathbb{E}\left[{\left({\widehat{a}}_{\tau ,\mathbf{k}}-a_{\tau ,\mathbf{k}}\right)}^2\right]=O\left({\displaystyle \frac{2^{2m(d/4+|\beta \left|\right)}}{n}}\right),\mathrm{as}n\to \infty .$$

(29)

Proof of Lemma 6.

We present the following decomposition:

$${\widehat{a}}_{\tau ,\mathbf{k}}-a_{\tau ,\mathbf{k}}=\left({\widehat{a}}_{\tau ,\mathbf{k}}-{\tilde{a}}_{\tau ,\mathbf{k}}\right)+\left({\tilde{a}}_{\tau ,\mathbf{k}}-a_{\tau ,\mathbf{k}}\right).$$

(30)

Therefore, we conclude that

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\left({\widehat{a}}_{\tau ,\mathbf{k}}-a_{\tau ,\mathbf{k}}\right)}^2\right]& =\mathbb{E}\left[{\left({\widehat{a}}_{\tau ,\mathbf{k}}-{\tilde{a}}_{\tau ,\mathbf{k}}\right)}^2\right]+\mathbb{E}\left[{\left({\tilde{a}}_{\tau ,\mathbf{k}}-a_{\tau ,\mathbf{k}}\right)}^2\right]\hfill \\ & +2\mathbb{E}\left[\left({\widehat{a}}_{\tau ,\mathbf{k}}-{\tilde{a}}_{\tau ,\mathbf{k}}\right)\left({\tilde{a}}_{\tau ,\mathbf{k}}-a_{\tau ,\mathbf{k}}\right)\right]\hfill \\ \hfill & =A_{m,\mathbf{k},1}+A_{m,\mathbf{k},2}+A_{m,\mathbf{k},3}.\hfill \end{array}$$

(31)

By applying using Lemma 4 and its assumptions, especially statement (22), one obtains

$$A_{m,\mathbf{k},2}=o\left(1\right).$$

(32)

Moreover, using the same statement (22), we deduce

$$\begin{array}{cc}\hfill A_{m,\mathbf{k},3}& =o\left(1\right)\mathbb{E}\left[{\widehat{a}}_{\tau ,\mathbf{k}}-{\tilde{a}}_{\tau ,\mathbf{k}}\right]\hfill \\ \hfill & =o\left(1\right){\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}\left(\mathbb{E}\left[\rho \left(Y_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)\right]-\mathbb{E}\left[\mathbb{E}\left[\rho \left(Y_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)|{\mathcal{F}}_{i-1}\right]\right]\right)\hfill \\ \hfill & =0.\hfill \end{array}$$

(33)

We now direct our focus to the first term in the decomposition (31). Observe that

$$\begin{array}{cc}\hfill {\widehat{a}}_{\tau ,\mathbf{k}}-{\tilde{a}}_{\tau ,\mathbf{k}}& ={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}\left(\rho \left(Y_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)-\mathbb{E}\left[\rho \left(Y_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)|{\mathcal{F}}_{i-1}\right]\right)\hfill \\ \hfill & = {\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n}}{\displaystyle \sum _{i=1}^n}{\mathrm{\Phi }}_{\tau ,\mathbf{k}}({\mathbf{X}}_i,Y_i).\hfill \end{array}$$

Observe that ${\left({\mathrm{\Phi }}_{\tau ,\mathbf{k}}\left({\mathbf{X}}_i\right)\right)}_{0\le k\le m_n}$ forms a martingale difference sequence with respect to the filtration ${\left({\mathcal{F}}_i\right)}_{0\le k\le m_n}$. Applying Lemma 2, we directly conclude:

$$A_{m,\mathbf{k},1}={\displaystyle \frac{{(-1)}^{2\left|\beta \right|}}{n^2}}\mathbb{E}\left[{\left|{\displaystyle \sum _{i=1}^n}{\mathrm{\Phi }}_{\tau ,\mathbf{k}}({\mathbf{X}}_i,Y_i)\right|}^2\right].$$

Moreover, the following holds:

$$\begin{array}{c}{\displaystyle {\left(\mathbb{E}\left[{\left|{\displaystyle \sum _{i=1}^n}{\mathrm{\Phi }}_{\tau ,\mathbf{k}}({\mathbf{X}}_i,Y_i)\right|}^2\right]\right)}^{\frac{1}{2}}}\hfill \\ \le \sqrt{n}{\parallel {\mathrm{\Phi }}_{\tau ,\mathbf{k}}({\mathbf{X}}_1,Y_1)\parallel }_2+{∥{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\mathrm{\Phi }}_{\tau ,\mathbf{k}}^2({\mathbf{X}}_i,Y_i)|{\mathcal{F}}_{i-1}\right]∥}_1^{\frac{1}{2}}\hfill \\ ={\mathrm{\Phi }}_{\left(1\right)}+{\mathrm{\Phi }}_{\left(2\right)}.\hfill \end{array}$$

(34)

To examine these components, we employ a standard decomposition approach and observe that ${\mathcal{F}}_0$ corresponds to the trivial $\sigma$-algebra. Consequently, the following result is derived:

$$\begin{array}{cc}\hfill {\displaystyle \frac{1}{n}}{\mathrm{\Phi }}_{\left(1\right)}^2& =\parallel {\mathrm{\Phi }}_{\tau ,\mathbf{k}}({\mathbf{X}}_1,Y_1){\parallel }_2^2\hfill \\ \hfill & =\mathbb{E}\left[{\left|\rho \left(Y_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)-\mathbb{E}\left[\rho \left(Y_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)|{\mathcal{F}}_0\right]\right|}^2\right]\hfill \\ \hfill & \le {\displaystyle \sum _{j=0}^2}\left({\displaystyle \genfrac{}{}{0pt}{}{2}{j}}\right)\mathbb{E}\left[{\left|\rho \left(Y_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|}^j{\left(\mathbb{E}\left[\left|\rho \left(Y_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|\right]\right)}^{2-j}\right]\hfill \\ \hfill & ={\displaystyle \sum _{j=0}^2}\left({\displaystyle \genfrac{}{}{0pt}{}{2}{j}}\right)\mathbb{E}\left[{\left|\rho \left(Y_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|}^j\right]{\left(\mathbb{E}\left[\left|\rho \left(Y_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|\right]\right)}^{2-j}\hfill \\ \hfill & =\mathbb{E}\left[{\left|\rho \left({\mathbf{Y}}_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|}^2\right]+3{\left(\mathbb{E}\left[\left|\rho \left(Y_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|\right]\right)}^2\hfill \\ \hfill & \le 4\mathbb{E}\left[{\left|\rho \left(Y_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|}^2\right].\hfill \end{array}$$

(35)

By invoking the Cauchy-Schwarz inequality we obtain

$$\begin{array}{ccc}\mathbb{E}\left[{\left|\rho \left(Y_1\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|}^2\right]\hfill & \le \hfill & {\mathbb{E}}^{1/2}\left[{\left|\rho \left(Y_1\right)\right|}^4\right]{\mathbb{E}}^{1/2}\left[{\left|\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|}^4\right]\hfill \end{array}$$

Under assumption (N.0), there exists a constant $C_{\rho ,4}$ such that

$$\mathbb{E}\left[{\left|\rho \left(Y_1\right)\right|}^4\right]\le C_{\rho ,4}<\infty .$$

on the other hand, by employing a first-order Taylor expansion alongside Equation (1) and assumption (C.4)(i), we obtain

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\left|\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_1\right)\right|}^4\right]& ={\int }_{{\mathbb{R}}^d}{\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)}^4\left(\mathbf{u}\right)f\left(\mathbf{u}\right)d\mathbf{u}\hfill \\ \hfill & =2^{m(2d+4|\beta \left|\right)}{\int }_{{\mathbb{R}}^d}{\left({\partial }^{\beta }\varphi \right)}^4\left(2^m\mathbf{u}-k\right)f\left(\mathbf{u}\right)d\mathbf{u}\hfill \\ \hfill & =2^{m(d+4|\beta \left|\right)}{\int }_{{\mathbb{R}}^d}{\left({\partial }^{\beta }\varphi \right)}^4\left(\mathbf{v}\right)f\left(\frac{\mathbf{v}+k}{2^m}\right)d\mathbf{v}.\hfill \\ \hfill & =2^{m(d+4|\beta \left|\right)}{\int }_{{\mathbb{R}}^d}{\left({\partial }^{\beta }\varphi \right)}^4\left(\mathbf{v}\right)\left(f\left(\mathbf{v}\right)+O\left(2^{-md}\right)\right)d\mathbf{v}\hfill \\ \hfill & \le 2^{m(d+4|\beta \left|\right)}{\left({\displaystyle \frac{A_i}{{\left(1+\parallel \mathbf{x}\parallel \right)}^i}}\right)}^4\hfill \\ \hfill & = O\left(2^{4m(d/4+|\beta \left|\right)}\right).\hfill \end{array}$$

(36)

This leads to

$$\begin{array}{ccc}{\mathrm{\Phi }}_{\left(1\right)}\hfill & =\hfill & O\left(2^{m(d/4+|\beta \left|\right)}{n}^{1/2}\right).\hfill \end{array}$$

(37)

Next, we analyze the second component of the decomposition in Equation (34). Specifically, we consider

$$\begin{array}{cc}\hfill {\mathrm{\Phi }}_{\left(2\right)}& ={\left(\mathbb{E}\left({\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\mathrm{\Phi }}_{\tau ,\mathbf{k}}^2({\mathbf{X}}_i,Y_i)|{\mathcal{F}}_{i-1}\right]\right)\right)}^{1/2}\hfill \\ \hfill & = {\left({\displaystyle \sum _{i=1}^n}\mathbb{E}\left(E\left[{\mathrm{\Phi }}_{\tau ,\mathbf{k}}^2({\mathbf{X}}_i,Y_i)|{\mathcal{F}}_{i-1}\right]\right)\right)}^{1/2}\hfill \\ \hfill & = {\left({\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\mathrm{\Phi }}_{\tau ,\mathbf{k}}^2({\mathbf{X}}_i,Y_i)\right]\right)}^{1/2}.\hfill \end{array}$$

Applying a standard identity, we find

$$\begin{array}{cc}\hfill \mathbb{E}\left[{\mathrm{\Phi }}_{\tau ,\mathbf{k}}^2({\mathbf{X}}_i,Y_i)\right]& =\mathbb{E}\left[\rho \left(Y_i\right){\left(\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)-\mathbb{E}\left[\rho \left(Y_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right)|{\mathcal{F}}_{i-1}\right]\right)}^2\right]\hfill \\ \hfill & \le 2\mathbb{E}\left[\mid \rho \left(Y_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right){\mid }^2\right]+2\mathbb{E}\left[\mathbb{E}\left[\mid \rho \left(Y_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right){\mid }^2|{\mathcal{F}}_{i-1}\right]\right]\hfill \\ \hfill & \le 4\mathbb{E}\left[\mid \rho \left(Y_i\right)\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right){\mid }^2\right].\hfill \end{array}$$

Using Equation (36), it follows that

$$\begin{array}{ccc}{\mathrm{\Phi }}_{\left(2\right)}\hfill & =\hfill & O\left(2^{m(d/4+|\beta \left|\right)}{n}^{1/2}\right).\hfill \end{array}$$

(38)

Combining the results from Equations (42) and (38), we derive

$$\begin{array}{c}{\left(\mathbb{E}\left[{\left|{\displaystyle \sum _{i=1}^n}{\mathrm{\Phi }}_{\tau ,\mathbf{k}}({\mathbf{X}}_i,Y_i)\right|}^2\right]\right)}^{\frac{1}{2}}=O\left(2^{m(d/4+|\beta \left|\right)}{n}^{1/2}\right).\hfill \end{array}$$

Consequently,

$$\begin{array}{cc}\hfill A_{m,\mathbf{k},1}& =\mathbb{E}\left[{\left({\widehat{a}}_{\tau ,\mathbf{k}}-{\tilde{a}}_{\tau ,\mathbf{k}}\right)}^2\right]\hfill \\ \hfill & ={\displaystyle \frac{{(-1)}^{2\left|\beta \right|}}{n^2}}\mathbb{E}\left[{\left|{\displaystyle \sum _{i=1}^n}{\mathrm{\Phi }}_{\tau ,\mathbf{k}}({\mathbf{X}}_i,Y_i)\right|}^2\right]\hfill \\ \hfill & = O\left(2^{2m(d/4+|\beta \left|\right)}{n}^{-1}\right).\hfill \end{array}$$

(39)

Finally, by integrating the findings from Equations (32), (33), and (39), we conclude that

$$\mathbb{E}\left[{\left({\widehat{a}}_{\tau ,\mathbf{k}}-a_{\tau ,\mathbf{k}}\right)}^2\right]=O\left({\displaystyle \frac{2^{2m(d/4+|\beta \left|\right)}}{n}}\right).$$

This completes the proof. □

Proof of Lemma 1.

The analysis of the bias term is purely analytical and follows from arguments analogous to those in [24], as it remains unaffected by the dependence structure. For brevity, we omit the detailed derivation. □

Proof of Theorem 1.

Building on standard wavelet estimation techniques (see [21] for a detailed exposition), we derive the main result in the following manner. First, by applying the definition of projector normality, we obtain

$$\begin{array}{c}{\displaystyle \mathbb{E}{∥{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})∥}_2^2}\hfill \\ = \mathbb{E}{∥\sum _{\mathbf{k}\in {\mathbb{Z}}^d}\left({\widehat{a}}_{\tau ,\mathbf{k}}-a_{\tau ,\mathbf{k}}\right){\varphi }_{\tau ,\mathbf{k}}∥}_2^2=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}\mathbb{E}\left[{\left({\widehat{a}}_{\tau ,\mathbf{k}}-a_{\tau ,\mathbf{k}}\right)}^2\right].\hfill \end{array}$$

Next, by invoking Lemma 6 and using the facts that $\left|V_{\ell }\right|\sim 2^m$ and $2^m\sim n^{\frac{1}{2s+1}}$, we deduce

$$\begin{array}{cc}\mathbb{E}{\parallel {\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})\parallel }_2^2\hfill & =\left(n^{\frac{1}{2s+1}}\right)O\left({\displaystyle \frac{2^{2m(d/4+|\beta \left|\right)}}{n}}\right)\hfill \\ \hfill & = O\left(n^{-\frac{2\left(s-\left|\beta \right|-d/4\right)}{2s+1}}\right).\hfill \end{array}$$

□

Proof of Theorem 2.

Now, we consider the following decomposition:

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in D}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-\mathbb{E}\left[{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})\right]\right|}\hfill \\ \le \underset{\mathbf{x}\in D}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-{\overline{r}}_n(\rho ;\mathbf{x})\right|+\underset{\mathbf{x}\in D}{sup}\left|{\overline{r}}_n(\rho ;\mathbf{x})-\mathbb{E}[{\widehat{({\partial }^{\beta }r)}}_n(\rho ;\mathbf{x})]\right|\hfill \\ = G_{n,1}(\rho ;\mathbf{x})+G_{n,2}(\rho ;\mathbf{x}).\hfill \end{array}$$

(40)

To establish the uniform consistency of the above term we will follow the method used in ([74]). Let us introduce the truncated version of ${\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})$ as follows. Let

$${\widehat{a}}_{\tau ,\mathbf{k}}^T={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}\rho \left({\mathbf{Y}}_i\right)\mathbb{1}\left\{\right|\rho \left({\mathbf{Y}}_i\right)|\le T_n\}\left({\partial }^{\beta }{\varphi }_{\tau ,\mathbf{k}}\right)\left({\mathbf{X}}_i\right),$$

(41)

and

$${\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{\tau ,\mathbf{k}}^T{\varphi }_{\tau ,\mathbf{k}}\left(\mathbf{x}\right).$$

(42)

Throughout this work, the notation $\mathbb{1}\left\{A\right\}$ represents the indicator function for the set A. In a similar way as in the (10), we write ${\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})$ as an extended kernel estimator

$${\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right).$$

(43)

Furthermore, we define:

$${\overline{r}}_n(\rho ;\mathbf{x})={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\rho \left(Y_i\right)K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\mid {\mathcal{F}}_{i-1}\right].$$

and

$${\overline{r}}_n^T\left(\mathbf{x}\right)={\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right].$$

(44)

We begin by breaking down the initial term of the Equation (40), $G_{n,1}(\rho ;\mathbf{x})$ into three distinct components, expressed as a sum through the following formulation:

$$\begin{array}{ccc}\hfill G_{n,1}(\rho ;\mathbf{x})& =\hfill & \underset{\mathbf{x}\in D}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-{\overline{r}}_n(\rho ;\mathbf{x})\right|\hfill \\ \hfill & \le \hfill & \underset{\mathbf{x}\in D}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})\right|+\underset{\mathbf{x}\in D}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})-{\overline{r}}_n^T(\rho ;\mathbf{x})\right|\hfill \\ & & +\underset{\mathbf{x}\in D}{sup}\left|{\overline{r}}_n^T(\rho ;\mathbf{x})-{\overline{r}}_n(\rho ;\mathbf{x})\right|\hfill \\ \hfill & =\hfill & W_{n,1}(\rho ;\mathbf{x})+W_{n,2}(\rho ;\mathbf{x})+W_{n,3}(\rho ;\mathbf{x}).\hfill \end{array}$$

(45)

Recalling statement (7), we obtain readily that

$$\underset{\mathbf{x}\in D}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})\right|\le {\displaystyle \frac{{(-1)}^{\left|\beta \right|}{h}_n{C}_{d+\left|\beta \right|+1}}{n}}{\displaystyle \sum _{i=1}^n}\left|\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|>T_n\}\right|.$$

(46)

Markov inequality, in turn, implies that

$$\mathbb{P}\left(|\rho \left(Y_n\right)|>T_n\right)\le T_n^{-\nu }\mathbb{E}\left(|\rho \left(Y_n\right){|}^{\nu }\right),$$

under assumption (N.0) and using the fact that

$$\sum _{n>1}{T}_n^{-\nu }<\infty ,$$

By the Borel-Cantelli lemma,

$$|\rho \left(Y_n\right)|\le T_n$$

almost surely for all sufficiently large n. Given the monotonicity of $T_n$, this implies $|\rho \left(Y_i\right)|\le T_n$ for all $i\le n$. Combining these results with the inequality (46) as in [74], we conclude that

$$W_{n,1}(\rho ;\mathbf{x})=\underset{\mathbf{x}\in D}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})\right|=o\left(h_n\right),a.s.$$

(47)

Once more, by (7), we infer that

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in D}{sup}\left|{\overline{r}}_n^T(\rho ;\mathbf{x})-{\overline{r}}_n(\rho ;\mathbf{x})\right|}\hfill \\ \le {\displaystyle \frac{1}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\left|\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\right.\hfill \\ \left.-\mathbb{E}\left[\rho \left(Y_i\right)K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\mid {\mathcal{F}}_{i-1}\right]\right|\hfill \\ \le {\displaystyle \frac{1}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|>T_n\}{K}^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ \le {\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|>T_n\}{\displaystyle \frac{C_{d+\left|\beta \right|+1}}{h_n^{d+\left|\beta \right|}(1+\parallel \mathbf{x}-{\mathbf{X}}_i{{\parallel }_2{h}_n^{-1})}^{d+\left|\beta \right|+1}}}|{\mathcal{F}}_{i-1}\right]\hfill \\ \le {\displaystyle \frac{h_n{C}_{d+\left|\beta \right|+1}}{n}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|>T_n\}|{\mathcal{F}}_{i-1}\right].\hfill \end{array}$$

Observe that the measurability of the functions $\rho (·)$ and $v\mapsto v^q$, combined with the ergodicity of ${\left(Y_n\right)}_{n\in \mathbb{N}}$, ensures that the process ${\left(g^{{\mathcal{F}}_{i-1}}\left(v\right)\right)}_{n\in \mathbb{N}}$ satisfies condition (N.1). Applying the Hölder and Markov inequalities, for any $ϵ>0$ and exponents $p,q$ fulfilling

$${\displaystyle \frac{1}{p}}+{\displaystyle \frac{1}{q}}=1,$$

that

$$\begin{array}{c}{\displaystyle \mathbb{E}\left[|\rho \left(Y_i\right)\left|\mathbb{1}\right\{|\rho \left(Y_i\right)|>T_n\}|{\mathcal{F}}_{i-1}\right]}\hfill \\ \le {\left(\mathbb{E}\left[|\rho \left(Y_i\right){|}^q|{\mathcal{F}}_{i-1}\right]\right)}^{1/q}{\left(\mathbb{P}\left\{\right|\rho \left(Y_i\right)|>T_n|{\mathcal{F}}_{i-1}\}\right)}^{1/p}\hfill \\ \le T_n^{-q}\mathbb{E}\left[|\rho \left(Y_i\right){|}^q|{\mathcal{F}}_{i-1}\right]\hfill \\ = T_n^{-q}{\int }_{\mathbb{R}}{\left|\rho \left(v\right)\right|}^q{g}^{{\mathcal{F}}_{i-1}}\left(v\right)dv.\hfill \end{array}$$

This, in turn, readily implies that

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in D}{sup}\left|{\overline{r}}_n^T(\rho ;\mathbf{x})-{\overline{r}}_n(\rho ;\mathbf{x})\right|}\hfill \\ \le h_n{C}_{d+\left|\beta \right|+1}\underset{y\in \mathbb{R}}{sup}∥{\displaystyle \frac{1}{ng\left(y\right)}}{\displaystyle \sum _{i=1}^n}{g}^{{\mathcal{F}}_{i-1}}\left(y\right)∥T_n^{-q}{\int }_{\mathbb{R}}{\left|\rho \left(v\right)\right|}^{q}g\left(v\right)dv\hfill \\ = h_n\underset{y\in \mathbb{R}}{sup}∥{\displaystyle \frac{1}{ng\left(y\right)}}{\displaystyle \sum _{i=1}^n}{g}^{{\mathcal{F}}_{i-1}}\left(y\right)∥T_n^{-q}\mathbb{E}\left[|\rho \left(Y_i\right){|}^q\right],\hfill \end{array}$$

which gives

$$W_{n,3}(\rho ;\mathbf{x})=\underset{\mathbf{x}\in D}{sup}\left|{\overline{r}}_n^T(\rho ;\mathbf{x})-{\overline{r}}_n(\rho ;\mathbf{x})\right|=O\left(h_n\right)a.s.$$

(48)

Lastly, we turn our attention to second term of the decomposition (45), given the compactness of D, there exists a finite covering by $L\left(n\right)$ cubes $I_{n,j}$, each centered at ${\mathbf{x}}_{n,j}$ with side length ${\ell }_n$, where $j=1,\dots ,L\left(n\right)$. It follows directly that

$${\ell }_n={\displaystyle \frac{\mathrm{const}}{L^{1/d}\left(n\right)}}.$$

where

$$L\left(n\right)={\left({\displaystyle \frac{2^{(d+2)m\left(n\right)}n{T}_n^2}{logn}}\right)}^{d/2}={\left({\displaystyle \frac{n{T}_n^2}{h_n^{d+2}logn}}\right)}^{d/2}.$$

we readily infer that

$$\begin{array}{ccc}\hfill W_{n,2}(\rho ;\mathbf{x})& =\hfill & \underset{\mathbf{x}\in D}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})-{\overline{r}}_n^T(\rho ;\mathbf{x})\right|\hfill \\ \hfill & =\hfill & \underset{1\le j\le L\left(n\right)}{max}\underset{\mathbf{x}\in D\cap I_j}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})-{\overline{r}}_n^T(\rho ;\mathbf{x})\right|\hfill \\ \hfill & \le \hfill & \underset{1\le j\le L\left(n\right)}{max}\underset{\mathbf{x}\in D\cap I_j}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})-{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;{\mathbf{x}}_j)\right|\hfill \\ & & +\underset{1\le j\le L\left(n\right)}{max}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;{\mathbf{x}}_j)-{\overline{r}}_n^T(\rho ;{\mathbf{x}}_j)\right|\hfill \\ & & +\underset{1\le j\le L\left(n\right)}{max}\underset{\mathbf{x}\in D\cap I_j}{sup}\left|{\overline{r}}_n^T(\rho ;{\mathbf{x}}_j)-{\overline{r}}_n^T(\rho ;\mathbf{x})\right|\hfill \\ \hfill & =\hfill & Q_1+Q_2+Q_3.\hfill \end{array}$$

(49)

The statement (9) allows us to infer that

$$\begin{array}{c}{\displaystyle \left|{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})-{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;{\mathbf{x}}_j)\right|}\hfill \\ = \left|{\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}\left(K\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)-K\left({\displaystyle \frac{{\mathbf{x}}_j}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)\right|\hfill \\ \le {\displaystyle \frac{d^{1/2}{C}_2{T}_n}{h_n^{d+\left|\beta \right|+1}}}\parallel \mathbf{x}-{\mathbf{x}}_i\parallel ,\hfill \end{array}$$

which implies that

$$\begin{array}{cc}\hfill Q_1& =\underset{1\le j\le L\left(n\right)}{max}\underset{\mathbf{x}\in D\cap I_j}{sup}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;\mathbf{x})-{\overline{r}}_n^T(\rho ;\mathbf{x})\right|\hfill \\ \hfill & \le {\displaystyle \frac{d^{1/2}{C}_2{T}_n{\ell }_n}{h_n^{d+\left|\beta \right|+1}}}={\displaystyle \frac{d^{1/2}{C}_2{T}_{n}const}{L^{1/d}\left(n\right)h_n^{d+\left|\beta \right|+1}}}\hfill \\ \hfill & = O\left({\left({\displaystyle \frac{logn}{n{h}_n^{d+2\left|\beta \right|}}}\right)}^{1/2}\right),a.s.\hfill \end{array}$$

(50)

A similar argument shows, likewise, that

$$\begin{array}{cc}\hfill Q_3& =\underset{1\le j\le L\left(n\right)}{max}\underset{\mathbf{x}\in D\cap I_j}{sup}\left|{\overline{r}}_n^T(\rho ;{\mathbf{x}}_j)-{\overline{r}}_n^T(\rho ;\mathbf{x})\right|\hfill \\ \hfill & \le {\displaystyle \frac{d^{1/2}{C}_2{T}_n{\ell }_n}{h_n^{d+\left|\beta \right|+1}}}={\displaystyle \frac{d^{1/2}{C}_2{T}_{n}const}{L^{1/d}\left(n\right)h_n^{d+\left|\beta \right|+1}}}\hfill \\ \hfill & = O\left({\left({\displaystyle \frac{logn}{n{h}_n^{d+2\left|\beta \right|}}}\right)}^{1/2}\right),a.s.\hfill \end{array}$$

(51)

We now analyze the term $Q_2$ in the right side of (49) and show that

$$Q_2=O\left({\left({\displaystyle \frac{logn}{n{h}_n^{d+2\left|\beta \right|}}}\right)}^{1/2}\right),a.s.$$

(52)

Observe that

$$\begin{array}{ccc}\hfill Q_2& =\hfill & \underset{1\le j\le L\left(n\right)}{max}\left|{\widehat{\left({\partial }^{\beta }r\right)}}_n^T(\rho ;{\mathbf{x}}_j)-{\overline{r}}_n^T(\rho ;{\mathbf{x}}_j)\right|\hfill \\ \hfill & =\hfill & \underset{1\le j\le L\left(n\right)}{max}\left|{\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\left(\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right.\right.\hfill \\ & & \left.\left.-\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\right)\right|\hfill \\ \hfill & =\hfill & \underset{1\le j\le L\left(n\right)}{max}\left|{\displaystyle \frac{1}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}Z(\rho ,{\mathbf{x}}_j)\right|,\hfill \end{array}$$

where, for $i=1,\dots ,n$,

$$\begin{array}{ccc}Z(\rho ,{\mathbf{x}}_j)\hfill & =\hfill & {(-1)}^{\left|\beta \right|}\left(\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{{\mathbf{x}}_j}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right.\hfill \\ & & \left.-\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{{\mathbf{x}}_j}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\right)\hfill \end{array}$$

forms a sequence of martingale-difference arrays relative to the $\sigma$-field ${\mathcal{F}}_i$. We now utilize the Lemma 3 for the partial sums of unbounded of martingale differences to obtain an upper bound. Observe that

$$\begin{array}{c}\hfill {\displaystyle {\left(Z(\rho ,{\mathbf{x}}_j)\right)}^p}\\ \hfill & =\hfill & {(-1)}^{p\left|\beta \right|}\left(\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right.\hfill \\ & & {\left.-\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\right)}^p\hfill \\ \hfill & =\hfill & {(-1)}^{p\left|\beta \right|}{\displaystyle \sum _{k=0}^p}{(-1)}^{p-k}{C}_p^k{\left(\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^k\hfill \\ & & {\left(\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\right)}^{p-k}.\hfill \end{array}$$

By using the fact that

$${\left(\mathbb{E}\left[\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{{\mathbf{x}}_j}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\right)}^{p-k}$$

is ${\mathcal{F}}_{i-1}$-measurable, it follows then

$$\begin{array}{c}{\displaystyle \left|\mathbb{E}\left[Z{(\rho ,{\mathbf{x}}_j)}^p|{\mathcal{F}}_{i-1}\right]\right|}\hfill \\ \le {\displaystyle \sum _{k=0}^p}{C}_p^k\mathbb{E}\left[\left|{\left(\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{{\mathbf{x}}_j}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^k\right||{\mathcal{F}}_{i-1}\right]\hfill \\ \times \mathbb{E}\left[\left|{\left(\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{{\mathbf{x}}_j}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^{p-k}\right||{\mathcal{F}}_{i-1}\right].\hfill \end{array}$$

(53)

recalling the fact that

$${\mathcal{F}}_{i-1}\subset {\mathcal{G}}_{i-1}$$

and applying the assumption (N.2)(ii), for any integer $\ell >1$, we obtain

$$\begin{array}{c}{\displaystyle \mathbb{E}\left[{\left|\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left(\frac{\mathbf{x}}{h_n},\frac{{\mathbf{X}}_i}{h_n}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right]}\hfill \\ = \mathbb{E}\left[\mathbb{E}\left[{\left|\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right|}^{\ell }|{\mathcal{G}}_{i-1}\right]|{\mathcal{F}}_{i-1}\right]\hfill \\ \le \mathbb{E}\left[\mathbb{E}\left[{\left|\rho \left(Y_i\right)\right|}^{\ell }|{\mathcal{G}}_{i-1}\right]{\left|K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right]\hfill \\ = \mathbb{E}\left[m_{\rho ,\ell }\left({\mathbf{X}}_i\right){\left|K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right],\hfill \end{array}$$

(54)

where $m_{\rho ,\ell }\left({\mathbf{X}}_i\right)$ is a positive function as stipulated in assumption (N.2)(ii), ensures that

$$\begin{array}{c}{\displaystyle \mathbb{E}\left[{\left|\rho \left(Y_i\right)\mathbb{1}\left\{\right|\rho \left(Y_i\right)|\le T_n\}{K}^{\left(\beta \right)}\left(\frac{\mathbf{x}}{h_n},\frac{{\mathbf{X}}_i}{h_n}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right]}\hfill \\ \le \mathbb{E}\left[\mid m_{\rho ,\ell }\left({\mathbf{X}}_i\right)-m_{\rho ,\ell }\left(\mathbf{x}\right)\mid {\left|K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right]\hfill \\ +m_{\rho ,\ell }\left(\mathbf{x}\right)\mathbb{E}\left[{\left|K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right]\hfill \\ =\left[\underset{\parallel \mathbf{u}-\mathbf{x}\parallel \le h_n}{sup}\mid m_{\rho ,\ell }\left(\mathbf{u}\right)-m_{\rho ,\ell }\left(\mathbf{x}\right)\mid +m_{\rho ,\ell }\left(\mathbf{x}\right)\right]\mathbb{E}\left[{\left|K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right]\hfill \\ \le C_0\mathbb{E}\left[{\left|K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right],\hfill \end{array}$$

where $C_0$ is a positive constant. It follows readily from (7),

$$\begin{array}{ccc}\hfill \mathbb{E}\left[{\left|K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right]& \le \hfill & \mathbb{E}\left[{\left|K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right|}^{\ell }|{\mathcal{F}}_{i-1}\right]\hfill \\ \hfill & =\hfill & {\int }_{{\mathbb{R}}^d}{\left|K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{y}}{h_n}}\right)\right|}^{\ell }{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{y}\right)d\mathbf{y}\hfill \\ \hfill & \le \hfill & {\int }_{{\mathbb{R}}^d}{\left({\displaystyle \frac{C_{d+\left|\beta \right|}}{(1+h_n^{-1}{\parallel \mathbf{x}-\mathbf{y}\parallel }_2{)}^{d+\left|\beta \right|}}}\right)}^{\ell }{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{y}\right)d\mathbf{y}\hfill \\ \hfill & \le \hfill & h_n^{\ell (d+|\beta \left|\right)}{C}_{d+\left|\beta \right|}^{\ell },\hfill \end{array}$$

(55)

statements (53), (54) and (55) give the following upper bound

$$\begin{array}{ccc}\hfill \left|\mathbb{E}\left[Z{(\rho ,{\mathbf{x}}_j)}^p|{\mathcal{F}}_{i-1}\right]\right|& \le \hfill & C_0^2{\displaystyle \sum _{k=0}^p}{C}_p^k{h}_n^{p(d+|\beta \left|\right)}{C}_{d+\left|\beta \right|}^p\hfill \\ \hfill & \le \hfill & C_0^2{2}^p{h}_n^{d+\left|\beta \right|}{C}_{d+\left|\beta \right|}^p\le p!C^{p-1}{d}_i^2.\hfill \end{array}$$

We apply Lemma 3 to the summation of the martingale differences $Z_n({\mathbf{x}}_j,{\mathbf{X}}_i)$, where

$$C=C_{d+\left|\beta \right|},d_i^2=C_0^2{C}_{d+\left|\beta \right|}{h}_n^{d+\left|\beta \right|},D_n={\displaystyle \sum _{i=1}^n}{d}_i^2=O\left(n{h}_n^{d+\left|\beta \right|}\right)$$

and

$${ϵ}_n={ϵ}_0{(logn/n{h}_n^{d+2\left|\beta \right|})}^{1/2}.$$

Consequently, there exists a positive constant $C_2>0$ such that the following inequalities hold:

$$\begin{array}{c}{\displaystyle \mathbb{P}\left(\underset{1\le j\le L\left(n\right)}{max}\left|\frac{1}{n{h}_n^{d+\left|\beta \right|}}{\displaystyle \sum _{i=1}^n}Z(\rho ,{\mathbf{x}}_j)\right|>{ϵ}_n\right)}\hfill \\ \le {\displaystyle \sum _{j=1}^{L\left(n\right)}}\mathbb{P}\left(\left|{\displaystyle \sum _{i=1}^n}Z(\rho ,{\mathbf{x}}_j)\right|>{ϵ}_{n}n{h}_n^{d+\left|\beta \right|}\right)\hfill \\ \le 2L\left(n\right)exp\left\{-{\displaystyle \frac{{ϵ}_0^2{\left(n{h}_n^{d+\left|\beta \right|}\right)}^2\left(logn/n{h}_n^{d+2\left|\beta \right|}\right)}{2(D_n+2C_{d+1}n{h}_n^{d+\left|\beta \right|}{\left(logn/n{h}_n^{d+2\left|\beta \right|}\right)}^{1/2})}}\right\}\hfill \\ \le {\left({\displaystyle \frac{n{T}_n^2}{h_n^{d+1}logn}}\right)}^{d/2}exp\left\{-{\displaystyle \frac{{ϵ}_0^{2}n{h}_n^{d+\left|\beta \right|}\left(logn/h_n^{\left|\beta \right|}\right)}{O\left(n{h}_n^{d+\left|\beta \right|}\right)(1+2C_{d+1}{(logn/n{h}_n^{d+2\left|\beta \right|})}^{1/2})}}\right\}\hfill \\ \le {\left(T_n{\left({\displaystyle \frac{logn}{n{h}_n^d}}\right)}^{1/2}\right)}^d{\left({\displaystyle \frac{1}{{\left(n{h}_n\right)}^{1/2}logn}}\right)}^d{n}^{-\left({\displaystyle \frac{C_2{ϵ}_0^2}{h_n^{\left|\beta \right|}}}-{\displaystyle \frac{3d}{2}}\right)}.\hfill \end{array}$$

Observe that

$$n{h}_n\to \infty \mathrm{as}n\to \infty ,\mathrm{and}{\displaystyle \frac{1}{h_n^{\left|\beta \right|}}}\to \infty \mathrm{as}n\to \infty .$$

this implies that when $n\to \infty$ we have

$${\displaystyle \frac{C_2{ϵ}_0^2}{h_n^{\left|\beta \right|}}}-{\displaystyle \frac{3d}{2}}>0,$$

we readily obtain, under condition (11), that

$$\sum _{n\ge 1}\mathbb{P}\left(\underset{1\le j\le L\left(n\right)}{max}\left|{\displaystyle \frac{1}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}Z(\rho ,{\mathbf{x}}_j)\right|>{ϵ}_n\right)<\infty .$$

(56)

The assertion (52) is established through a standard application of the Borel-Cantelli lemma. Consequently, the result stated in Theorem 2 follows by applying the decompositions (40) that resulted in two terms: the first one was rewritten in (45) as the sum of three terms, the statements (47) and (48) and the decomposition (49) in conjunction with estimates (50), (51), and (52). To complete the argument, we now analyze the second term on the right-hand side of (40). Employing the same reasoning as in (54), and in view of assumptions (N.2)(i) and (N.3)(i), we observe that

$$\begin{array}{c}{\displaystyle \underset{\mathbf{x}\in D}{sup}\left|{\overline{r}}_n(\rho ;\mathbf{x})-\mathbb{E}\left[{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})\right]\right|}\hfill \\ =\underset{\mathbf{x}\in D}{sup}\left|{\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\left(\mathbb{E}\left[\rho \left(Y_i\right)K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]-\mathbb{E}\left[\rho \left(Y_i\right)K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right]\right)\right|\hfill \\ =\underset{\mathbf{x}\in D}{sup}\left|{\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\left(\mathbb{E}\left[m_{\rho }\left({\mathbf{X}}_i\right)K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]-\mathbb{E}\left[m_{\rho }\left({\mathbf{X}}_i\right)K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right]\right)\right|\hfill \\ =\underset{\mathbf{x}\in D}{sup}\left|{\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}{\int }_{{\mathbb{R}}^d}{m}_{\rho }\left(\mathbf{y}\right)K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{y}}{h_n}}\right)\left(f^{{\mathcal{F}}_{i-1}}\left(\mathbf{y}\right)-f\left(\mathbf{y}\right)\right)d\mathbf{y}\right|\hfill \\ = \underset{\mathbf{x}\in D}{sup}\left|{\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{h_n^{d+\left|\beta \right|}}}{\int }_{{\mathbb{R}}^d}{m}_{\rho }\left(\mathbf{y}\right)K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{y}}{h_n}}\right)\left({\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{y}\right)-f\left(\mathbf{y}\right)\right)d\mathbf{y}\right|\hfill \\ \le C_{\mathbf{m}}\underset{\mathbf{x}\in D}{sup}\left|{\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{h_n^{d+\left|\beta \right|}}}{\int }_{{\mathbb{R}}^d}{K}^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{y}}{h_n}}\right)\left({\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{y}\right)-f\left(\mathbf{y}\right)\right)d\mathbf{y}\right|.\hfill \end{array}$$

Making use of the Cauchy-Schwarz inequality and statement (7) when

$$m=d+2\left|\beta \right|+1,$$

we obtain readily that

$$\begin{array}{c}{\displaystyle \left|\frac{{(-1)}^{\left|\beta \right|}}{h_n^{d+\left|\beta \right|}}{\int }_{{\mathbb{R}}^d}{K}^{\beta }\left(\frac{\mathbf{x}}{h_n},\frac{\mathbf{y}}{h_n}\right)\left(\frac{1}{n}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{y}\right)-f\left(\mathbf{y}\right)\right)d\mathbf{y}\right|}\hfill \\ \le {\left({\int }_{{\mathbb{R}}^d}{\left|{\displaystyle \frac{1}{h_n^{d+\left|\beta \right|}}}{K}^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{y}}{h_n}}\right)\right|}^{2}d\mathbf{y}\right)}^{1/2}{\left({\int }_{{\mathbb{R}}^d}|{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{y}\right)-f\left(\mathbf{y}\right){|}^{2}d\mathbf{y}\right)}^{1/2}\hfill \\ \le \parallel {\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}-f{\parallel }_{L^2}{\left({\int }_{{\mathbb{R}}^d}\left({\displaystyle \frac{1}{h_n^{d+2\left|\beta \right|}}}{\displaystyle \frac{C_{d+2\left|\beta \right|+1}}{(1+h_n^{-1}{\parallel \mathbf{x}-\mathbf{y}\parallel )}^{d+2\left|\beta \right|+1}}}\right)\left|{\displaystyle \frac{1}{h_n^d}}{K}^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{y}}{h_n}}\right)\right|d\mathbf{y}\right)}^{1/2}\hfill \\ \le h_n^{1/2}{C}_{d+2\left|\beta \right|+1}\parallel {\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}-f{\parallel }_{L^2}{\left({\int }_{{\mathbb{R}}^d}\left|K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},\mathbf{z}\right)\right|d\mathbf{z}\right)}^{1/2}\hfill \\ \le h_n^{1/2}{C}_{d+2\left|\beta \right|+1}\parallel {\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}-f{\parallel }_{L^2}{G}_1^{1/2}\left(d\right).\hfill \end{array}$$

Under Assumption (C.2), we deduce that

$$\begin{array}{cc}{G}_{n,2}\left(\mathbf{x}\right)\hfill & =\underset{\mathbf{x}\in D}{sup}\left|{\overline{r}}_n(\rho ;\mathbf{x})-\mathbb{E}\left[{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})\right]\right|\hfill \\ \hfill & \le C_{\mathbf{m}}{h}_n^{1/2}{C}_{d+2\left|\beta \right|+1}{G}_1^{1/2}\left(d\right)\parallel {\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}-f{\parallel }_{L^2}\hfill \\ \hfill & = O\left(h_n^{1/2}\right).\hfill \end{array}$$

(57)

Hence the proof is complete. □

Proof of Theorem 3.

Recall the decomposition

$$\begin{array}{c}{\displaystyle \sqrt{n{h}_n^{d+2\left|\beta \right|}}\left({\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})\right)}\hfill \\ = \sqrt{n{h}_n^{d+2\left|\beta \right|}}\left(\left({\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})-{\overline{r}}_n(\rho ;\mathbf{x})\right)\right.\hfill \\ +\left.\left({\overline{r}}_n(\rho ;\mathbf{x})-\widehat{\left({\partial }^{\beta }r\right)}(\rho ;\mathbf{x})\right)\right)\hfill \\ = \sqrt{n{h}_n^{d+2\left|\beta \right|}}\left(Q_n(\rho ;\mathbf{x})+B_n(\rho ;\mathbf{x})\right).\hfill \end{array}$$

(58)

Observe that

$$\begin{array}{ccc}{B}_n(\rho ;\mathbf{x})\hfill & =\hfill & \left({\overline{r}}_n(\rho ;\mathbf{x})-\mathbb{E}\left[{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})\right]\right)\hfill \\ \hfill & & +\left(\mathbb{E}\left[{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})\right]-\widehat{\left({\partial }^{\beta }r\right)}(\rho ;\mathbf{x})\right)\hfill \\ \hfill & =\hfill & B_{n,1}(\rho ;\mathbf{x})+B_{n,2}(\rho ;\mathbf{x}).\hfill \end{array}$$

(59)

Thus, employing reasoning analogous to that used for (57), under hypothesis (C.2), statement (7) with $m=2(d+|\beta \left|\right)$, and Condition (12), we directly conclude that

$$\begin{array}{c}{\displaystyle {\left(n{h}_n^{d+2\left|\beta \right|}\right)}^{1/2}{B}_{n,1}\left(\mathbf{x}\right)}\hfill \\ ={\left(n{h}_n^{d+2\left|\beta \right|}\right)}^{1/2}\left({\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\left(\mathbb{E}\left[\rho \left(Y_i\right)K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]-\mathbb{E}\left[\rho \left(Y_i\right)K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right]\right)\right)\hfill \\ =O\left(h_n^{d/2}\right){\left(n{h}_n^{d+2\left|\beta \right|}\right)}^{1/2}\hfill \\ =o\left(1\right).\hfill \end{array}$$

(60)

We turn our attention to the term $B_{n,2}(\rho ;\mathbf{x})$ and observe that under assumption (C.3), we have

$$\begin{array}{ccc}\mathbb{E}\left[{\widehat{\left({\partial }^{\beta }r\right)}}_n(\rho ;\mathbf{x})\right]\hfill & =\hfill & {\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\rho \left(Y_i\right)K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right]\hfill \\ \hfill & =\hfill & {\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\mathbb{E}\left[\rho \left(Y_i\right)|{\mathbf{X}}_i\right]K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right]\hfill \\ \hfill & =\hfill & {\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{n{h}_n^{d+\left|\beta \right|}}}{\displaystyle \sum _{i=1}^n}{\int }_{{\mathbb{R}}^d}{m}_{\rho }\left(\mathbf{u}\right)K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{u}}{h_n}}\right)f\left(\mathbf{u}\right)d\mathbf{u}\hfill \\ \hfill & =\hfill & {\displaystyle \frac{1}{h_n^d}}{\int }_{{\mathbb{R}}^d}\left({\partial }^{\beta }r\right)(\rho ;\mathbf{u})K\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{u}}{h_n}}\right)d\mathbf{u}\hfill \\ \hfill & =\hfill & {\int }_{{[-2L,2L]}^d}\left({\partial }^{\beta }r\right)(\rho ;\mathbf{x}+h_n\mathbf{v})K\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{x}}{h_n}}+\mathbf{v}\right)d\mathbf{v}\hfill \\ \hfill & =\hfill & \left({\partial }^{\beta }r\right)(\rho ;\mathbf{x})+h_n\left({\partial }^{\beta +1}r\right)(\rho ;\mathbf{x}){\int }_{{[-2L,2L]}^d}K\left(0,\mathbf{v}\right)d\mathbf{v}.\hfill \end{array}$$

(61)

On the other hand, by Condition (12), we have

$${\left(n{h}_n^{d+2\left|\beta \right|}\right)}^{1/2}{B}_{n,2}(\rho ;\mathbf{x})=O\left(h_n^{\delta }{\left(n{h}_n^{d+2|\beta }\right)}^{1/2}\right)=o\left(1\right).$$

(62)

Observe that

$$\begin{array}{ccc}\sqrt{n{h}_n^{d+2\left|\beta \right|}}{Q}_n\left(\mathbf{x}\right)\hfill & =\hfill & {\displaystyle \sum _{i=1}^n}\left\{\left({\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{\sqrt{n{h}_n^d}}}\rho \left(Y_i\right)K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)\right.\hfill \end{array}$$

$$\begin{array}{ccc}& & \left.-\mathbb{E}\left[{\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{\sqrt{n{h}_n^d}}}\rho \left(Y_i\right)K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\right\}\hfill \\ \hfill & =\hfill & {\displaystyle \sum _{i=1}^n}\left\{{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)-\mathbb{E}\left[{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]\right\}\hfill \end{array}$$

(63)

$$\begin{array}{c} = {\displaystyle \sum _{i=1}^n}{\chi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i),\hfill \end{array}$$

(64)

Here, ${\chi }_{ni}(\mathbf{x},\rho )$ constitutes a martingale difference array adapted to the filtration ${\mathcal{F}}_i$. This justifies applying the martingale central limit theorem for discrete-time arrays (cf. [75]) to demonstrate the asymptotic normality of $\sqrt{n{h}_n^{d+2\left|\beta \right|}}{Q}_n\left(\mathbf{x}\right)$. To achieve this, it suffices to verify the following conditions:

(a)

Lyapunov condition:

$${\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\chi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]\stackrel{\mathbb{P}}{\to }{\Sigma }^2\left(\mathbf{x}\right);$$

(b)

Lindberg condition:

$$n\mathbb{E}\left[{\chi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i){\mathbb{1}}_{\left\{\left|{\chi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\right|>ϵ\right\}}\right]=o\left(1\right),\mathrm{holds} \mathrm{true} \mathrm{for} \mathrm{any}ϵ>0.$$

Proof of part (a).

Observe that

$$\left|{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\xi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]-{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\chi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]\right|={\displaystyle \sum _{i=1}^n}{\left(\mathbb{E}\left[{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]\right)}^2.$$

Employing the same reasoning as in (54), and employing a first-order Taylor expansion alongside Equation (7), and in view of assumptions (N.2)(i), (N.3)(i), and (C.4)(ii), we obtain

$$\begin{array}{ccc}\mathbb{E}\left[{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]\hfill & =\hfill & {\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{\sqrt{n{h}_n^d}}}\mathbb{E}\left[\rho \left(Y_i\right)K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ \hfill & =\hfill & {\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{\sqrt{n{h}_n^d}}}\mathbb{E}\left[\mathbb{E}\left[\rho \left(Y_i\right)|{\mathcal{G}}_{i-1}\right]K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)|{\mathcal{F}}_{i-1}\right]\hfill \\ \hfill & =\hfill & {\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{\sqrt{n{h}_n^d}}}{\int }_{{\mathbb{R}}^d}{m}_{\rho }\left(y\right)K^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{y}}{h_n}}\right)f^{{\mathcal{F}}_{i-1}}\left(\mathbf{y}\right)d\mathbf{y}\hfill \\ \hfill & \le \hfill & {\displaystyle \frac{{(-1)}^{\left|\beta \right|}{C}_{\mathbf{m}}\sqrt{h_n^d}}{\sqrt{n}}}{\int }_{{\mathbb{R}}^d}{K}^{\beta }\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{x}}{h_n}}+\mathbf{u}\right)f^{{\mathcal{F}}_{i-1}}(\mathbf{x}+h_n\mathbf{u})d\mathbf{u}\hfill \\ \hfill & \le \hfill & \left({\displaystyle \frac{{(-1)}^{\left|\beta \right|}{C}_{\mathbf{m}}\sqrt{h_n^d}}{\sqrt{n}}}\right)\left({\displaystyle \frac{C_{d+\left|\beta \right|}}{{(1+\parallel \mathbf{u}\parallel }_2{)}^{d+\left|\beta \right|}}}\right)\left(f^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)+o\left(1\right)\right).\hfill \end{array}$$

Therefore, we infer

$$\begin{array}{c}{\displaystyle {\displaystyle \sum _{i=1}^n}{\left(\mathbb{E}\left[{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]\right)}^2}\hfill \\ =\left(h_n^d\right){\left({\displaystyle \frac{C_{d+\left|\beta \right|}{C}_{\mathbf{m}}}{{(1+\parallel \mathbf{u}\parallel }_2{)}^{d+\left|\beta \right|}}}\right)}^2\left({\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\left(f^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\right)}^2+o\left(1\right)\right).\hfill \end{array}$$

The ergodicity of the process ${\left({\mathbf{X}}_n\right)}_{n\ge 1}$ implies that the process defined by ${\left({\left(f^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\right)}^2\right)}_{n\ge 1}$ is ergodic too and verifies the condition (C.2), which means that

$${\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\left(f^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)\right)}^2={\left(f\left(\mathbf{x}\right)\right)}^2,$$

in both the almost sure and $L^2$ senses. This implies

$${\displaystyle \sum _{i=1}^n}{\left(\mathbb{E}\left[{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]\right)}^2=O\left(h_n^d\right).$$

The statement (a) follows then from

$$\underset{n\to \infty }{lim}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\xi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]\stackrel{\mathbb{P}}{=}{\Sigma }_{\rho }^2\left(\mathbf{x}\right).$$

Observe that by the fact that (see [28])

$$K^{\left(\beta \right)}(\mathbf{u},\mathbf{v})=K^{\left(\beta \right)}(\mathbf{u}+\mathbf{k},\mathbf{v}+\mathbf{k}),for\mathbf{k}\in {\mathbb{Z}}^d.$$

By assumption (N.2)(ii), we have

$$\begin{array}{c}{\displaystyle {\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\xi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{F}}_{i-1}\right]}\hfill \\ = {\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\left({\displaystyle \frac{{(-1)}^{\left|\beta \right|}}{\sqrt{n{h}_n^d}}}\rho \left(Y_i\right)K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^2|{\mathcal{F}}_{i-1}\right]\hfill \\ = {\displaystyle \frac{1}{n{h}_n^d}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\mathbb{E}\left[{\rho }^2\left(Y_i\right)|{\mathcal{G}}_{i-1}\right]{\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^2|{\mathcal{F}}_{i-1}\right]\hfill \\ = {\displaystyle \frac{1}{n{h}_n^d}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[m_{\rho ,2}\left({\mathbf{X}}_i\right){\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^2|{\mathcal{F}}_{i-1}\right]\hfill \\ \le {\displaystyle \frac{1}{n{h}_n^d}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\underset{\{\mathbf{u}:\parallel \mathbf{x}-\mathbf{u}\parallel <h\}}{sup}\left|m_{\rho ,2}\left({\mathbf{X}}_i\right)-m_{\rho ,2}\left(\mathbf{x}\right)\right|{\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^2|{\mathcal{F}}_{i-1}\right]\hfill \\ +{\displaystyle \frac{1}{n{h}_n^d}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[m_{\rho ,2}\left(\mathbf{x}\right){\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^2|{\mathcal{F}}_{i-1}\right]\hfill \\ = I_{n,1}\left(\mathbf{x}\right)+I_{n,2}\left(\mathbf{x}\right).\hfill \end{array}$$

(65)

Under assumptions (C.1), (C.4)(ii), and (N.3)(iii) we have

$$\begin{array}{ccc}{I}_{n,1}\left(\mathbf{x}\right)\hfill & =\hfill & {\displaystyle \frac{1}{n{h}_n^d}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[\underset{\{\mathbf{u}:\parallel \mathbf{x}-\mathbf{u}\parallel <h\}}{sup}\left|m_{\rho ,2}\left({\mathbf{X}}_i\right)-m_{\rho ,2}\left(\mathbf{x}\right)\right|{\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^2|{\mathcal{F}}_{i-1}\right]\hfill \\ \hfill & =\hfill & {\displaystyle \frac{o\left(1\right)}{n{h}_n^d}}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^2|{\mathcal{F}}_{i-1}\right]\hfill \\ \hfill & =\hfill & o\left(1\right){\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\int }_{{[-2L,2L]}^d}{\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{x}}{h_n}}+\mathbf{v}\right)\right)}^2{f}^{{\mathcal{F}}_{i-1}}(\mathbf{x}+h_n\mathbf{v})d\mathbf{v}\hfill \\ \hfill & =\hfill & o\left(1\right)\left({\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)+o\left(1\right)\right){\int }_{{[-2L,2L]}^d}{\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{x}}{h_n}}+\mathbf{v}\right)\right)}^{2}d\mathbf{v}\hfill \\ \hfill & =\hfill & o\left(1\right)\left(f\left(\mathbf{x}\right)+o\left(1\right)\right){\int }_{{[-2L,2L]}^d}{\left(K^{\left(\beta \right)}\left(\mathbf{0},\mathbf{v}\right)\right)}^{2}d\mathbf{v}\hfill \\ \hfill & =\hfill & o\left(1\right).\hfill \end{array}$$

(66)

Observe that

$$\begin{array}{ccc}{I}_{n,2}\left(\mathbf{x}\right)\hfill & =\hfill & m_{\rho ,2}\left(\mathbf{x}\right)\left({\displaystyle \frac{1}{n{h}_n^d}}\right){\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)}^2|{\mathcal{F}}_{i-1}\right]\hfill \\ \hfill & =\hfill & m_{\rho ,2}\left(\mathbf{x}\right)\left({\displaystyle \frac{1}{n{h}_n^d}}\right){\displaystyle \sum _{i=1}^n}{\int }_{{\mathbb{R}}^d}{\left(K^{\left(\beta \right)}\right)}^2\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{\mathbf{u}}{h_n}}\right)f^{{\mathcal{F}}_{i-1}}\left(\mathbf{u}\right)d\mathbf{u}\hfill \\ \hfill & =\hfill & m_{\rho ,2}\left(\mathbf{x}\right){\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\int }_{{\mathbb{R}}^d}{\left(K^{\left(\beta \right)}\left(\mathbf{0},\mathbf{v}\right)\right)}^2{f}^{{\mathcal{F}}_{i-1}}(\mathbf{x}+h_n\mathbf{v})d\mathbf{v}\hfill \\ \hfill & =\hfill & m_{\rho ,2}\left(\mathbf{x}\right){\int }_{{\mathbb{R}}^d}{\left(K^{\left(\beta \right)}\left(\mathbf{0},\mathbf{v}\right)\right)}^2\left({\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{f}^{{\mathcal{F}}_{i-1}}\left(\mathbf{x}\right)+o\left(1\right)\right)d\mathbf{v}.\hfill \end{array}$$

(67)

Combining Equations (66) and (67) we deduce

$$\begin{array}{cc}\underset{n\to \infty }{lim}{\displaystyle \sum _{i=1}^n}\mathbb{E}\left[{\xi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i)\mid {\mathcal{G}}_{i-1}\right]\hfill & \stackrel{\mathbb{P}}{=}{\Sigma }_{\rho }^2\left(\mathbf{x}\right)\hfill \\ \hfill & = m_{\rho ,2}\left(\mathbf{x}\right)f\left(\mathbf{x}\right){\int }_{{\mathbb{R}}^d}{\left(K^{\left(\beta \right)}\left(\mathbf{0},\mathbf{v}\right)\right)}^{2}d\mathbf{v}.\hfill \end{array}$$

(68)

Proof of (b).

The Lindberg condition results from Corollary 9.5.2 in [76] which implies that

$$n\mathbb{E}\left[{\chi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i){\mathbb{1}}_{\left\{\left|{\chi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\right|>ϵ\right\}}\right]\le 4n\mathbb{E}\left[{\xi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i){\mathbb{1}}_{\left\{\left|{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\right|>ϵ/2\right\}}\right].$$

Let $a>1$ and $b>1$ such that

$${\displaystyle \frac{1}{a}}+{\displaystyle \frac{1}{b}}=1.$$

Making use of Hölder and Markov inequalities one can write, for all $ϵ>0,$

$$\begin{array}{ccc}\mathbb{E}\left[{\xi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i){\mathbb{1}}_{\left\{\left|{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\right|>ϵ/2\right\}}\right]\hfill & \le \hfill & {\displaystyle \frac{\mathbb{E}{\left|{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\right|}^{2a}}{{(ϵ/2)}^{2a/b}}}.\hfill \end{array}$$

Therefore, by using the condition (7) when $m=d+1$, we obtain

$$\begin{array}{c}{\displaystyle 4n\mathbb{E}\left[{\xi }_{ni}^2({\mathbf{x}}_j,{\mathbf{X}}_i){\mathbb{1}}_{\left\{\left|{\xi }_{ni}({\mathbf{x}}_j,{\mathbf{X}}_i)\right|>ϵ/2\right\}}\right]}\hfill \\ \le {\displaystyle \frac{4}{n^{a-1}{h}_n^{ad}{(ϵ/2)}^{2a/b}}}\mathbb{E}{\left|\rho \left(Y_i\right)\left(K^{\left(\beta \right)}\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right)\right)\right|}^{2a}\hfill \\ \le {\displaystyle \frac{4}{n^{a-1}{h}_n^{ad}{(ϵ/2)}^{2a/b}}}\mathbb{E}{\left|\rho \left(Y_i\right)\left({\displaystyle \frac{C_{d+1}}{(1+h_n^{-1}\parallel \mathbf{x}-{\mathbf{X}}_i{{\parallel }_2)}^{d+1}}}\right)\right|}^{2a}\hfill \\ = {\displaystyle \frac{4C_{d+1}^{2a}}{n^{a-1}{h}_n^{(a-1)d}{(ϵ/2)}^{2a/b}}}\mathbb{E}{\left|\rho \left(Y_i\right)\right|}^{2a}.\hfill \end{array}$$

Hence, under (N.0) we infer that

$$\begin{array}{cc}4n\mathbb{E}\left[{\xi }_{ni}^2(\mathbf{x},\rho ){\mathbb{1}}_{\left\{\left|{\xi }_{ni}(\mathbf{x},\rho )\right|>ϵ/2\right\}}\right]\hfill & \le {\displaystyle \frac{4C_{d+1}^{2a}}{n^{a-1}{h}_n^{(a-1)d}{(ϵ/2)}^{2a/b}}}\hfill \\ \hfill & = O\left({\left({\displaystyle \frac{1}{n{h}_n^d}}\right)}^{a-1}\right).\hfill \end{array}$$

(69)

Combining statements (68) and (69), we achieve the proof of the theorem. □