Optimal Almost Sure Rate of Convergence for the Wavelets Estimator in the Partially Linear Additive Models

Abstract

In this article, we examine a class of partially linear additive models (PLAM) defined via a measurable mapping $\mathsf{\Psi }:{\mathbb{R}}^q\to \mathbb{R}$. More precisely, we consider $\mathsf{\Psi }\left({\mathbf{Y}}_i\right):={\mathcal{Y}}_i={\mathbf{Z}}_i^{\top }\mathit{\beta }+{\sum }_{\mathcal{l}=1}^d{m}_{\mathcal{l}}\left(X_{\mathcal{l},i}\right)+{\epsilon }_i,i=1,\dots ,n,$ where ${\mathbf{Z}}_i={(Z_{i,1},\dots ,Z_{i,p})}^{\top }$ and ${\mathbf{X}}_i={(X_{1,i},\dots ,X_{d,i})}^{\top }$ denote vectors of explanatory variables. The unknown parameter vector is $\mathit{\beta }={({\beta }_1,\dots ,{\beta }_p)}^{\top }$, and $m_1,\dots ,m_d$ are real-valued functions of a single variable whose forms are not specified. The error terms ${\epsilon }_1,\dots ,{\epsilon }_n$ are identically distributed with mean zero and finite variance ${\sigma }_{\epsilon }$, and they fulfill the condition $\mathbb{E}(\epsilon \mid \mathbf{X},\mathbf{Z})=0$ almost surely. These models are broadly applicable in finance, biology, and engineering, where capturing intricate nonlinear effects is essential. We propose an estimation method that leverages marginal integration in conjunction with linear wavelet-based techniques to obtain estimators for the unknown components $m_1,\dots ,m_d$. Under suitable regularity conditions, we establish strong uniform convergence of these estimators, demonstrating that they achieve practically relevant convergence rates. Our theoretical results indicate that these estimators converge uniformly at rates that are favorable for practical applications, underscoring the adaptability and scope of this partially linear additive model.

1. Introduction

Consider a random vector $\left(\mathbf{X},\mathbf{Y},\mathbf{Z}\right)\in {\mathbb{R}}^d\times {\mathbb{R}}^q\times {\mathbb{R}}^p$, and let $\mathsf{\Psi }:{\mathbb{R}}^q\to \mathbb{R}$ be a measurable mapping from some function class $\mathcal{F}$. We investigate a modeling structure where the relation between the response variables and the covariates is captured by a partially linear specification of the form

$$\mathsf{\Psi }\left(\mathbf{Y}\right)=\mathcal{Y}=m_0+{\mathbf{Z}}^{\top }\mathit{\beta }+m\left(\mathbf{X}\right)+\epsilon ,$$

(1)

where $m_0$ is a constant, $\mathit{\beta }\in {\mathbb{R}}^p$ is a vector of unknown coefficients, and $m(·)$ is a nonparametric function designed to capture potential nonlinear effects of the variables in $\mathbf{X}$. The random error term $\epsilon$ represents noise. Throughout, the transpose of any vector $\mathbf{V}$ will be written as ${\mathbf{V}}^{\top }$. Model (1) is notably flexible, accommodating a variety of established statistical frameworks as particular cases (see, for example, [1]). In practice, it is common to collect a large set of variables, which then requires the elimination of those that are not influential before finalizing the model. Hence, the selection of significant variables is a key consideration in both parametric and nonparametric regression contexts; for a comprehensive discussion, see [2]. One illustrative application concerns the study of factors influencing East Germans’ decisions to migrate to West Germany one year after reunification [3,4], where certain covariates (e.g., discrete variables) may exert linear effects on the response, whereas others act in a nonlinear manner. Similar scenarios arise in various domains, such as civil engineering materials [5], the Pima Indian diabetes dataset [6,7], and investigations related to guidelines for urinary incontinence discussion and evaluation [8].

Introducing a linear component ${\mathbf{Z}}^{\top }\mathit{\beta }$ alongside a nonparametric function $m\left(\mathbf{X}\right)$ in the model (1) offers an appealing compromise between purely parametric and fully nonparametric approaches. Parametric models are easy to use but can be severely biased if their assumptions are violated, whereas nonparametric techniques can accommodate complex data structures but often need large sample sizes for reliable inference. By incorporating both a linear and a nonparametric term, the partially linear formulation in (1) combines the interpretability and efficiency of parametric methods with the flexibility to capture nonlinear relationships via $m\left(\mathbf{X}\right)$. A salient special case is isotonic regression, which arises if $\mathsf{\Psi }\left(y\right)=y$ and the linear term is omitted. This isotonic setting dates back to the pioneering work of [9] and remains a rich area of investigation in statistics.

The partially linear framework, initially proposed by [10] for the classical situation $\mathsf{\Psi }\left(y\right)=y$, has been extensively used in fields such as economics, epidemiology, engineering, and the life sciences. Foundational contributions by [11,12,13] deepened the theoretical underpinnings for estimation and inference, motivating a large body of ensuing research. In recent decades, numerous efforts have extended (1) to the case $\mathsf{\Psi }\left(y\right)=y$ and refined inference for both the parametric and nonparametric components; see [14,15,16,17,18,19,20,21,22], as well as the comprehensive volume [23]. More contemporary developments appear in [24,25,26,27,28,29,30,31,32,33].

When parametric models are correctly specified, they can yield a strong performance but may introduce significant biases if crucial assumptions are violated. In contrast, fully nonparametric smoothing does not rely on strict parametric constraints, though it can be hindered by the “curse of dimensionality” in moderate or high dimensions. To address this challenge, refs. [34,35] introduced additive models, which decompose multivariate nonparametric relationships into sums of lower-dimensional functions. This decomposition promotes interpretability, reduces computational overhead, and can lead to performance gains over fully nonparametric techniques. Detailed expositions of additive models and their development appear in [36,37], and related sources.

In our specific context, we handle the potentially high dimension of $m\left(\mathbf{X}\right)$ in (1) by imposing an additive structure. Concretely, we replace $m\left(\mathbf{X}\right)$ with

$$\mathsf{\Psi }\left(\mathbf{Y}\right)=m_0+{\mathbf{Z}}^{\top }\mathit{\beta }+\sum _{\mathcal{l}=1}^d{m}_{\mathcal{l}}\left(X_{\mathcal{l}}\right)+\epsilon ,$$

(2)

where $X_{\mathcal{l}}$ denotes the ℓ-th component of $\mathbf{X}$ and $m_{\mathcal{l}}(·)$ captures its influence on $\mathsf{\Psi }\left(\mathbf{Y}\right)$. This partially linear additive design retains a transparent linear part ${\mathbf{Z}}^{\top }\mathit{\beta }$ while permitting each covariate $X_{\mathcal{l}}$ to appear in a potentially nonlinear form. Empirical results such as those in [38] suggest that exploiting this additive framework can substantially improve efficiency. In related work, refs. [39,40] illustrate how additional constraints in semiparametric models can also boost estimation precision.

Researchers have long taken an interest in partially linear additive models across diverse domains. Kernel-based techniques are frequently employed to estimate (2), partly because of their relative ease of implementation and strong theoretical foundations. In-depth reviews of kernel-based methods can be found in [41,42,43]. In parallel, ref. [30] introduced a fusion-penalized inverse probability weighted least squares method designed for missing responses in partially linear additive setups with latent subgroups. Meanwhile, ref. [32] proposed a smooth-threshold generalized method of moments estimator and a variable selection procedure for partially linear additive spatial autoregressive models, utilizing basis expansions for the nonparametric components.

Recent work by [29] examined partially linear additive models in pooled data settings, supporting their approach with simulations and an application based on environmental health information from the National Health and Nutrition Examination Survey. Independently, ref. [26] introduced a weighted quantile regression methodology with inverse probability weighting to address selective data retention, establishing large-sample behavior for both parametric and nonparametric components. Other significant studies include [44], which proposed a scalar-on-function paradigm incorporating nonparametric functional predictors and semiparametric covariates, and [45], which developed generalized partial functional linear additive models (GPFLAM) and employed a quasi-likelihood approach. Additionally, ref. [46] investigated partially linear additive dynamic panels with fixed effects and autocorrelation, and [47] examined transformation models for right-censored survival data under the same partially linear additive framework. Wavelet-based methods have also been explored; for example, ref. [48] advanced a spline-backfitted kernel smoothing algorithm displaying oracle-type properties under stationarity and geometric mixing conditions.

Although kernel-based estimators are effective, they may face complications near boundaries or with discontinuous target functions. Wavelet methods inherently address such localized irregularities by allowing for abrupt changes and flexible boundary treatments. Foundational contributions to wavelet theory—see [49,50,51,52] and the summary in [53]—established a robust mathematical basis for wavelet-based estimation. Important theoretical expansions of this framework can be found in [54,55,56,57].

Wavelet techniques have seen extensive application in density estimation and regression. For instance, ref. [58] analyzed a linear wavelet estimator, proving its strong consistency under $L_2$-type smoothing with the help of sophisticated empirical process tools. Subsequently, ref. [59] introduced statistical inference procedures for wavelet estimators of symmetric positive-definite function curves, employing a log-Euclidean metric to maintain key properties such as positive-definiteness and permutation invariance. Wavelet-based methods have also been used to detect and categorize local symmetry in images [60], to characterize refinement masks for symmetric wavelets meeting particular sum rules [61], and to examine DNA sequence symmetries [62]. Further advances include [63], which developed nearly frame-like wavelets with specified approximation properties, and [64], focusing on block thresholding for nonparametric regression. Broader contexts of wavelet transforms involving symmetry groups or tube domains are discussed in [65,66], while [67] investigated perfect reconstruction of two-dimensional signals on hexagonal grids with enhanced regularity. In [68,69], a methodology based on the k-th correlation coefficient was introduced to verify whether the true model error in partial linear settings follows a symmetric distribution.

A notable takeaway from this body of literature is that linear wavelet estimation can asymptotically achieve performance comparable to the well-known nonlinear hard-threshold wavelet estimator developed by [56], reinforcing the practical advantages of wavelet-based approaches. Under $\alpha$-mixing processes, ref. [70] showed that wavelet estimators for densities on bounded domains in ${\mathbb{R}}^d$ attain optimal or near-optimal almost sure convergence rates, mirroring those in the i.i.d. context. Extending this line of inquiry, ref. [71] demonstrated strong uniform convergence for wavelet regression estimators on compact subsets of ${\mathbb{R}}^d$. Discussions of linear wavelet estimators appear in [70,71,72,73,74].

That said, the distributional properties of wavelet estimators are comparatively less well-established. Early progress was made by [75], who proved almost sure uniform convergence and asymptotic normality for wavelet estimators of density and regression functions under ergodic ${\mathbb{R}}^d$-valued processes, later extending these insights to continuous-time ergodic processes in [76]. Subsequent work has refined and broadened these findings; see, for example, [77,78,79,80]. Collectively, this evolving literature underscores the versatility and strength of wavelet-based methods in a wide variety of applications.

• Contributions of this paper. In this study, we seek to establish the optimal almost sure convergence rate of wavelet estimators for each component function in partially linear additive regression models (PLAM). Our work substantially extends and refines earlier theoretical findings, in particular those reported by [43,80,81,82]. Rather than merely synthesizing existing methodologies, we rigorously examine the dual challenges posed by mixing dependence structures—commonly arising in both time series and spatial data—and the complexities introduced by wavelet expansions. To meet these challenges, we employ advanced large-sample theory, the mixing conditions proposed by [71], and the sophisticated wavelet expansion techniques discussed in [79]. By integrating these elements, we can derive optimal almost sure convergence results that remain valid beyond the standard assumption of independent and identically distributed observations. Crucially, even in the simpler i.i.d. setting, the optimal almost sure convergence rate of wavelet estimators for component functions in additive regression models has not been fully determined. Addressing this unresolved issue constitutes a key contribution to our work and highlights the significant theoretical and technical developments required to advance the field. In previous works [71,75,76,78,79,83], wavelet-based techniques have been employed to address multivariate regression. However, in these studies, the convergence rate was shown to depend on the dimension of the covariates, causing performance to deteriorate as the number of covariates increases. As a result, larger sample sizes become necessary to draw robust conclusions. In contrast, the principal contribution of the present paper lies in achieving a dimension-free convergence rate, which offers a significant advantage in practical applications.
• Paper organization. The subsequent sections provide the essential background, introduce the primary model, and present the theoretical findings. In Section 2, we offer a concise overview of wavelets, Besov spaces, and wavelet estimators, which collectively serve as the foundational framework for our later developments. Section 3 then elaborates on the partially linear additive model and specifies the estimators that constitute the core of our analysis. Next, Section 4 outlines the principal assumptions and formally states our main theoretical contributions, including detailed assertions regarding the asymptotic normality of each additive component. Section 5 concludes by underscoring the key results and proposing directions for further study. To maintain a coherent narrative, the complete proofs of the central theorems are deferred to Section 6, while the Appendix A consolidates additional lemmas and other technical derivations, thereby preserving the continuity of the main exposition.

2. Wavelets and Besov Space

In this section, we introduce the notation associated with wavelets and Besov spaces. We begin by providing a brief overview of wavelet multiresolution theory as developed in [49], and further elaborated in [70,71,72]. Consider a multiresolution analysis ${\left\{V_j\right\}}_{j=1}^{\infty }$ of the space ${\mathbf{L}}^2\left({\mathbb{R}}^d\right)$, where ${\mathbf{L}}^p\left({\mathbb{R}}^d\right)$ (for $1\le p<\infty$) is the set of all measurable functions $f:{\mathbb{R}}^d\to \mathbb{R}$ satisfying

$${\int }_{{\mathbb{R}}^d}{|f\left(x\right)|}^{p}dx<\infty$$

and is equipped with the norm

$${\parallel f\parallel }_{{\mathbf{L}}^p}={\left({\int }_{{\mathbb{R}}^d}{|f\left(x\right)|}^{p}dx\right)}^{1/p}.$$

A multiresolution analysis is defined as a decomposition of ${\mathbf{L}}^2\left({\mathbb{R}}^d\right)$ into an increasing sequence of closed subspaces $\{V_j:j\in \mathbb{Z}\}$, satisfying the following properties:

1.

For all integers j,

$$V_j\subseteq V_{j+1}.$$

2.

$$\bigcap _j{V}_j=\left\{0\right\},\overline{{\bigcup }_j{V}_j}={\mathbf{L}}^2\left({\mathbb{R}}^d\right).$$

3.

For any $f(·)\in V_j$,

$$f(2·)\in V_{j+1},\mathrm{and}f(·+k)\in V_j\mathrm{for}\mathrm{all}k\in {\mathbb{Z}}^d.$$

4.

There exists a scaling function $\varphi \in {\mathbf{L}}^2\left({\mathbb{R}}^d\right)$, normalized such that

$${\int }_{{\mathbb{R}}^d}\varphi \left(x\right)dx=1,$$

and the set

$$\{{\varphi }_k\left(x\right)=\varphi (x-k):k\in {\mathbb{Z}}^d\}$$

forms an orthonormal basis of $V_0$. Hence,

$$V_0=\overline{\mathrm{span}}\{\varphi (·-k):k\in {\mathbb{Z}}^d\}.$$

By scaling, we obtain an orthonormal basis for $V_j$:

$$\{{\varphi }_{j,k}\left(x\right)=2^{jd/2}\varphi (2^{j}x-k):k\in {\mathbb{Z}}^d\}.$$

Throughout this paper, we assume that $\varphi (·)$ is r-regular ($r\ge 1$), meaning that $\varphi \in {\mathcal{C}}^r\left({\mathbb{R}}^d\right)$ and satisfies, for each $i>0$ and all multi-indices $\beta =({\beta }_1,\dots ,{\beta }_d)$ with

$$|\beta |=\sum _{i=1}^d{\beta }_i\le r,$$

the bound

$$|D^{\beta }\varphi \left(x\right)|\le {\displaystyle \frac{C_i}{{(1+\parallel x\parallel )}^i}},$$

(3)

where

$$D^{\beta }\varphi \left(x\right)={\displaystyle \frac{{\partial }^{|\beta |}\varphi }{\partial x_1^{{\beta }_1}\cdots \partial x_d^{{\beta }_d}}}\left(x\right),$$

and $C_i$ is a constant depending only on i.

Define the wavelet space $W_j$ by

$$V_j\oplus W_j=V_{j+1}.$$

Following the construction in [50], one obtains $N=2^d-1$ wavelet functions

$$\{{\psi }_i:i=1,\dots ,N\}$$

associated with the scaling function $\varphi$, such that the following applies:

1.

Orthonormal Basis for $W_0$:

$$\{{\psi }_i(x-k):k\in {\mathbb{Z}}^d,i=1,\dots ,N\}$$

forms an orthonormal basis for $W_0$.

2.

Orthonormal Basis for ${\mathbf{L}}^2\left({\mathbb{R}}^d\right)$:

$$\{{\psi }_{i,j,k}\left(x\right)=2^{jd/2}{\psi }_i(2^{j}x-k):j\in \mathbb{Z},k\in {\mathbb{Z}}^d,i=1,\dots ,N\}$$

forms an orthonormal basis for ${\mathbf{L}}^2\left({\mathbb{R}}^d\right)$.

3.

Regularity and Compact Support: Each ${\psi }_i(·)$ shares the same regularity as $\varphi (·)$, and both $\varphi$ and ${\psi }_i$ have compact support contained within ${[-L,L]}^d$ for some $L>0$.

Starting from some integer $j_0$, any $f(·)\in {\mathbf{L}}^2\left({\mathbb{R}}^d\right)$ can be expanded as

$$f\left(x\right)=\sum _{k\in {\mathbb{Z}}^d}{a}_{j_0,k}{\varphi }_{j_0,k}\left(x\right)+\sum _{i=1}^{2^d-1}\sum _{j=j_0}^{\infty }\sum _{k\in {\mathbb{Z}}^d}{b}_{i,j,k}{\psi }_{i,j,k}\left(x\right),$$

(4)

where

$$a_{j_0,k}={\int }_{{\mathbb{R}}^d}f\left(u\right){\varphi }_{j_0,k}\left(u\right)du,$$

and

$$b_{i,j,k}={\int }_{{\mathbb{R}}^d}f\left(u\right){\psi }_{i,j,k}\left(u\right)du.$$

For any $\mathcal{l}\ge j_0$, the orthogonal projection of f onto $V_{\mathcal{l}}$ can be written in two equivalent forms:

$$\begin{array}{ccc}\hfill \left(P_{V_{\mathcal{l}}}f\right)\left(x\right)& =& \sum _{k\in {\mathbb{Z}}^d}{a}_{\mathcal{l},k}{\varphi }_{\mathcal{l},k}\left(x\right)\hfill \\ & =& \sum _{k\in {\mathbb{Z}}^d}{a}_{j_0,k}{\varphi }_{j_0,k}\left(x\right)+\sum _{j=j_0}^{\mathcal{l}}\sum _{i=1}^N\sum _{k\in {\mathbb{Z}}^d}{b}_{i,j,k}{\psi }_{i,j,k}\left(x\right).\hfill \end{array}$$

Besov spaces ${\mathbf{B}}_{s,p,q}$ are of central importance in statistical estimation and approximation theory as they provide a flexible framework for characterizing smoothness properties; for instance, see [84,85,86,87,88]. They include a wide variety of both homogeneous and inhomogeneous function spaces frequently employed in statistical analysis. Using wavelet coefficients, ref. [49] characterizes the Besov space as follows. Consider a real-valued smoothness parameter $0<s<r$ and let $f\in {\mathbf{L}}^p\left({\mathbb{R}}^d\right)$. Then $f\in {\mathbf{B}}_{s,p,q}$ if and only if

(B.1)

$$J_{s,p,q}\left(f\right)={\parallel P_{V_0}f\parallel }_{{\mathbf{L}}^p}+{\left(\sum _{j\ge 0}{\left(2^{js}{\parallel P_{W_j}f\parallel }_{{\mathbf{L}}^p}\right)}^q\right)}^{1/q}<\infty ,$$

(B.2)

$$J_{s,p,q}^{\prime }\left(f\right)={\parallel a_0·\parallel }_{l_p}+{\left(\sum _{j\ge 0}{\left(2^{j(s+d(1/2-1/p\left)\right)}{\parallel b_j·\parallel }_{l_p}\right)}^q\right)}^{1/q}<\infty ,$$

where

$$\parallel a_0{·\parallel }_{l_p}={\left(\sum _{k\in {\mathbb{Z}}^d}{|a_{0,k}|}^p\right)}^{1/p},$$

and

$$\parallel b_j{·\parallel }_{l_p}={\left(\sum _{i=1}^N\sum _{k\in {\mathbb{Z}}^d}{|b_{i,j,k}|}^p\right)}^{1/p}.$$

For $q=\infty$, the usual sup-norm modification applies. Additional equivalent characterizations and advantages of Besov spaces in approximation theory and statistics can be found in [56,89,90,91] and in the Appendix A.

Linear Wavelets Regression Estimators

We begin by introducing the notation and definitions that will be used in the following sections. We start with a reminder of the definition of strong mixing. To this end, we introduce the following notation.

Let ${\mathcal{F}}_i^k\left(Z\right)$ denote the $\sigma$-algebra generated by the random variables $\{Z_j:i\le j\le k\}$. The strong mixing coefficient as introduced by [92] is defined as follows. For additional details, the reader may consult [93].

Definition 1. 

Let $Z=\{Z_i:i=1,2,\dots \}$ be a strictly stationary sequence of random variables. For a positive integer n, we set

$$\alpha \left(n\right)=sup\left\{|\mathbb{P}(A\cap B)-\mathbb{P}\left(A\right)\mathbb{P}\left(B\right)|:A\in {\mathcal{F}}_1^k\left(Z\right),B\in {\mathcal{F}}_{k+n}^{\infty }\left(Z\right),k\in {\mathbb{N}}^{*}\right\}.$$

The sequence Z is said to be α-mixing (or strongly mixing) if the mixing coefficient $\alpha \left(n\right)$ converges to 0 as $n\to \infty$.

In the study of mixing conditions, $\alpha$-mixing emerges as a relatively weak yet broadly applicable criterion, encompassing a wide range of stochastic processes, including many time series models. The seminal contributions of [94,95] established sufficient conditions for linear processes to be $\alpha$-mixing. These works further demonstrated that both linear autoregressive and bilinear models possess strong mixing properties, with mixing coefficients that decay at an exponential rate. In addition, ref. [96] highlighted the critical role of $\alpha$-mixing in understanding nonlinear time series models, including aspects of geometric ergodicity, a concept further explored and extended in [75]. Along similar lines, ref. [97] showed that functional autoregressive processes can achieve geometric ergodicity under appropriate conditions.

Moreover, the studies [98,99] demonstrated that, even with minimal assumptions, autoregressive conditional heteroscedastic (ARCH) processes and nonlinear additive autoregressive models incorporating exogenous variables remain both stationary and $\alpha$-mixing.

Next, let $\{{\mathbf{X}}_i,Y_i\}$ be jointly stationary processes, and let $\mathsf{\Psi }(·)$ be a Borel measurable function on the real line. Define

$$m(\mathbf{x},\mathsf{\Psi })=\mathbb{E}[\mathsf{\Psi }\left(Y_1\right)\mid {\mathbf{X}}_1=\mathbf{x}],$$

(5)

for $\mathbf{x}\in {\mathbb{R}}^d$, whenever it exists. That is, we assume

$$\mathbb{E}\left(|\mathsf{\Psi }(Y_1)|\right)<\infty .$$

Our framework includes several special cases of the regression model (5), depending on the choice of the function $\mathsf{\Psi }(·)$. Notable examples include the following:

• $$\mathsf{\Psi }\left(Y\right)=\mathbb{1}\{Y\le y\},$$

  which yields the conditional distribution function of $Y_1$ given ${\mathbf{X}}_1=\mathbf{x}$.
• $$\mathsf{\Psi }\left(Y\right)=Y^k,$$

  which provides the k-th conditional moment of $Y_1$ given ${\mathbf{X}}_1=\mathbf{x}$.

Throughout, we shall assume

$$\mathbb{E}\left(|\mathsf{\Psi }\left(Y_1\right){|}^p\right)<\infty ,p\ge 2.$$

(6)

Following the approach of previous works, our estimator of $m(\mathsf{\Psi },\mathbf{x})$ will be obtained as the ratio of wavelet estimators of

$$g(\mathsf{\Psi },\mathbf{x})=m(\mathbf{x},\mathsf{\Psi })f_{\mathbf{X}}\left(\mathbf{x}\right)$$

and $f_{\mathbf{X}}\left(\mathbf{x}\right),$ where $f_{\mathbf{X}}\left(\mathbf{x}\right)$ is the unknown density function of $\mathbf{X}$. First, we recall that from (4), the linear wavelet estimator of $f_{\mathbf{X}}\left(\mathbf{x}\right)\in {\mathbf{L}}^2\left({\mathbb{R}}^d\right)$ is given by

$${\widehat{f}}_{\mathbf{X}}\left(\mathbf{x}\right)=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{\tau ,\mathbf{k}}{\varphi }_{\tau ,\mathbf{k}}\left(\mathbf{x}\right),$$

(7)

or equivalently,

$${\widehat{f}}_{\mathbf{X}}\left(\mathbf{x}\right)=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{j_0,\mathbf{k}}{\varphi }_{j_0,\mathbf{k}}\left(\mathbf{x}\right)+\sum _{j=j_0}^{\tau }\sum _{i=1}^N\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{b}}_{ij\mathbf{k}}{\psi }_{i,j,\mathbf{k}}\left(\mathbf{x}\right),$$

(8)

where the estimators of the coefficients $\left\{a_{\tau ,\mathbf{k}}\right\}$ and $\left\{b_{ij\mathbf{k}}\right\}$ are given by

$${\widehat{a}}_{\tau ,\mathbf{k}}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\varphi }_{\tau ,\mathbf{k}}\left({\mathbf{X}}_i\right),\mathrm{and}{\widehat{b}}_{ij\mathbf{k}}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\psi }_{i,j,\mathbf{k}}\left({\mathbf{X}}_i\right),$$

(9)

for any fixed $j_0\le \tau$, where $\tau =\lambda \left(n\right)$ represents the resolution level, a strictly positive integer depending on n and tending to infinity at a specified rate.

By the regularity and compact support conditions on $\varphi (·)$ and $\psi (·)$, the above sums are finite for any fixed $\mathbf{x}$, which is crucial for practical implementation. The supports of $\varphi (·)$ and ${\psi }_i(·)$ grow only as a function of their degree of differentiability; see [50]. Next, since $g(\mathsf{\Psi },·)\in {\mathbf{L}}^2\left({\mathbb{R}}^d\right)$, it admits a wavelet expansion as well. Thus, a linear wavelet estimator for $g(\mathsf{\Psi },\mathbf{x})$ can be written as

$${\widehat{g}}_n(\mathsf{\Psi },\mathbf{x})=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{\tau ,\mathbf{k}}^{\prime }{\varphi }_{\tau ,\mathbf{k}}\left(\mathbf{x}\right),$$

(10)

or equivalently,

$${\widehat{g}}_n(\mathsf{\Psi },\mathbf{x})=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{j_0,\mathbf{k}}^{\prime }{\varphi }_{j_0,\mathbf{k}}\left(\mathbf{x}\right)+\sum _{j=j_0}^{\tau }\sum _{i=1}^N\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{b}}_{ij\mathbf{k}}^{\prime }{\psi }_{i,j,\mathbf{k}}\left(\mathbf{x}\right),$$

(11)

where the coefficient estimators are

$${\widehat{a}}_{\tau ,\mathbf{k}}^{\prime }={\displaystyle \frac{1}{n}}\sum _{i=1}^n\mathsf{\Psi }\left(Y_i\right){\varphi }_{\tau ,\mathbf{k}}\left({\mathbf{X}}_i\right),$$

and

$${\widehat{b}}_{ij\mathbf{k}}^{\prime }={\displaystyle \frac{1}{n}}\sum _{i=1}^n\mathsf{\Psi }\left(Y_i\right){\psi }_{i,j,\mathbf{k}}\left({\mathbf{X}}_i\right),$$

for any $j_0\le \tau$. From a practical perspective, the multiresolution properties of wavelets facilitate computationally efficient estimation algorithms that are memory-efficient. In particular, the bandwidth is chosen in the form $2^j$, where j can be selected from a small set of values (e.g., three or four); see [57].

Returning to the wavelet-based estimation, for $\mathbf{u},\mathbf{v}\in {\mathbb{R}}^d$, we define the kernel $K(\mathbf{u},\mathbf{v})$ by

$$K(\mathbf{u},\mathbf{v}):=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}\varphi (\mathbf{u}-\mathbf{k})\varphi (\mathbf{v}-\mathbf{k}).$$

(12)

Since

$$|\varphi \left(\mathbf{x}\right)|\le {\displaystyle \frac{A_{d+1}}{{(1+\parallel \mathbf{x}\parallel )}^{d+1}}},$$

the kernel function $K(·,·)$ defined in (12) converges uniformly in $\mathbf{u},\mathbf{v}\in {\mathbb{R}}^d$. In particular,

$$\underset{\mathbf{u},\mathbf{v}\in {\mathbb{R}}^d}{sup}\left|\sum _{\mathbf{k}\in {\mathbb{Z}}^d}\varphi (\mathbf{u}-\mathbf{k})\varphi (\mathbf{v}-\mathbf{k})\right|<\infty .$$

Moreover, for any $j\ge 1$, there exists a constant $C_j$ such that ([49], p. 33)

$$|K(\mathbf{v},\mathbf{u})|\le {\displaystyle \frac{C_j}{{(1+\parallel \mathbf{v}-\mathbf{u}\parallel )}^j}}.$$

(13)

From (14), it follows that

$${\int }_{{\mathbb{R}}^d}{|K(\mathbf{v},\mathbf{u})|}^{j}d\mathbf{v}\le G_j\left(d\right),$$

(14)

where

$$G_j\left(d\right)=2{\pi }^{d/2}{\displaystyle \frac{\mathsf{\Gamma }\left(d\right)\mathsf{\Gamma }(j+d(j-1\left)\right)}{\mathsf{\Gamma }(d/2)\mathsf{\Gamma }\left(\right(d+1\left)j\right)}}{C}_{d+1}^j,$$

and $\mathsf{\Gamma }\left(t\right)$ denotes the Gamma function,

$$\mathsf{\Gamma }\left(t\right):={\int }_0^{\infty }{y}^{t-1}{e}^{-y}dy.$$

Furthermore, under the assumptions on $\varphi (·)$, for any partial derivative of order $|\beta |=1$, we have ([49], p. 33)

$$\left|{\displaystyle \frac{\partial K(\mathbf{u},\mathbf{v})}{\partial {\mathbf{u}}_i}}\right|\le {\displaystyle \frac{C_2}{{(1+\parallel \mathbf{u}-\mathbf{v}\parallel )}^2}},i=1,\dots ,d.$$

(15)

Combining (7), (9), and (12), we see that the linear estimates of the regression function ${\widehat{m}}_n(\phi ,\mathbf{x})$ can be expressed as an extended kernel estimator:

$$K_{h_n,\mathbf{x}}\left({\mathbf{X}}_i\right)={\displaystyle \frac{1}{h_n^d}}K\left({\displaystyle \frac{\mathbf{x}}{h_n}},{\displaystyle \frac{{\mathbf{X}}_i}{h_n}}\right),\mathrm{with}{h}_n=2^{-\lambda \left(n\right)}.$$

Thus, we have

$${\widehat{m}}_n(\mathsf{\Psi },\mathbf{x})=\left\{\begin{array}{cc}{\displaystyle \frac{{\widehat{g}}_n(\mathsf{\Psi },\mathbf{x})}{{\widehat{f}}_{\mathbf{X}}\left(\mathbf{x}\right)}=\frac{{\widehat{m}}_{2,n}(\mathsf{\Psi },\mathbf{x})}{{\widehat{m}}_{1,n}\left(\mathbf{x}\right)},}\hfill & \mathrm{if}{\widehat{f}}_{\mathbf{X}}\left(\mathbf{x}\right)\ne 0,\hfill \\ {\displaystyle \frac{1}{n}\sum _{i=1}^n\phi \left(Y_i\right),}\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

Here,

$${\widehat{m}}_{1,n}(\mathsf{\Psi },\mathbf{x})={\displaystyle \frac{{\displaystyle \sum _{i=1}^n{K}_{h_n,\mathbf{x}}\left({\mathbf{X}}_i\right)}}{{\displaystyle n\mathbb{E}\left[K_{h_n,\mathbf{x}}\left({\mathbf{X}}_1\right)\right]}}}$$

and

$${\widehat{m}}_{2,n}(\mathsf{\Psi },\mathbf{x})={\displaystyle \frac{{\displaystyle \sum _{i=1}^n\mathsf{\Psi }\left(Y_i\right)K_{h_n,\mathbf{x}}\left({\mathbf{X}}_i\right)}}{{\displaystyle n\mathbb{E}\left[K_{h_n,\mathbf{x}}\left({\mathbf{X}}_1\right)\right]}}}.$$

This formulation neatly integrates wavelet-based estimation of regression functions with kernel-like estimation procedures.

3. Presentation of Estimators

Consider a stationary sequence ${\left\{({\mathbf{X}}_i,{\mathbf{Y}}_i,{\mathbf{Z}}_i)\right\}}_{i\ge 1}$ of random replicates from the random vector $(\mathbf{X},\mathbf{Y},\mathbf{Z})$. Let $f_{\mathbf{X}\mathbf{Y}\mathbf{Z}}(·)$ be the joint density of $(\mathbf{X},\mathbf{Y},\mathbf{Z})$ with respect to the Lebesgue measure, and let $f_{\mathbf{X}}(·)$, $f_{\mathbf{Y}}(·)$, and $f_{\mathbf{Z}}(·)$ denote the corresponding marginal densities. For any $\mathbf{x}\in {\mathbb{R}}^d$, we define the kernel estimator of $f_{\mathbf{X}}(·)$ by

$$f_n\left(\mathbf{x}\right)={\displaystyle \frac{2^{\lambda \left(n\right)d}}{n}}\sum _{i=1}^{n}K\left(2^{\lambda \left(n\right)}\mathbf{x},2^{\lambda \left(n\right)}{\mathbf{X}}_i\right),$$

where $K(·)$ is a bounded, non-negative kernel function integrating to 1, and $\lambda \left(n\right)$ controls the bandwidth growth rate.

For simplicity in what follows, we assume $m_0=0$. Observing that model (1) can be recast for a measurable function $\mathsf{\Psi }(·)\in \mathcal{F}$ as

$$\mathsf{\Psi }\left(\mathbf{Y}\right)-{\mathbf{Z}}^{\top }\mathit{\beta }=m(\mathsf{\Psi },\mathbf{X})+\epsilon ,$$

(16)

we propose an internal estimator of the nonparametric component in (16). For any $\mathbf{x}\in {\mathbb{R}}^d$, this estimator is given by

$${\widehat{m}}_n^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{x},h)=\sum _{i=1}^n{\displaystyle \frac{2^{\lambda \left(n\right)d}\left(\mathsf{\Psi }\left({\mathbf{Y}}_i\right)-{\mathbf{Z}}_i^{\top }\mathit{\beta }\right)}{n{f}_n\left({\mathbf{X}}_i\right)}}K\left(2^{\lambda \left(n\right)}\mathbf{x},2^{\lambda \left(n\right)}{\mathbf{X}}_i\right),$$

(17)

where $h=2^{-\lambda \left(n\right)}$ is a smoothing parameter tending to zero as specified by Condition (H.2).

Since ${\widehat{m}}_n^{\mathit{\beta }}(·)$ depends on the unknown $\mathit{\beta }$, we recognize that the true function $m(·)$ in (16) may also naturally depend on $\mathit{\beta }$. In particular, when $m(·)$ has an additive form, we can write

$$m_{add}^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{x})=\sum _{\mathcal{l}=1}^d{m}_{\mathcal{l}}^{\mathit{\beta }}(\mathsf{\Psi },x_{\mathcal{l}}),$$

where $x_{\mathcal{l}}$ is the ℓ-th component of $\mathbf{x}$. To ensure identifiability of each additive component $m_{\mathcal{l}}(·)$, we impose

$$\mathbb{E}\left[m_{\mathcal{l}}^{\mathit{\beta }}(\mathsf{\Psi },X_{\mathcal{l}})\right]=0\mathrm{for}1\le \mathcal{l}\le d.$$

There are various approaches to estimating the additive components in regression models. Two notable examples are the marginal integration technique (see, for instance, [100,101,102,103,104]) and back-fitting algorithms (cf. [105,106], and references therein). While back-fitting poses more intricate asymptotic challenges (though progress is reported in [107,108]), marginal integration methods tend to exhibit more transparent asymptotic properties, making them particularly appealing in statistical and econometric contexts.

Guided by these advantages, we adopt a marginal integration framework to estimate the linear parameter $\mathit{\beta }$ through a least squares criterion. Crucially, our estimators require only kernel functions, smoothing parameters, and the density estimators, avoiding the need for complex optimization techniques or weighting schemes that depend on unknown quantities (as is sometimes required in other methods; see, e.g., [109]).

3.1. The Marginal Integration Method

In this work, we rely on marginal integration to build our estimators. To outline this approach, let us first establish some notation. For $1\le \mathcal{l}\le d$, let

$${\mathbf{x}}_{-\mathcal{l}}=(x_1,\dots ,x_{\mathcal{l}-1},x_{\mathcal{l}+1},\dots ,x_d).$$

We then define the functions

$$q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)=\prod _{\begin{array}{c}j=1\\ j\ne \mathcal{l}\end{array}}^d{q}_j\left(x_j\right)\mathrm{and}q\left(\mathbf{x}\right)=\prod _{\mathcal{l}=1}^d{q}_{\mathcal{l}}\left(x_{\mathcal{l}}\right),$$

where $q_{\mathcal{l}}$ is a known univariate density for each $\mathcal{l}\in \{1,\dots ,d\}$. Throughout, all integrals over continuous variables are assumed to be taken with respect to the Lebesgue measure. By applying marginal integration, the additive regression function can be represented as

$$m_{add}^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{x})=\sum _{\mathcal{l}=1}^d{\eta }_{\mathcal{l}}^{\mathit{\beta }}(\mathsf{\Psi },x_{\mathcal{l}})+{\int }_{{\mathbb{R}}^d}{m}_{add}^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{z})q\left(\mathbf{z}\right)d\mathbf{z},$$

(18)

where each component ${\eta }_{\mathcal{l}}^{\mathit{\beta }}$ is given by

$${\eta }_{\mathcal{l}}^{\mathit{\beta }}(\mathsf{\Psi },x_{\mathcal{l}})={\int }_{{\mathbb{R}}^{d-1}}{m}_{add}^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{x})q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)d{\mathbf{x}}_{-\mathcal{l}}-{\int }_{{\mathbb{R}}^d}{m}_{add}^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{x})q\left(\mathbf{x}\right)d\mathbf{x}.$$

(19)

For notational convenience, we will often write $m_{add}(·)$ instead of $m_{add}^{\mathit{\beta }}(·)$ and ${\eta }_{\mathcal{l}}(·)$ instead of ${\eta }_{\mathcal{l}}^{\mathit{\beta }}(·)$.

Given the parameter $\mathit{\beta }$, we define an estimator of $m_{add}(·)$ via marginal integration as follows:

$${\widehat{m}}_{add}^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{x})=\sum _{\mathcal{l}=1}^d{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}(\mathsf{\Psi },x_{\mathcal{l}})+{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{z})q\left(\mathbf{z}\right)d\mathbf{z},$$

(20)

where

$${\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}(\mathsf{\Psi },x_{\mathcal{l}})={\int }_{{\mathbb{R}}^{d-1}}{\widehat{m}}_n^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{x})q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)d{\mathbf{x}}_{-\mathcal{l}}-{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathit{\beta }}(\mathsf{\Psi },\mathbf{x})q\left(\mathbf{x}\right)d\mathbf{x}.$$

(21)

Throughout the remainder of the paper, we will simplify the notation by writing ${\widehat{m}}_{add}(·)$ for ${\widehat{m}}_{add}^{\mathit{\beta }}(·)$, ${\widehat{m}}_n(·)$ for ${\widehat{m}}_n^{\mathit{\beta }}(·)$, and ${\widehat{\eta }}_{\mathcal{l}}(·)$ for ${\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}(·)$. Consequently, once $\mathit{\beta }$ has been estimated, we obtain a fully specified estimator ${\widehat{m}}_{add}(·)$ for the additive regression function.

3.2. Estimation of the Model’s Parameter

Consider the following partially linear additive regression model:

$$\begin{array}{c}\hfill \mathsf{\Psi }\left(\mathbf{Y}\right)={\mathbf{Z}}^{\top }\mathit{\beta }+m_{add}\left(\mathbf{X}\right)+\epsilon .\end{array}$$

Throughout the subsequent discussion, for $\mathcal{l}=1,\dots ,d$, we define

$$K_{\mathcal{l}}\left(u_{\mathcal{l}}\right)={\int }_{{\mathbb{R}}^{d-1}}K\left(\mathbf{u}\right)d{\mathbf{u}}_{-\mathcal{l}}\mathrm{and}{K}_{-\mathcal{l}}\left({\mathbf{u}}_{-\mathcal{l}}\right)={\int }_{\mathbb{R}}K\left(\mathbf{u}\right)d{u}_{\mathcal{l}}.$$

Following the methodology outlined in [42], we estimate $\mathit{\beta }$ using

$$\widehat{\mathit{\beta }}={\left[\tilde{\mathbf{Z}}{\tilde{\mathbf{Z}}}^{\top }\right]}^{-1}\tilde{\mathbf{Z}}\tilde{Y},$$

(22)

where

$$\begin{array}{ccc}\hfill \tilde{Y}& =& {\left[\psi \left({\mathbf{Y}}_i\right)-\sum _{j=1}^n{W}_{nj}\left({\mathbf{X}}_i\right)\mathsf{\Psi }\left({\mathbf{Y}}_j\right)\right]}_{1\le i\le n}^{\top },\hfill \end{array}$$

(23)

$$\begin{array}{ccc}\hfill \tilde{\mathbf{Z}}& =& {\left[{\mathbf{Z}}_i-\sum _{j=1}^n{W}_{nj}\left({\mathbf{X}}_i\right){\mathbf{Z}}_j\right]}_{1\le i\le n},\hfill \end{array}$$

(24)

$$W_{nj}\left({\mathbf{X}}_i\right)={\displaystyle \frac{U_{nj}\left({\mathbf{X}}_i\right)}{n{f}_n\left({\mathbf{X}}_j\right)}},$$

and

$$\begin{array}{ccc}\hfill U_{nj}\left({\mathbf{X}}_i\right)& =& \sum _{\mathcal{l}=1}^d{2}^{\lambda \left(n\right)}{K}_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{X}_{\mathcal{l},i},2^{\lambda \left(n\right)}{X}_{j,\mathcal{l}}\right)D_{-\mathcal{l}}\hfill \\ & & -(d-1){\int }_{{\mathbb{R}}^d}{2}^{\lambda \left(n\right)d}K\left(2^{\lambda \left(n\right)}\mathbf{z},2^{\lambda \left(n\right)}{\mathbf{X}}_j\right)q\left(\mathbf{z}\right)d\mathbf{z}.\hfill \end{array}$$

Here,

$$\begin{array}{c}\hfill D_{-\mathcal{l}}={\int }_{{\mathbb{R}}^{d-1}}{2}^{\lambda \left(n\right)(d-1)}K\left(2^{\lambda \left(n\right)}{\mathbf{z}}_{-\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{-\mathcal{l},j}\right)q_{-\mathcal{l}}\left({\mathbf{z}}_{-\mathcal{l}}\right)d{\mathbf{z}}_{-\mathcal{l}}.\end{array}$$

For additional information regarding the estimation of $\mathit{\beta }$, we refer the reader to [16]. Finally, to estimate the regression function and the additive components, we use

$$\begin{array}{c}\hfill {\widehat{m}}_{add}(\mathsf{\Psi },\mathbf{x},h)=\sum _{\mathcal{l}=1}^d{\widehat{\eta }}_{\mathcal{l}}\left(x_{\mathcal{l}}\right)+{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z},\end{array}$$

(25)

and

$$\begin{array}{c}\hfill {\widehat{\eta }}_{\mathcal{l}}(\mathsf{\Psi },x_{\mathcal{l}},h)={\int }_{{\mathbb{R}}^{d-1}}{\widehat{m}}_n\left(\mathbf{x}\right)q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)d{\mathbf{x}}_{-\mathcal{l}}-{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n\left(\mathbf{x}\right)q\left(\mathbf{x}\right)d\mathbf{x}.\end{array}$$

(26)

4. Main Results

To present our findings, we introduce an additional structural assumption on the model. Specifically, we assume that

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}\sum _{i=1}^n{\tilde{\mathbf{Z}}}_i^{\top }{\tilde{\mathbf{Z}}}_i=B\mathrm{a}.\mathrm{s}.,$$

(27)

where B is a $p\times p$ positive-definite matrix. Let

$$I^d=\prod _{\mathcal{l}=1}^d{I}_{\mathcal{l}}=\prod _{\mathcal{l}=1}^d[a_{\mathcal{l}},c_{\mathcal{l}}],$$

and

$$J^d=\prod _{\mathcal{l}=1}^d{J}_{\mathcal{l}}=\prod _{\mathcal{l}=1}^d[a_{\mathcal{l}}^{\prime },c_{\mathcal{l}}^{\prime }],$$

be two fixed partitions of ${\mathbb{R}}^d$ such that

$$a_{\mathcal{l}}^{\prime }<a_{\mathcal{l}}<c_{\mathcal{l}}<c_{\mathcal{l}}^{\prime },\mathrm{for}\mathrm{all}1\le \mathcal{l}\le d.$$

Additional assumptions regarding the density of $\mathbf{X}$, the regression function $m(·)$, the kernel functions $\mathbb{K}(·)$, $K(·)$, and the smoothing parameters are collected below for ease of reference and the reader’s convenience. The first set of conditions concerns the density function $f_{\mathbf{X}}(·)$ and the regression function $m(·)$.

(G.1)

The density $f_{\mathbf{X}}(·)$ is continuous and bounded away from 0 on ${\mathbb{R}}^d$.

(G.2)

One of the following holds:

(i)

$f_{\mathbf{X}}(·)$ is bounded and uniformly continuous on ${\mathbb{R}}^d$,

or

(ii)

$f_{\mathbf{X}}(·)$ belongs to a Besov space $B_{s,p,q}$ for some $0<s<r$, $1\le p,q\le \infty$.

(G.3)

For all $\mathbf{x},\mathbf{y}$, we have

$$|f_j(\mathbf{x},\mathbf{y})-f_{\mathbf{X}}\left(\mathbf{x}\right)f_{\mathbf{X}}\left(\mathbf{y}\right)|\le M<\infty \mathrm{for}\mathrm{all}j\ge 1,$$

where $f_j(\mathbf{x},\mathbf{y})$ is the joint probability density function of the vectors $({\mathbf{X}}_0,{\mathbf{X}}_j)$, assumed to exist.

(G.4)

(i)

For all $1\le \mathcal{l}\le d$, the functions $m_{\mathcal{l}}^{\mathit{\beta }}(\mathsf{\Psi },·)$ are Lipschitz continuous of order $\left(s-{\displaystyle \frac{1}{p}}\right)$.

(ii)

${sup}_{\mathbf{x}\in I^d}g(\phi ,\mathbf{x})<\infty$.

(iii)

The function $H(·)$ belongs to the Besov space $B_{s,p,q}$ for some $0<s<r$, $1\le p,q\le \infty$.

(G.5)

The process $\{{\mathbf{X}}_i,{\mathbf{Y}}_i,{\mathbf{Z}}_i\}$ is $\alpha$-mixing with a mixing coefficient $\alpha \left(s\right)$ satisfying

$$\alpha \left(s\right)=\mathcal{O}\left(s^{-\nu }\right),\mathrm{where}\nu >2d+3.$$

In what follows, let $\lambda \left(n\right)$ denote a sequence of positive constants satisfying the following conditions:

(H.1)

As $n\to \infty$,

$$n^{1/2^{\lambda \left(n\right)}}\to \infty ,2^{\lambda \left(n\right)}\to \infty ,n{2}^{-{\displaystyle \frac{\lambda \left(n\right)}{a}}}\to 0,$$

and

$${\left(n{2}^{-\lambda \left(n\right)}\right)}^{1/2}{\left(n^{-1}logn\right)}^{{\displaystyle \frac{s}{d+2s}}}\to 0.$$

(H.2)

As $n\to \infty$,

$$n^{-1}{2}^{\lambda \left(n\right)}logn{log}_2\left(n\right)\to 0.$$

(H.3)

As $n\to \infty$,

$$\sqrt{{\displaystyle \frac{2^{3\lambda \left(n\right)}}{n}}}{\left({\displaystyle \frac{n}{logn}}\right)}^{a(s-{\displaystyle \frac{1}{p}})}\to 0.$$

Next, we state assumptions on the random variables $\mathbf{Z}$ and Y.

(M.1)

$\mathsf{\Psi }\left(Y\right)$ and $\mathbf{Z}$ are bounded.

To present our main results, we impose conditions on the density functions $q_{\mathcal{l}}(·)$ for $1\le \mathcal{l}\le d$.

(Q.1)

Each $q_{\mathcal{l}}(·)$ is bounded and continuous for all $1\le \mathcal{l}\le d$. Moreover, $q_{\mathcal{l}}(·)$ is bounded and uniformly continuous on $\mathbb{R}$ for all $1\le \mathcal{l}\le d$.

Alternatively, we assume the following:

(Q.2)

The function $q_{\mathcal{l}}(·)$ belongs to a Besov space ${\mathbf{B}}_{s,p,q}$ for some $0<s<r$, $1\le p, q\le \infty$, for all $1\le \mathcal{l}\le d$.

(R.1)

The multiresolution analysis is r-regular.

Our principal finding is encapsulated in the following result.

Theorem 1. 

Suppose that all the conditions (G.1)–(G.5), (H.1)–(H.3), (M.1), (Q.1), (Q.2), and (R.1) are satisfied. Then, for a given measurable function $\mathsf{\Psi }(·)\in \mathcal{F}$, it holds that

$$\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\widehat{m}}_{\mathrm{add}}^{\widehat{\beta }}(\mathsf{\Psi },\mathbf{x})-m\left(\mathbf{x}\right)\right|=\mathcal{O}\left({\left[{\displaystyle \frac{logn}{n}}\right]}^{a(s-{\displaystyle \frac{1}{p}})}\right).$$

(28)

Remark 1. 

It is noteworthy that the quantity $2^{-\lambda \left(n\right)}$ appearing in the preceding theorems corresponds directly to the bandwidth $h_n$ utilized in Parzen–Rosenblatt convolution kernel density estimators. From a practical perspective, selecting the multiresolution level ${\tau }_n$ associated with wavelets is often more straightforward than choosing the bandwidth $h_n$ itself. Indeed, only a relatively small number of candidate values for $\lambda \left(n\right)$ (e.g., three or four) need to be considered in practice, thus rendering the procedure more accessible and computationally feasible.

Remark 2. 

It is important to emphasize that the precise logarithmic convergence rates are influenced by both the chosen resolution level and the underlying smoothness parameters s associated with $f(·)$, as posited in (G.2). The function $f(·)$ lies within a Besov-type space ${\mathbf{B}}_{s,p,q}$, which allows for more general and subtle smoothness characterizations than classical integer-order differentiability conditions. Such complexity in smoothness assumptions is well-known in nonparametric estimation and aligns with established results in the literature. While s may be unknown in practice, these convergence rates are standard and commonly observed across nonparametric procedures. To render the estimator fully data-driven, various adaptive techniques have been proposed for selecting the resolution level τ, including methods inspired by Stein’s unbiased risk estimation, heuristic “rules of thumb,” and cross-validation approaches. For detailed discussions on the design and analysis of asymptotically optimal bandwidth selection rules, the interested reader may refer to [53,57].

Remark 3. 

The moment condition (M.1) can be generalized substantially. Instead of requiring boundedness conditions, one may impose more general assumptions on the moments of $\mathbf{Y}$, as outlined in [110]. Specifically, condition (M.1) can be replaced by one of the following stronger but more flexible alternatives:

(M.1)′ 

There exists a measurable envelope function $F:{\mathbb{R}}^q\to [0,\infty )$ such that for all $\mathbf{y}\in {\mathbb{R}}^q$,

$$F\left(\mathbf{y}\right)\ge \underset{\mathsf{\Psi }\in \mathcal{F}}{sup}|\mathsf{\Psi }\left(\mathbf{y}\right)|,$$

and for some $s>2$,

$$\underset{\mathbf{x}\in I^d}{sup}\mathbb{E}\left({|F\left(\mathbf{Y}\right)|}^s\mid \mathbf{X}=\mathbf{x}\right)<\infty .$$

(M.1)″ 

Consider a nonnegative, continuous, and nondecreasing function $\left\{\mathcal{M}\right(x):x\ge 0\}$ defined on $[0,\infty )$. Assume that for some $s>2$, as $x\to \infty$:

$$\left(i\right)x^{-s}\mathcal{M}\left(x\right)\mathit{is}\mathit{eventually}\mathit{decreasing}\mathit{,}\mathit{and}\left(ii\right)x^{-1}log\mathcal{M}\left(x\right)\mathit{is}\mathit{eventually}\mathit{increasing}.$$

(29)

For any $t\ge \mathcal{M}\left(0\right)$, define ${\mathcal{M}}^{inv}\left(t\right)\ge 0$ by

$$\mathcal{M}\left({\mathcal{M}}^{inv}\left(t\right)\right)=t.$$

We further require

$$\underset{\mathbf{x}\in I^d}{sup}\mathbb{E}\left(\mathcal{M}(|F\left(\mathbf{Y}\right)|)\mid \mathbf{X}=\mathbf{x}\right)<\infty .$$

Notable examples of functions $\mathcal{M}(·)$ that satisfy these criteria include the following:

1.

$\mathcal{M}\left(x\right)=x^p$ for some $p>2$;

2.

$\mathcal{M}\left(x\right)=exp\left(sx\right)$ for some $s>0$.

In other words, the assumption (M1) may be replaced by weaker, but more intricate, finite-moment-type conditions. This change, however, significantly complicates the technical details of the proofs. For further insights and a more comprehensive discussion, the reader may consult [111]. Finally, the introduction of the function $\psi (·)$ into our framework is motivated by findings and observations stated in Remark 1.2 of [111].

4.1. The Conditional Distribution Function

By incorporating $\mathsf{\Psi }\left(\mathbf{Y}\right)=\mathbb{1}\{\mathbf{Y}\le \mathbf{t}\}$, for $\mathbf{t}\in {\mathbb{R}}^q$, into (17) under the assumption $\beta =\mathbf{0}$, we obtain the following kernel estimator of the conditional distribution function

$$F(\mathbf{t}\mid \mathbf{x}):=\mathbb{P}\left(\mathbf{Y}\le \mathbf{t}\mid \mathbf{X}=\mathbf{x}\right),$$

namely,

$${\tilde{m}}_n(\mathbf{t}\mid \mathbf{x}):=\sum _{i=1}^n{\displaystyle \frac{2^{\lambda \left(n\right)d}\mathbb{1}({\mathbf{Y}}_i\le \mathbf{t})}{n{f}_n\left({\mathbf{X}}_i\right)}}K\left(2^{\lambda \left(n\right)}\mathbf{x},2^{\lambda \left(n\right)}{\mathbf{X}}_i\right).$$

(30)

ref. [112] refers to this as the conditional empirical distribution function and was the first to establish uniform consistency results for it. Next, consider the following additive structure:

$$\begin{array}{c}\hfill F_{add}(\mathbf{t}\mid \mathbf{x})=\sum _{\mathcal{l}=1}^d{m}_{\mathcal{l}}(\mathbf{t},x_{\mathcal{l}}).\end{array}$$

Following the marginal integration procedure, for any $\mathbf{x}\in {\mathbb{R}}^d$, the corresponding additive regression function takes the form

$$\begin{array}{c}\hfill F_{add}(\mathbf{t}\mid \mathbf{x})=\sum _{\mathcal{l}=1}^d{\eta }_{\mathcal{l}}(\mathbf{t},x_{\mathcal{l}})+{\int }_{{\mathbb{R}}^d}{F}_{add}(\mathbf{t}\mid \mathbf{z})q\left(\mathbf{z}\right)d\mathbf{z},\end{array}$$

where

$$\begin{array}{c}\hfill {\eta }_{\mathcal{l}}(\mathbf{t},x_{\mathcal{l}})={\int }_{{\mathbb{R}}^{d-1}}{F}_{add}(\mathbf{t}\mid \mathbf{x})q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)d{\mathbf{x}}_{-\mathcal{l}}-{\int }_{{\mathbb{R}}^d}{F}_{add}(\mathbf{t}\mid \mathbf{x})q\left(\mathbf{x}\right)d\mathbf{x}.\end{array}$$

(31)

To estimate both the conditional empirical distribution and the additive components, we define

$$\begin{array}{c}\hfill {\tilde{F}}_{add}(\mathbf{t},\mathbf{x},h_n)=\sum _{\mathcal{l}=1}^d{\tilde{\eta }}_{\mathcal{l}}(\mathbf{t},x_{\mathcal{l}},h_n)+{\int }_{{\mathbb{R}}^d}{\tilde{m}}_n(\mathbf{t}\mid \mathbf{z})q\left(\mathbf{z}\right)d\mathbf{z},\end{array}$$

(32)

and

$$\begin{array}{c}\hfill {\tilde{\eta }}_{\mathcal{l}}(\mathbf{t},x_{\mathcal{l}},h_n)={\int }_{{\mathbb{R}}^{d-1}}{\tilde{m}}_n(\mathbf{t}\mid \mathbf{x})q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)d{\mathbf{x}}_{-\mathcal{l}}-{\int }_{{\mathbb{R}}^d}{\tilde{m}}_n(\mathbf{t}\mid \mathbf{x})q\left(\mathbf{x}\right)d\mathbf{x}.\end{array}$$

(33)

By applying Theorem 1, we obtain the uniform convergence rate

$$\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\tilde{F}}_{add}(\mathbf{t},\mathbf{x},h)-F_{add}(\mathbf{t}\mid \mathbf{x})\right|=\mathcal{O}({\left[\frac{logn}{n}\right]}^{a(s-\frac{1}{p})}).$$

(34)

4.2. Relative-Error Prediction

Classical regression typically estimates the operator $r(·)$ by minimizing the conditional expectation of the squared error, $\mathbb{E}\left[{(Y-r\left(X\right))}^2\mid X\right]$. While widely used, this approach imposes equal emphasis on all observations, rendering it sensitive to outliers and potentially unsuitable in situations involving large variability in the response values.

To address these drawbacks, we focus on minimizing the mean squared relative error (MSRE) defined by

$$\mathbb{E}\left[{\left({\displaystyle \frac{Y-r\left(X\right)}{Y}}\right)}^2\mid X\right],\mathrm{with}Y>0\mathrm{almost}\mathrm{surely}.$$

This criterion offers a more robust assessment of predictive performance when the range of predicted values is extensive. Moreover, the unique feature of the MSRE-based approach is that its solution can be expressed as a ratio of the first two conditional inverse moments of Y. Specifically, letting

$$g_{\gamma }\left(x\right)=\mathbb{E}\left[Y^{-\gamma }\mid X=x\right],\gamma =1,2,$$

and assuming these moments exist and are finite almost surely, one can show, following [113,114,115], that the optimal MSRE predictor of Y given X is

$$\breve{r}\left(x\right)={\displaystyle \frac{\mathbb{E}\left[Y^{-1}\mid ,X=x\right]}{\mathbb{E}\left[Y^{-2}\mid X=x\right]}},\mathrm{a}.\mathrm{s}.$$

By incorporating $\mathsf{\Psi }\left(Y\right)=Y^{-\gamma }$, into (17) under the assumption $\mathbf{\beta }=\mathbf{0}$, we obtain the following kernel estimator

$${\breve{r}}_n\left(x\right)={\displaystyle \frac{{\widehat{m}}_{\mathrm{add}}^{\mathbf{0}}(y^{-1},\mathbf{x})}{{\widehat{m}}_{\mathrm{add}}^{\mathbf{0}}(y^{-2},\mathbf{x})}}$$

By leveraging Theorem 1 for the particular cases $\mathsf{\Psi }\left(y\right)=y^{-1}$ and $\mathsf{\Psi }\left(y\right)=y^{-2}$, we establish the following uniform convergence rate:

$$\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\breve{r}}_n\left(x\right)-\breve{r}\left(x\right)\right|=\mathcal{O}\left({\left[\frac{logn}{n}\right]}^{a\left(s-\frac{1}{p}\right)}\right).$$

This result not only corroborates previous findings but also extends and enhances the analyses presented in [116].

Remark 4. 

It has long been recognized that kernel estimators become less reliable when the dimensionality of the data increases. This phenomenon, commonly known as the curse of dimensionality [117], emerges because, in high-dimensional settings, local regions typically require exceedingly large sample sizes to accumulate enough observations. As a result, unless sample sizes are enormous, bandwidths must be made so broad that the notion of “local” averaging is compromised. A thorough exploration of these issues, with numerical illustrations, can be found in [118], and more recent references include [119,120]. Despite their popularity, however, there remains significant uncertainty regarding the asymptotic properties of penalized splines, even in standard nonparametric settings, and only a few theoretical investigations address their performance. In parallel, it should be noted that most functional regression techniques rely on minimizing an $L_2$-norm and are thus sensitive to outliers. Other less conventional but potentially valuable tools include methods using delta sequences and wavelet-based approaches.

Remark 5. 

In statistical learning for time series, it is often assumed that the underlying data-generating process exhibits some form of asymptotic independence, commonly referred to as “mixing”. While many standard processes are presumed to follow known types of mixing rates, the precise coefficients that govern these rates generally remain unspecified. Moreover, such mixing assumptions are seldom examined empirically, and there are currently no established methods to estimate mixing coefficients directly from observed data. To address this gap, ref. [121] introduces an estimator for beta-mixing coefficients based on a single stationary realization. Subsequently, ref. [122] proposes strongly consistent estimators for the ${\mathcal{l}}_1$-norm of both α- and β-mixing coefficients in a stationary ergodic process. These estimators then serve as the foundation for strongly consistent hypothesis tests for goodness-of-fit. In particular, ref. [122] develops tests to determine whether the α-mixing or β-mixing coefficients of a process remain bounded above by a specified rate function, under the same summability assumptions.

Remark 6. 

Nonlinear thresholding approaches yield the following estimators for $g(\mathsf{\Psi },\mathbf{x})$ and $f_{\mathbf{X}}\left(\mathbf{x}\right)$. Specifically, we define

$${\widehat{f}}_{\mathbf{X}}\left(\mathbf{x}\right)=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{j_0,\mathbf{k}}{\varphi }_{j_0,\mathbf{k}}\left(\mathbf{x}\right)+\sum _{j=j_0}^{\tau }\sum _{i=1}^N\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{b}}_{ij\mathbf{k}}\mathbf{1}\left\{\left|{\widehat{b}}_{ij\mathbf{k}}\right|>{\delta }_{j,n}\right\}{\psi }_{i,j,\mathbf{k}}\left(\mathbf{x}\right),$$

(35)

and

$${\widehat{g}}_n(\mathsf{\Psi },\mathbf{x})=\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{a}}_{j_0,\mathbf{k}}^{\prime }{\varphi }_{j_0,\mathbf{k}}\left(\mathbf{x}\right)+\sum _{j=j_0}^{\tau }\sum _{i=1}^N\sum _{\mathbf{k}\in {\mathbb{Z}}^d}{\widehat{b}}_{ij\mathbf{k}}^{\prime }\mathbf{1}\left\{\left|{\widehat{b}}_{ij\mathbf{k}}^{\prime }\right|>{\delta }_{j,n}\right\}{\psi }_{i,j,\mathbf{k}}\left(\mathbf{x}\right),$$

(36)

where ${\delta }_{j,n}$ is an appropriately selected threshold. In the one-dimensional setting ($d=1$), the estimator ${\widehat{f}}_X$ was originally proposed by [56]. Exploring the applicability and performance of these estimators for higher-dimensional cases would be a valuable direction for future research.

Remark 7. 

Numerous investigations, such as [54,55,56], report that thresholded nonlinear wavelet-based methods frequently outperform linear alternatives when assessed by mean integrated error. However, Theorem 3 and its corollary reveal that, for any fixed function g in the Besov space $B_{s,p,q}$, linear wavelet estimators already attain the classic optimal convergence rates under the almost sure sup-norm criterion in the standard i.i.d. setting. This implies that, in terms of convergence rates, the nonlinear thresholded estimators in (35)–(36) confer no additional advantage over the linear estimators in (8)–(11) once the almost sure sup-norm is adopted as the performance measure.

This result naturally raises the question of why the sup-norm metric is preferable to the mean integrated error. Many functions in the broad Besov space $B_{s,p,q}$ exhibit significant local oscillations, which can cause large estimation errors in small subsets of ${\mathbb{R}}^d$. Such localized errors have a negligible impact on mean integrated metrics yet are readily detected by the sup-norm. This issue is especially acute for discontinuous functions, as substantial misestimation around a discontinuity may remain hidden under mean integrated error. Empirical findings by [55] corroborate this, illustrating how pointwise bias and variance in wavelet-based estimators can oscillate considerably while appearing largely diminished in the mean integrated squared error.

Moreover, in most practical scenarios, we typically observe only one realization of the underlying process. An almost sure sup-norm analysis reveals how estimators perform for almost every realization, in contrast to metrics such as mean-square error or mean integrated squared error, which capture only average behavior across multiple realizations. Therefore, we propose that the almost sure sup-norm constitutes a more fitting benchmark for density and regression estimation in $B_{s,p,q}$. Additional insights on this perspective can be found in the introduction of [71].

Remark 8. 

Our approach, which integrates marginal integration with linear wavelet-based techniques, facilitates the effective separation of linear influences from complex nonlinear patterns. This capability is especially valuable in managerial settings, where comprehending both direct and subtle impacts on outcomes is crucial. For instance:

1.

Finance: Managers can utilize this model to enhance risk assessment and optimize portfolio strategies. By capturing nonlinear dependencies among market variables, the model yields more precise forecasts and deeper insights into asset behavior, thereby supporting more informed investment decisions.

2.

Biology: In research and development, the ability to detect and quantify nonlinear effects assists in guiding resource allocation and experimental design, ultimately leading to more efficient innovation and product development strategies.

3.

Engineering: In engineering contexts, our method’s robust estimation of critical performance indicators—despite the presence of complex nonlinear interactions—substantially enhances operational control and system optimization. This increased precision enables managers to implement quality control measures that are both accurate and reliable, ensuring that engineering systems consistently perform within established standards.

5. Concluding Remarks

In this work, we establish strong uniform convergence results for estimators of nonlinear additive components derived from integrating marginal methods with linear wavelet-based estimation strategies. These findings significantly advance the theoretical underpinnings of additive modeling under mixing conditions, which characterizes the dependence structure of the underlying random variable sequences. In particular, wavelet estimators excel in detecting discontinuities, offering a distinct advantage over kernel-based techniques. This benefit is most pronounced when target functions exhibit spatial irregularities, especially for $1\le p<2$, or when function smoothness is not specified. In such scenarios, purely linear smoothing strategies cannot attain the minimax optimal convergence rate, making nonlinear wavelet estimators (e.g., soft and hard thresholding) indispensable. Nevertheless, constructing and analyzing these nonlinear approaches introduces considerable mathematical complexity, which remains an important avenue for future research to broaden the applicability and bolster the performance of wavelet-based estimators.

A promising direction for further exploration involves the application of wavelet techniques to conditional U-statistics. Integrating wavelet-based methods into this setting may open up new theoretical insights and spur the development of enhanced inference procedures. Advancing this line of research is therefore highly encouraged as it has the potential to substantially enrich methodological progress in the analysis of complex stochastic processes.

The advent of high-throughput genomic technologies—covering genome-wide single nucleotide polymorphisms and RNA sequencing—has generated extensive datasets that require robust statistical models to quantify and interpret how genetic variation and gene expression influence quantitative traits [123]. In contrast to conventional data structures, where the number of variables is typically much smaller than the sample size, modern high-dimensional data often have as many or more variables (p) than observations (n). This imbalance poses serious challenges to traditional statistical methods, which generally assume $n\gg p$.

Consequently, it is critical to examine the viability of partial linear additive models (PLAM) in high-dimensional frameworks, particularly where both linear and nonlinear components can exceed the sample size, as emphasized in [123]. Such investigations promise to refine our analytical toolkit for complex genomic data and deepen our understanding of the genetic mechanisms that underlie quantitative traits.

Finally, to reinforce the practical significance of this work, comprehensive simulation studies and applications to real datasets would be highly advantageous. We also intend to perform a comprehensive sensitivity analysis to assess the robustness of our findings under various conditions. Such empirical investigations will offer more concrete guidelines for the effective use of these methods, which will underscore their potential to advance contemporary statistical analysis.

6. Proofs

This section is dedicated to proving our main result. Throughout this proof, we will continue to use the notation introduced earlier. To unburden our notation a bit, ${\widehat{m}}_{\mathrm{add}}^{\widehat{\mathit{\beta }}}(\mathsf{\Psi },·)$ will be denoted by ${\widehat{m}}_{\mathrm{add}}^{\widehat{\mathit{\beta }}}(·)$. We begin by establishing the result under the assumption that the marginal density $f(·)$ is known. Subsequently, we provide a brief outline of the proof in which the marginal density is estimated.

Proof of Theorem 1. 

Since $\mathcal{C}$ is compact, we can cover it with a finite collection of balls $B_p(t_p,r_p)$ such that

$$\mathcal{C}\subset \bigcup _{p=1}^{r\left(n\right)}{B}_p,$$

where $r\left(n\right)$ will be specified later. Therefore, for any point $x\in \mathcal{C}$, there exists a ball centered at $t\left(x\right)$ that contains x. Consequently, the approximation error can be decomposed as follows:

$$\begin{array}{l}{\displaystyle \underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\widehat{m}}_{add}^{\widehat{\mathit{\beta }}}\left(\mathbf{x}\right)-m\left(\mathbf{x}\right)\right|}\\ \le {\displaystyle \underset{\mathbf{x}\in \mathcal{C}}{sup}}\left|{\widehat{m}}_{add}^{\widehat{\beta }}\left(\mathbf{x}\right)-{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)\right|+{\displaystyle \underset{\mathbf{x}\in \mathcal{C}}{sup}}\left|{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)-{\widehat{m}}_{add}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right|\\ +{\displaystyle \underset{\mathbf{x}\in \mathcal{C}}{sup}}\left|{\widehat{m}}_{add}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)-\mathbb{E}\left[{\widehat{m}}_{add}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right]\right|+{\displaystyle \underset{\mathbf{x}\in \mathcal{C}}{sup}}\left|\mathbb{E}\left[{\widehat{m}}_{add}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right]-\mathbb{E}\left[{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)\right]\right|\\ +{\displaystyle \underset{\mathbf{x}\in \mathcal{C}}{sup}}\left|\mathbb{E}\left[{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)\right]-m\left(\mathbf{x}\right)\right|.\hfill \end{array}$$

(37)

This decomposition shows how each term contributes to the overall approximation error. Recall that

$$\begin{array}{c}\hfill {\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)=\sum _{\mathcal{l}=1}^d{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)+{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}.\end{array}$$

Hence,

$$\begin{array}{l}{\displaystyle \left|\mathbb{E}\left[{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)\right]-m\left(\mathbf{x}\right)\right|}\\ = \left|\mathbb{E}\left[{\displaystyle \sum _{\mathcal{l}=1}^d}{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)+{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}\right]-{\displaystyle \sum _{\mathcal{l}=1}^d}{\eta }_{\mathcal{l}}\left(x_{\mathcal{l}}\right)-{\int }_{{\mathbb{R}}^d}m\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}\right|\hfill \\ = \left|{\displaystyle \sum _{\mathcal{l}=1}^d}\left(\mathbb{E}\left[{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)\right]-{\eta }_{\mathcal{l}}\left(x_{\mathcal{l}}\right)\right)+{\int }_{{\mathbb{R}}^d}\left(\mathbb{E}\left[{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)\right]-m\left(\mathbf{z}\right)\right)q\left(\mathbf{z}\right)d\mathbf{z}\right|\hfill \\ =:\left|{\mathcal{B}}_1+{\mathcal{B}}_2\right|.\hfill \end{array}$$

(38)

Here, ${\mathcal{B}}_1$ captures the main bias for the additive components ${\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}(·)$, while ${\mathcal{B}}_2$ corresponds to the integral term involving ${\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)$. Next, recall that

$${\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)={\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)-{\int }_{\mathbb{R}}{\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(u_{\mathcal{l}}\right)q_{\mathcal{l}}\left(u_{\mathcal{l}}\right)d{u}_{\mathcal{l}},$$

where

$${\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)={\displaystyle \frac{2^{\lambda \left(n\right)}}{n}}\sum _{i=1}^n{\displaystyle \frac{{\tilde{Y}}_{i,n}^{\mathit{\beta }}}{f_{\mathcal{l}}\left(X_{\mathcal{l},i}\right)}}{K}_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{X}_{\mathcal{l},i}\right),$$

and

$${\tilde{Y}}_{i,n}^{\mathit{\beta }}=\left(\mathsf{\Psi }\left({\mathbf{Y}}_i\right)-{\mathbf{Z}}_i^{\top }\mathit{\beta }\right){\int }_{{\mathbb{R}}^{d-1}}{2}^{(d-1)\lambda \left(n\right)}{K}_{-\mathcal{l}}\left(2^{\lambda \left(n\right)}{\mathbf{x}}_{-\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{-\mathcal{l},i}\right){\displaystyle \frac{q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)}{f\left({\mathbf{X}}_{-\mathcal{l},i}\mid X_{\mathcal{l},i}\right)}}d{\mathbf{x}}_{-\mathcal{l}}.$$

The term ${\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}(·)$ can be viewed as a univariate regression estimator of

$${\tilde{m}}_n(·)=\mathbb{E}[{\tilde{Y}}_{i,n}^{\mathit{\beta }}\mid {\mathbf{X}}_{\mathcal{l},i}=·].$$

In accordance with the approach of [103], let

$$C_n:=\mu +{\int }_{{\mathbb{R}}^{d-1}}\sum _{j=1,j\ne \mathcal{l}}^d{m}_j\left(z_j\right)g_n\left({\mathbf{z}}_{-\mathcal{l}}\right)d{\mathbf{z}}_{-\mathcal{l}},$$

where

$$g_n\left({\mathbf{z}}_{-\mathcal{l}}\right):={\int }_{{\mathbb{R}}^{d-1}}{2}^{(d-1)\lambda \left(n\right)}{K}_{-\mathcal{l}}\left(2^{\lambda \left(n\right)}{\mathbf{u}}_{-\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{z}}_{-\mathcal{l}}\right)q_{-\mathcal{l}}\left({\mathbf{u}}_{-\mathcal{l}}\right)d{\mathbf{u}}_{-\mathcal{l}}.$$

Then

$${\tilde{m}}_n\left(x_{\mathcal{l}}\right)=m_{\mathcal{l}}\left(x_{\mathcal{l}}\right)+C_n.$$

(39)

Furthermore,

$${\eta }_{\mathcal{l}}\left(x_{\mathcal{l}}\right)=m_{\mathcal{l}}\left(x_{\mathcal{l}}\right)-{\int }_{\mathbb{R}}{m}_{\mathcal{l}}\left(u_{\mathcal{l}}\right)q_{\mathcal{l}}\left(u_{\mathcal{l}}\right)d{u}_{\mathcal{l}}.$$

(40)

Combining (39) and (40) leads to

$${\eta }_{\mathcal{l}}\left(x_{\mathcal{l}}\right)={\tilde{m}}_n\left(x_{\mathcal{l}}\right)-C_n-{\int }_{\mathbb{R}}{m}_{\mathcal{l}}\left(u_{\mathcal{l}}\right)q_{\mathcal{l}}\left(u_{\mathcal{l}}\right)d{u}_{\mathcal{l}}.$$

(41)

Additionally, for $1\le \mathcal{l}\le d$, we know

$${\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)={\int }_{{\mathbb{R}}^{d-1}}{\widehat{m}}_n^{\mathsf{\Psi },\mathit{\beta }}\left(\mathbf{x}\right)q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)d{\mathbf{x}}_{-\mathcal{l}}-{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathsf{\Psi },\mathit{\beta }}\left(\mathbf{x}\right)q\left(\mathbf{x}\right)d\mathbf{x},$$

which implies

$${\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)={\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)-{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathsf{\Psi },\mathit{\beta }}\left(\mathbf{x}\right)q\left(\mathbf{x}\right)d\mathbf{x}.$$

(42)

Therefore, exploiting (41) and (42), we can write the first bias term ${\mathcal{B}}_1$ in (38) as

$$\begin{array}{ccc}\hfill {\mathcal{B}}_1\left(x_{\mathcal{l}}\right)& =& \mathbb{E}\left[{\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)\right]-{\tilde{m}}_n\left(x_{\mathcal{l}}\right)\hfill \\ \hfill & & -\left(\mathbb{E}\left[{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathsf{\Psi },\mathit{\beta }}\left(\mathbf{x}\right)q\left(\mathbf{x}\right)d\mathbf{x}\right]-{\int }_{\mathbb{R}}{m}_{\mathcal{l}}\left(u_{\mathcal{l}}\right)q_{\mathcal{l}}\left(u_{\mathcal{l}}\right)d{u}_{\mathcal{l}}-C_n\right).\hfill \end{array}$$

(43)

To further estimate $\left|\mathbb{E}\left[{\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)\right]-{\tilde{m}}_n\left(x_{\mathcal{l}}\right)\right|$, observe:

$$\begin{array}{l}{\displaystyle \left|\mathbb{E}\left[{\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)\right]-{\tilde{m}}_n\left(x_{\mathcal{l}}\right)\right|}\\ =\left|\mathbb{E}\left[{\displaystyle \frac{2^{\lambda \left(n\right)}{\tilde{Y}}_{i,n}^{\mathit{\beta }}}{f_{\mathcal{l}}\left(X_{\mathcal{l},i}\right)}}{K}_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{X}_{\mathcal{l},i}\right)\right]-{\tilde{m}}_n\left(x_{\mathcal{l}}\right)\right|\hfill \\ =\left|{\int }_{\mathbb{R}}{2}^{\lambda \left(n\right)}{\tilde{m}}_n\left(u_{\mathcal{l}}\right)K_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{u}_{\mathcal{l}}\right)d{u}_{\mathcal{l}}-{\tilde{m}}_n\left(x_{\mathcal{l}}\right)\right|\hfill \\ =\left|{\int }_{\mathbb{R}}\left({\tilde{m}}_n\left(x_{\mathcal{l}}+2^{-\lambda \left(n\right)}{v}_{\mathcal{l}}\right)-{\tilde{m}}_n\left(x_{\mathcal{l}}\right)\right)K_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{x}_{\mathcal{l}}+v_{\mathcal{l}}\right)d{v}_{\mathcal{l}}\right|\hfill \\ \le {\int }_{\mathbb{R}}\left|{\tilde{m}}_n\left(x_{\mathcal{l}}+2^{-\lambda \left(n\right)}{v}_{\mathcal{l}}\right)-{\tilde{m}}_n\left(x_{\mathcal{l}}\right)\right|\left|K_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{x}_{\mathcal{l}}+v_{\mathcal{l}}\right)\right|d{v}_{\mathcal{l}}\hfill \\ \le 2^{-\lambda \left(n\right)\left(s-{\displaystyle \frac{1}{p}}\right)}C{\int }_{\mathbb{R}}{\left|v_{\mathcal{l}}\right|}^{\left(s-{\displaystyle \frac{1}{p}}\right)}\left|K_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{x}_{\mathcal{l}}+v_{\mathcal{l}}\right)\right|d{v}_{\mathcal{l}},\hfill \end{array}$$

where in the final step we use the fact that ${\tilde{m}}_n(·)$ is Lipschitz of order $\left(s-{\displaystyle \frac{1}{p}}\right)$. Note additionally that each kernel $K_{\mathcal{l}}(·)$ has compact support, which simplifies bounding the above integrals. We begin by noting that under the assumption

$$2^{-\lambda \left(n\right)}\sim {\left({\displaystyle \frac{logn}{n}}\right)}^a,$$

it follows that

$$\begin{array}{ccc}\hfill \left|\mathbb{E}{\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)-{\tilde{m}}_n\left(x_{\mathcal{l}}\right)\right|& =& \mathcal{O}{\left(\frac{logn}{n}\right)}^{a\left(s-\frac{1}{p}\right)}.\hfill \end{array}$$

(44)

To proceed, we next examine

$$\begin{array}{l}{\displaystyle \mathbb{E}\left[{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathsf{\Psi },\mathit{\beta }}\left(\mathbf{x}\right)q\left(\mathbf{x}\right)d\mathbf{x}-C_n\right]}\\ ={\int }_{{\mathbb{R}}^d}\mathbb{E}\left[{\widehat{m}}_n^{\mathsf{\Psi },\mathit{\beta }}\left(\mathbf{x}\right)\right]q\left(\mathbf{x}\right)d\mathbf{x}-{\int }_{{\mathbb{R}}^{d-1}}{\displaystyle \sum _{\begin{array}{c}j=1\\ j\ne \mathcal{l}\end{array}}^d}{m}_j\left(z_j\right)g_n\left({\mathbf{z}}_{-\mathcal{l}}\right)d{\mathbf{z}}_{-\mathcal{l}}-\mu \hfill \\ =2^{\lambda \left(n\right)d}{\int }_{{\mathbb{R}}^d}\mathbb{E}\left[{\displaystyle \frac{\left(\mathsf{\Psi }\left(Y_i\right)-{\mathbf{Z}}_i^{\top }\mathit{\beta }\right)K\left(2^{\lambda \left(n\right)}\mathbf{x},2^{\lambda \left(n\right)}{\mathbf{X}}_i\right)}{f\left({\mathbf{X}}_i\right)}}\right]q\left(\mathbf{x}\right)d\mathbf{x}\hfill \\ -{\int }_{{\mathbb{R}}^{d-1}}{\displaystyle \sum _{\begin{array}{c}j=1\\ j\ne \mathcal{l}\end{array}}^d}{m}_j\left(z_j\right)g_n\left({\mathbf{z}}_{-\mathcal{l}}\right)d{\mathbf{z}}_{-\mathcal{l}}-\mu \hfill \\ =2^{\lambda \left(n\right)d}{\int }_{{\mathbb{R}}^d}{\int }_{{\mathbb{R}}^d}{m}_{add}\left(\mathbf{u}\right)K\left(2^{\lambda \left(n\right)}\mathbf{x},2^{\lambda \left(n\right)}\mathbf{u}\right)q\left(\mathbf{x}\right)d\mathbf{x}d\mathbf{u}\hfill \\ -{\int }_{{\mathbb{R}}^{d-1}}{\displaystyle \sum _{\begin{array}{c}j=1\\ j\ne \mathcal{l}\end{array}}^d}{m}_j\left(z_j\right)g_n\left({\mathbf{z}}_{-\mathcal{l}}\right)d{\mathbf{z}}_{-\mathcal{l}}-\mu \hfill \\ =2^{\lambda \left(n\right)d}{\int }_{{\mathbb{R}}^d}{\int }_{{\mathbb{R}}^d}{\displaystyle \sum _{j=1}^n}{m}_j\left(u_j\right)K\left(2^{\lambda \left(n\right)}\mathbf{x},2^{\lambda \left(n\right)}\mathbf{u}\right)q\left(\mathbf{x}\right)d\mathbf{x}d\mathbf{u}\hfill \\ -{\int }_{{\mathbb{R}}^{d-1}}{\displaystyle \sum _{\begin{array}{c}j=1\\ j\ne \mathcal{l}\end{array}}^d}{m}_j\left(z_j\right)g_n\left({\mathbf{z}}_{-\mathcal{l}}\right)d{\mathbf{z}}_{-\mathcal{l}}\hfill \\ =2^{\lambda \left(n\right)}{\int }_{\mathbb{R}}{\int }_{\mathbb{R}}{m}_{\mathcal{l}}\left(u_{\mathcal{l}}\right)K_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{u}_{\mathcal{l}}\right)q_{\mathcal{l}}\left(x_{\mathcal{l}}\right)d{x}_{\mathcal{l}}d{u}_{\mathcal{l}}\hfill \\ \le {\int }_{\mathbb{R}}{m}_{\mathcal{l}}\left(x_{\mathcal{l}}\right)q_{\mathcal{l}}\left(x_{\mathcal{l}}\right)d{x}_{\mathcal{l}}+{\int }_{\mathbb{R}}{q}_{\mathcal{l}}\left(x_{\mathcal{l}}\right){\int }_{\mathbb{R}}\left[m_{\mathcal{l}}\left(x_{\mathcal{l}}+2^{-\lambda \left(n\right)}{v}_{\mathcal{l}}\right)-m_{\mathcal{l}}\left(x_{\mathcal{l}}\right)\right]\hfill \\ \times K_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{x}_{\mathcal{l}}+v_{\mathcal{l}}\right)d{x}_{\mathcal{l}}d{v}_{\mathcal{l}}\hfill \\ \le {\int }_{\mathbb{R}}{m}_{\mathcal{l}}\left(x_{\mathcal{l}}\right)q_{\mathcal{l}}\left(x_{\mathcal{l}}\right)d{x}_{\mathcal{l}}\hfill \\ +{\int }_{\mathbb{R}}{q}_{\mathcal{l}}\left(x_{\mathcal{l}}\right){\int }_{Supp\left(K_{\mathcal{l}}\right)}{2}^{-\lambda \left(n\right)\left(s-\frac{1}{p}\right)}{\left|v_{\mathcal{l}}\right|}^{\left(s-\frac{1}{p}\right)}{K}_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{x}_{\mathcal{l}}+v_{\mathcal{l}}\right)d{x}_{\mathcal{l}}d{v}_{\mathcal{l}}\hfill \\ ={\int }_{\mathbb{R}}{m}_{\mathcal{l}}\left(x_{\mathcal{l}}\right)q_{\mathcal{l}}\left(x_{\mathcal{l}}\right)d{x}_{\mathcal{l}}+\mathcal{O}{\left(\frac{logn}{n}\right)}^{a\left(s-\frac{1}{p}\right)}.\hfill \end{array}$$

(45)

Combining (44) and (45) immediately implies

$${\mathcal{B}}_1=\mathcal{O}{\left(\frac{logn}{n}\right)}^{a\left(s-\frac{1}{p}\right)}.$$

(46)

We now consider the additional term:

$$\begin{array}{l}{\displaystyle \left|\mathbb{E}\left[{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)\right]-m\left(\mathbf{z}\right)\right|}\\ =\left|\mathbb{E}\left(2^{\lambda \left(n\right)d}{\displaystyle \sum _{i=1}^n}{\displaystyle \frac{\mathsf{\Psi }\left({\mathbf{Y}}_i\right)-{\mathbf{Z}}_i^{\top }\mathit{\beta }}{n{f}_{\mathbf{X}}\left({\mathbf{X}}_i\right)}}K\left(2^{\lambda \left(n\right)}\mathbf{z},2^{\lambda \left(n\right)}{\mathbf{X}}_i\right)\right)-m\left(\mathbf{z}\right)\right|\hfill \\ =\left|{\int }_{{\mathbb{R}}^d}{2}^{\lambda \left(n\right)d}m\left(\mathbf{u}\right)K\left(2^{\lambda \left(n\right)}\mathbf{z},2^{\lambda \left(n\right)}\mathbf{u}\right)d\mathbf{u}-m\left(\mathbf{z}\right)\right|\hfill \\ =\left|{\int }_{{\mathbb{R}}^d}\left(m(\mathbf{z}+2^{-\lambda \left(n\right)}\mathbf{v})-m\left(\mathbf{z}\right)\right)K\left(2^{\lambda \left(n\right)}\mathbf{z},2^{\lambda \left(n\right)}\mathbf{z}+\mathbf{v}\right)d\mathbf{v}\right|\hfill \\ \le 2^{-\lambda \left(n\right)\left(s-\frac{1}{p}\right)}\left|{\int }_{{\mathbb{R}}^d}{\parallel \mathbf{v}\parallel }^{\left(s-\frac{1}{p}\right)}K\left(2^{\lambda \left(n\right)}\mathbf{z},2^{\lambda \left(n\right)}\mathbf{z}+\mathbf{v}\right)d\mathbf{v}\right|.\hfill \end{array}$$

Hence, we conclude that

$${\mathcal{B}}_2=\mathcal{O}{\left(\frac{logn}{n}\right)}^{a\left(s-\frac{1}{p}\right)}.$$

(47)

From (38), (46), and (47), we deduce the following:

$$\begin{array}{c}\hfill \underset{\mathbf{x}\in \mathcal{C}}{sup}\left|\mathbb{E}\left[{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)\right]-m\left(\mathbf{x}\right)\right|=\mathcal{O}{\left(\frac{logn}{n}\right)}^{a\left(s-\frac{1}{p}\right)}.\end{array}$$

(48)

Finally, we consider the difference

$$\begin{array}{ccc}\hfill \left|{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)-{\widehat{m}}_{add}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right|& =& \left|\sum _{\mathcal{l}=1}^d\left({\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)-{\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(t\left(x_{\mathcal{l}}\right)\right)\right)\right|,\hfill \end{array}$$

where

$${\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)={\displaystyle \frac{2^{\lambda \left(n\right)}}{n}}\sum _{i=1}^n{\displaystyle \frac{{\tilde{Y}}_{i,n}^{\mathit{\beta }}}{f_{\mathcal{l}}\left(X_{\mathcal{l},i}\right)}}{K}_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{\mathcal{l},i}\right),$$

and

$$\begin{array}{cc}\hfill {\displaystyle {\tilde{Y}}_{i,n}^{\mathit{\beta }}}& =\left(\mathsf{\Psi }\left({\mathbf{Y}}_i\right)-{\mathbf{Z}}_i^{\top }\mathit{\beta }\right)\hfill \\ & \times {\int }_{{\mathbb{R}}^{d-1}}{2}^{(d-1)\lambda \left(n\right)}{K}_{-\mathcal{l}}\left(2^{\lambda \left(n\right)}{\mathbf{x}}_{-\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{-\mathcal{l},i}\right){\displaystyle \frac{q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)}{f\left({\mathbf{X}}_{-\mathcal{l},i}\mid X_{\mathcal{l},i}\right)}}d{\mathbf{x}}_{-\mathcal{l}}.\hfill \end{array}$$

Since $K_{\mathcal{l}}$ is a Lipschitz function, there exists a suitable Lipschitz constant L; thus,

$$\begin{array}{ccc}\hfill \left|{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)-{\widehat{m}}_{add}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right|& =& \mathcal{O}\left(2^{2\lambda \left(n\right)}{r}_p\right),\hfill \end{array}$$

where $r_p$ is the radius of $B_p(t_p,r_p)$. For

$$r_p={\left({\displaystyle \frac{\epsilon \left(n\right)}{2^{\lambda \left(n\right)}}}\right)}^{\gamma }\mathrm{with}\gamma \ge 2,$$

we obtain

$$\begin{array}{c}\hfill \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)-{\widehat{m}}_{add}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right|\ge \epsilon \left(n\right)/4\right)=0,\end{array}$$

(49)

provided $\epsilon \left(n\right)$ is chosen sufficiently small. Similarly, one can establish that

$$\begin{array}{c}\hfill \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|\mathbb{E}\left[{\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(\mathbf{x}\right)\right]-\mathbb{E}\left[{\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right]\right|\ge \epsilon \left(n\right)/4\right)=0.\end{array}$$

(50)

Next, let ${\left\{(p\left(n\right),q\left(n\right))\right\}}_{n\in {\mathbb{N}}^{*}}$ be sequences of positive integers such that

$$2p\left(n\right)q\left(n\right)\le n<2p\left(n\right)\left(q\left(n\right)+1\right).$$

We then obtain

$$\begin{array}{l}{\displaystyle {\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)-\mathbb{E}\left[{\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right]}\\ ={\displaystyle \sum _{\mathcal{l}=1}^d}{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(t\left(x_{\mathcal{l}}\right)\right)+{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}\hfill \\ -\mathbb{E}\left[{\displaystyle \sum _{\mathcal{l}=1}^d}{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(t\left(x_{\mathcal{l}}\right)\right)+{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}\right]\hfill \\ ={\displaystyle \frac{1}{n}}{\displaystyle \sum _{j=1}^{q\left(n\right)}}{\xi }_j^1\left(t\left(\mathbf{x}\right)\right)+{\displaystyle \frac{1}{n}}{\displaystyle \sum _{j=1}^{q\left(n\right)}}{\xi }_j^2\left(t\left(\mathbf{x}\right)\right)+{\displaystyle \frac{1}{n}}{\delta }_n\left(t\left(\mathbf{x}\right)\right),\hfill \end{array}$$

where

$$\begin{array}{ccc}\hfill {\xi }_j^1& =& \sum _{i=2(j-1)p\left(n\right)+1}^{(2j-1)p\left(n\right)}{\zeta }_i-\mathbb{E}\left[{\zeta }_i\right],\hfill \\ \hfill {\xi }_j^2& =& \sum _{i=(2j-1)p\left(n\right)+1}^{2jp\left(n\right)}{\zeta }_i-\mathbb{E}\left[{\zeta }_i\right],\hfill \\ \hfill {\delta }_n& =& \sum _{i=2q\left(n\right)p\left(n\right)+1}^n{\zeta }_i-\mathbb{E}\left[{\zeta }_i\right],\hfill \\ \hfill {\zeta }_i& =& \sum _{\mathcal{l}=1}^d{\displaystyle \frac{2^{\lambda \left(n\right)}}{n}}{\displaystyle \frac{{\tilde{Y}}_{i,n}^{\mathit{\beta }}}{f_{\mathcal{l}}\left(X_{\mathcal{l},i}\right)}}{K}_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{\mathcal{l},i}\right)\hfill \\ & & -{\int }_{\mathbb{R}}{\displaystyle \frac{{\tilde{Y}}_{i,n}^{\mathit{\beta }}}{f_{\mathcal{l}}\left(X_{\mathcal{l},i}\right)}}{K}_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{\mathcal{l},i}\right)\hfill \\ & & +2^{\lambda \left(n\right)d}{\displaystyle \frac{\mathsf{\Psi }\left(Y_i\right)-{\mathbf{Z}}_i^{\top }\mathit{\beta }}{f\left({\mathbf{X}}_i\right)}}{\int }_{{\mathbb{R}}^d}K\left(2^{\lambda \left(n\right)}\mathbf{x},2^{\lambda \left(n\right)}{\mathbf{X}}_i\right)q\left(\mathbf{x}\right)d\mathbf{x}.\hfill \end{array}$$

It is straightforward to show that for $1\le j\le q\left(n\right)$, there exist constants m and M such that

$$mp\left(n\right)\le {\xi }_j^i\le Mp\left(n\right)\mathrm{for}i=1,2.$$

Consequently, we have

$$\begin{array}{l}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)-E\left[{\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right]\right|\ge \epsilon \left(n\right)\right)}\\ \le \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\displaystyle \sum _{j=1}^{q\left(n\right)}}{\xi }_j^1\left(t\left(\mathbf{x}\right)\right)\right|\ge \frac{2n\epsilon \left(n\right)}{5}\right)+\mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\displaystyle \sum _{j=1}^{q\left(n\right)}}{\xi }_j^2\left(t\left(\mathbf{x}\right)\right)\right|\ge \frac{2n\epsilon \left(n\right)}{5}\right)\hfill \\ +\mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\delta }_n\left(t\left(\mathbf{x}\right)\right)\right|\ge \frac{n\epsilon \left(n\right)}{5}\right).\hfill \end{array}$$

(51)

We begin by examining the leading term of (51). Specifically, we have

$$\begin{array}{c}\hfill \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^1\left(t\left(\mathbf{x}\right)\right)\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}\right)\le r\left(n\right)\underset{p=1,\dots ,r\left(n\right)}{max}\mathbb{P}\left(\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^1\left(t_p\right)\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}\right).\end{array}$$

(52)

Next, by choosing

$$\zeta =min\left(\frac{n\epsilon \left(n\right)}{5q\left(n\right)},\frac{mp\left(n\right)}{2}\right),$$

we obtain

$$\begin{array}{ccc}\hfill \left\{\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^1\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}\right\}& \subset & \left\{\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^1\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}\mathrm{and}{\left|{\xi }_j^1-{\xi }_j^{1*}\right|}_{1\le j\le q\left(n\right)}\le \zeta \right\}\hfill \\ & & \cup \left\{{\cup }_{j=1}^{q\left(n\right)}\left|{\xi }_j^1-{\xi }_j^{1*}\right|>\zeta \right\},\hfill \end{array}$$

where ${\xi }_j^{1*}$ are defined as in Lemma A1. Consequently,

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^1\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}\right)& \le & \mathbb{P}\left(\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^{1*}\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}-q\left(n\right)\zeta \right)\hfill \\ & & +\sum _{j=1}^{q\left(n\right)}\mathbb{P}\left(\left|{\xi }_j^1-{\xi }_j^{1*}\right|\ge \zeta \right).\hfill \end{array}$$

(53)

By applying Hoeffding’s inequality [124], we deduce

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^{1*}\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}-q\left(n\right)\zeta \right)& \le & \mathbb{P}\left(\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^{1*}\right|\ge {\displaystyle \frac{n\epsilon \left(n\right)}{5}}\right)\hfill \\ & \le & 2exp\left(-{\displaystyle \frac{2n^2{\epsilon }^2\left(n\right)/5^2}{q\left(n\right)4M^{2}p{\left(n\right)}^2}}\right)\hfill \\ & \le & 2exp\left(-{\displaystyle \frac{n{\epsilon }^2\left(n\right)}{5^2{M}^{2}p\left(n\right)}}\right).\hfill \end{array}$$

(54)

Next, applying Bradley’s Lemma A1 with $\gamma =+\infty$ gives

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\xi }_j-{\xi }_j^{*}\right|\ge \zeta \right)& \le & 11\sqrt{{\displaystyle \frac{Mp\left(n\right)}{min\left(\frac{n\epsilon \left(n\right)}{5p\left(n\right)},\frac{mp\left(n\right)}{2}\right)}}}\alpha \left(p\right)\hfill \\ & \le & 11{\left({\displaystyle \frac{5}{2}}\right)}^{\frac{1}{2}}{\left(min\left({\displaystyle \frac{\epsilon \left(n\right)}{M}},{\displaystyle \frac{5m}{4M}}\right)\right)}^{-\frac{1}{2}}\alpha \left(p\right),\hfill \end{array}$$

since $n\ge 2p\left(n\right)q\left(n\right)$. If we additionally require $\epsilon \left(n\right)\le M$, then

$$min\left({\displaystyle \frac{\epsilon \left(n\right)}{M}},{\displaystyle \frac{5m}{4M}}\right)\ge {\displaystyle \frac{\epsilon \left(n\right)M}{m^2}}.$$

Hence,

$$\begin{array}{c}\hfill \mathbb{P}\left(\left|{\xi }_j-{\xi }_j^{*}\right|\ge \zeta \right)\le 11{\left({\displaystyle \frac{5}{2}}\right)}^{\frac{1}{2}}\sqrt{{\displaystyle \frac{M^2}{m\epsilon \left(n\right)}}}\alpha \left(p\right).\end{array}$$

(55)

Combining (53), (54), and (55) then yields

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^1\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}\right)& \le & 2exp\left(-{\displaystyle \frac{n{\epsilon }^2\left(n\right)}{5^2{M}^{2}p\left(n\right)}}\right)\hfill \\ & & +11{\left({\displaystyle \frac{5}{2}}\right)}^{\frac{1}{2}}\sqrt{{\displaystyle \frac{M^2}{m\epsilon (n}}}q\left(n\right)\alpha \left(p\right).\hfill \end{array}$$

(56)

Finally, by combining (52) and (56), we obtain

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^1\left(t\left(\mathbf{x}\right)\right)\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}\right)& \le & r\left(n\right)(2exp\left(-{\displaystyle \frac{n{\epsilon }^2\left(n\right)}{5^2{M}^{2}p\left(n\right)}}\right)\hfill \\ & & +11{\left({\displaystyle \frac{5}{2}}\right)}^{\frac{1}{2}}\sqrt{{\displaystyle \frac{M^2}{m\epsilon (n}}}q\left(n\right)\alpha \left(p\right)).\hfill \end{array}$$

(57)

By the same reasoning, we can also prove

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|\sum _{j=1}^{q\left(n\right)}{\xi }_j^2\left(t\left(\mathbf{x}\right)\right)\right|\ge {\displaystyle \frac{2n\epsilon \left(n\right)}{5}}\right)& \le & r\left(n\right)(2exp\left(-{\displaystyle \frac{n{\epsilon }^2\left(n\right)}{5^2{M}^{2}p\left(n\right)}}\right)\hfill \\ & & +11{\left({\displaystyle \frac{5}{2}}\right)}^{\frac{1}{2}}\sqrt{{\displaystyle \frac{M^2}{m\epsilon (n}}}q\left(n\right)\alpha \left(p\right)).\hfill \end{array}$$

(58)

Now, let $\nu >0$. Then we have

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\delta }_n\right|\ge \frac{n\epsilon \left(n\right)}{5}\right)& \le & exp\left(-\frac{\nu n\epsilon \left(n\right)}{5}\right)\mathbb{E}exp\left(\nu \left|{\delta }_n\right|\right)\hfill \\ \hfill & \le & exp\left(-\nu \left(\frac{n\epsilon \left(n\right)}{5}-M\left(n-2p\left(n\right)q\left(n\right)\right)\right)\right).\hfill \end{array}$$

Since $M\nu \left(n-2p\left(n\right)q\left(n\right)\right)=1$ and $n<2p\left(n\right)\left(q\left(n\right)+1\right)$, we obtain

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\delta }_n\right|\ge \frac{n\epsilon \left(n\right)}{5}\right)& \le & eexp\left(-\frac{n\epsilon \left(n\right)}{5M\left(n-2p\left(n\right)q\left(n\right)\right)}\right)\hfill \\ & \le & eexp\left(-\frac{\epsilon \left(n\right)}{5M}q\left(n\right)\right).\hfill \end{array}$$

In addition, if $\epsilon \left(n\right)\le M$, then

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\left|{\delta }_n\right|\ge \frac{n\epsilon \left(n\right)}{5}\right)& \le & eexp\left(-\frac{\epsilon {\left(n\right)}^2}{5M^2}q\left(n\right)\right).\hfill \end{array}$$

(59)

Hence,

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\delta }_n\left(t\left(\mathbf{x}\right)\right)\right|\ge \frac{n\epsilon \left(n\right)}{5}\right)& \le & r\left(n\right)\underset{p=1,\dots ,r\left(n\right)}{max}\mathbb{P}\left(\left|{\delta }_n\left(t_p\right)\right|\ge \frac{n\epsilon \left(n\right)}{5}\right)\hfill \\ & \le & er\left(n\right)exp\left(-\frac{\epsilon {\left(n\right)}^2}{5M^2}q\left(n\right)\right).\hfill \end{array}$$

(60)

Recall that

$$2p\left(n\right)q\left(n\right)\le n<2p\left(n\right)\left(q\left(n\right)+1\right),$$

which implies

$$q\left(n\right)={\displaystyle \frac{n}{2p\left(n\right)}}-\theta ,0\le \theta <1.$$

Then, we have

$$\begin{array}{ccc}\hfill \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\delta }_n\left(t\left(\mathbf{x}\right)\right)\right|\ge \frac{n\epsilon \left(n\right)}{5}\right)& \le & er\left(n\right)exp\left(-\frac{n\epsilon {\left(n\right)}^2}{10M^{2}p\left(n\right)}\right)exp\left(\frac{\epsilon {\left(n\right)}^2}{5M^2}\right)\hfill \\ & \le & e^{6/5}r\left(n\right)exp\left(-\frac{n\epsilon {\left(n\right)}^2}{10M^{2}p\left(n\right)}\right),\hfill \end{array}$$

(61)

since $\epsilon \left(n\right)\le M$.

It follows from (51), (57), (58), and (61) that

$$\begin{array}{l}{\displaystyle \mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)-E\left[{\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right]\right|\ge \epsilon \left(n\right)\right)}\\ \le 2\left(r\left(n\right)\left(2exp\left(-\frac{n{\epsilon }^2\left(n\right)}{25M^{2}p\left(n\right)}\right)+11{\left(\frac{5}{2}\right)}^{\frac{1}{2}}\sqrt{\frac{M^2}{m\epsilon \left(n\right)}}q\left(n\right)\alpha \left(p\right)\right)\right)\hfill \\ +e^{6/5}r\left(n\right)exp\left(-\frac{n\epsilon {\left(n\right)}^2}{10M^{2}p\left(n\right)}\right)\hfill \\ \le r\left(n\right)\left(4+e^{6/5}\right)exp\left(-\frac{n{\epsilon }^2\left(n\right)}{25M^{2}p\left(n\right)}\right)+22r\left(n\right){\left(\frac{5}{2}\right)}^{\frac{1}{2}}\sqrt{\frac{M^2}{m\epsilon (n}}q\left(n\right)\alpha \left(p\right).\hfill \end{array}$$

(62)

Therefore, by the Borel–Cantelli lemma, we obtain

$$\begin{array}{c}\hfill \sum _{n\ge 1}\mathbb{P}\left(\underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)-\mathbb{E}\left[{\widehat{m}}_{\mathrm{add}}^{\mathit{\beta }}\left(t\left(\mathbf{x}\right)\right)\right]\right|\ge \epsilon \left(n\right)\right)<\infty .\end{array}$$

(63)

We begin by observing the following relationship:

$$\begin{array}{l}{\displaystyle \left|{\widehat{m}}_{add}^{\widehat{\mathit{\beta }}}\left(\mathbf{x}\right)-{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)\right|}\\ =\sum _{\mathcal{l}=1}^d{\widehat{\eta }}_{\mathcal{l}}^{\widehat{\mathit{\beta }}}\left(x_{\mathcal{l}}\right)+{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\widehat{\mathit{\beta }}}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}-\sum _{\mathcal{l}=1}^d{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)-{\int }_{{\mathbb{R}}^d}{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}\hfill \\ =\sum _{\mathcal{l}=1}^d\left({\widehat{\eta }}_{\mathcal{l}}^{\widehat{\mathit{\beta }}}\left(x_{\mathcal{l}}\right)-{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)\right)+{\int }_{{\mathbb{R}}^d}\left({\widehat{m}}_n^{\widehat{\mathit{\beta }}}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}-{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}\right).\hfill \end{array}$$

(64)

Recall that

$${\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)={\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)-{\int }_{\mathbb{R}}{\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)q_{\mathcal{l}}\left({\mathbf{x}}_{\mathcal{l}}\right)d{\mathbf{x}}_{\mathcal{l}},$$

where

$${\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)={\displaystyle \frac{2^{\lambda \left(n\right)}}{n}}\sum _{i=1}^n{\displaystyle \frac{{\tilde{Y}}_{i,n}^{\mathit{\beta }}}{f_{\mathcal{l}}\left(X_{\mathcal{l},i}\right)}}{K}_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{\mathcal{l},i}\right),$$

and

$${\tilde{Y}}_{i,n}^{\mathit{\beta }}=\left(\mathsf{\Psi }\left({\mathbf{Y}}_i\right)-{\mathbf{Z}}_i^{\top }\mathit{\beta }\right){\int }_{{\mathbb{R}}^{d-1}}{2}^{(d-1)\lambda \left(n\right)}{K}_{-\mathcal{l}}\left(2^{\lambda \left(n\right)}{\mathbf{x}}_{-\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{-\mathcal{l},i}\right){\displaystyle \frac{q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)}{f\left({\mathbf{X}}_{-\mathcal{l},i}\mid X_{\mathcal{l},i}\right)}}d{\mathbf{x}}_{-\mathcal{l}}.$$

Consequently,

$$\begin{array}{l}{\displaystyle {\widehat{\eta }}_{\mathcal{l}}^{\widehat{\mathit{\beta }}}\left(x_{\mathcal{l}}\right)-{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)}\\ ={\widehat{\zeta }}_{n,\mathcal{l}}^{\widehat{\mathit{\beta }}}\left(x_{\mathcal{l}}\right)-{\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)-{\int }_{\mathbb{R}}\left({\widehat{\zeta }}_{n,\mathcal{l}}^{\widehat{\mathit{\beta }}}\left(x_{\mathcal{l}}\right)-{\widehat{\zeta }}_{n,\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)\right)q_{\mathcal{l}}\left({\mathbf{x}}_{\mathcal{l}}\right)d{\mathbf{x}}_{\mathcal{l}}\hfill \\ ={\displaystyle \sum _{i=1}^n}{\displaystyle \frac{2^{\lambda \left(n\right)}\left({\tilde{Y}}_{i,n}^{\widehat{\mathit{\beta }}}-{\tilde{Y}}_{i,n}^{\mathit{\beta }}\right)}{n{f}_{\mathcal{l}}\left(X_{\mathcal{l},i}\right)}}\left[K_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{\mathcal{l},i}\right)-{\int }_{\mathbb{R}}{K}_{\mathcal{l}}\left(2^{\lambda \left(n\right)}{x}_{\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{\mathcal{l},i}\right)q_{\mathcal{l}}\left({\mathbf{x}}_{\mathcal{l}}\right)d{\mathbf{x}}_{\mathcal{l}}\right].\hfill \end{array}$$

(65)

It is immediately apparent that

$$\begin{array}{l}{\displaystyle {\tilde{Y}}_{i,n}^{\widehat{\mathit{\beta }}}-{\tilde{Y}}_{i,n}^{\mathit{\beta }}}\\ ={\mathbf{Z}}_i^{\top }\left(\widehat{\mathit{\beta }}-\mathit{\beta }\right){\int }_{{\mathbb{R}}^{d-1}}{2}^{(d-1)\lambda \left(n\right)}{K}_{-\mathcal{l}}\left(2^{\lambda \left(n\right)}{\mathbf{x}}_{-\mathcal{l}},2^{\lambda \left(n\right)}{\mathbf{X}}_{-\mathcal{l},i}\right){\displaystyle \frac{q_{-\mathcal{l}}\left({\mathbf{x}}_{-\mathcal{l}}\right)}{f\left({\mathbf{X}}_{-\mathcal{l},i}\mid X_{\mathcal{l},i}\right)}}d{\mathbf{x}}_{-\mathcal{l}}.\hfill \end{array}$$

(66)

Observe that

$$\begin{array}{l}\sqrt{{\displaystyle \frac{n}{2^{\lambda \left(n\right)}}}}\left(\widehat{\mathit{\beta }}-\mathit{\beta }\right)\hfill \\ ={\left(n^{-1}{\tilde{\mathbf{Z}}}^{\top }\tilde{\mathbf{Z}}\right)}^{-1}{\displaystyle \frac{1}{\sqrt{n{2}^{\lambda \left(n\right)}}}}\left[{\displaystyle \sum _{i=1}^n}{\tilde{\mathbf{Z}}}_i{\tilde{m}}_{add}\left({\mathbf{X}}_i\right)-{\displaystyle \sum _{i=1}^n}{\tilde{\mathbf{Z}}}_i{\displaystyle \sum _{j=1}^n}{W}_{nj}\left({\mathbf{X}}_i\right){\epsilon }_j+{\displaystyle \sum _{i=1}^n}{\tilde{\mathbf{Z}}}_i{\epsilon }_i\right].\hfill \end{array}$$

(67)

Then,

$$\begin{array}{ccc}\hfill {\displaystyle \sum _{i=1}^n}{\tilde{\mathbf{Z}}}_i{\tilde{m}}_{add}\left({\mathbf{X}}_i\right)& =& {\displaystyle \sum _{i=1}^n}{\mathbf{Z}}_i{\tilde{m}}_{add}\left({\mathbf{X}}_i\right)-{\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n}{W}_{nj}\left({\mathbf{X}}_i\right){\mathbf{Z}}_j{\tilde{m}}_{add}\left({\mathbf{X}}_i\right),\hfill \end{array}$$

(68)

where

$$\begin{array}{ccc}\hfill \underset{1\le i\le n}{max}\left|{\tilde{m}}_{add}\left({\mathbf{X}}_i\right)\right|& =& \underset{1\le i\le n}{max}\left|m_{add}\left({\mathbf{X}}_i\right)-{\displaystyle \sum _{j=1}^n}{m}_{add}\left({\mathbf{X}}_j\right)W_{nj}\left({\mathbf{X}}_i\right)\right|\hfill \\ & \le & \underset{1\le i\le n}{max}\left|m_{add}\left({\mathbf{X}}_i\right)-{\widehat{m}}_{add}^{\mathsf{\Psi },\mathit{\beta }}\left({\mathbf{X}}_i\right)\right|+\underset{1\le i\le n}{max}\left|{\displaystyle \sum _{j=1}^n}{W}_{nj}\left({\mathbf{X}}_i\right){\epsilon }_j\right|.\hfill \end{array}$$

(69)

By Lemma 1 of [125], under hypotheses (G.1-3), (K.1-3), (M.1), and (Q.1-2), it follows that

$$\begin{array}{c}\hfill \underset{1\le i\le n}{max}\left|m_{add}\left({\mathbf{X}}_i\right)-{\widehat{m}}_{add}^{\mathsf{\Psi },\mathit{\beta }}\left({\mathbf{X}}_i\right)\right|=\mathcal{O}\left(\sqrt{\frac{2^{\lambda \left(n\right)}logn}{n}}\right).\end{array}$$

(70)

Moreover, it is straightforward to see that

$$\underset{1\le i\le n}{max}\left|{\displaystyle \sum _{j=1}^n}{W}_{nj}\left({\mathbf{X}}_i\right)\right|=\mathcal{O}\left(1\right).$$

By Theorem 1.1 of [126], choosing $c_n=\sqrt{2n{log}_2\left(n\right)}$ yields

$$\begin{array}{c}\hfill {\displaystyle \sum _{i=1}^n}{\mathbf{Z}}_i=\mathcal{O}\left(\sqrt{2n{log}_2\left(n\right)}\right).\end{array}$$

(71)

Hence, combining (69), (70), and (71) with hypothesis (H.2), we obtain

$$\begin{array}{c}\hfill {\displaystyle \sum _{i=1}^n}{\mathbf{Z}}_i{\tilde{m}}_{add}\left({\mathbf{X}}_i\right)=o\left(\sqrt{n{2}^{\lambda \left(n\right)}}\right).\end{array}$$

(72)

Next, consider

$$\begin{array}{l}{\displaystyle {\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n}{W}_{nj}\left({\mathbf{X}}_i\right){\mathbf{Z}}_j{\tilde{m}}_{add}\left({\mathbf{X}}_i\right)}\\ \le n{\displaystyle \underset{1\le i\le n}{max}}{\tilde{m}}_{add}\left({\mathbf{X}}_i\right){\displaystyle \underset{1\le i\le n}{max}}{\displaystyle \underset{1\le j\le n}{max}}{W}_{nj}\left({\mathbf{X}}_i\right){\displaystyle \sum _{j=1}^n}{\mathbf{Z}}_j.\hfill \end{array}$$

(73)

A similar argument, using

$$\underset{1\le i\le n}{max}\underset{1\le j\le n}{max}{W}_{nj}\left({\mathbf{X}}_i\right)=\mathcal{O}\left(n^{-1}{2}^{\lambda \left(n\right)}\right),$$

implies

$$\begin{array}{c}\hfill {\displaystyle \sum _{i=1}^n}{\displaystyle \sum _{j=1}^n}{W}_{nj}\left({\mathbf{X}}_i\right){\mathbf{Z}}_j{\tilde{m}}_{add}\left({\mathbf{X}}_i\right)=o\left(\sqrt{n{2}^{\lambda \left(n\right)}}\right).\end{array}$$

(74)

Therefore, by (68), (72), and (74), it follows that

$$\begin{array}{c}\hfill {\displaystyle \sum _{i=1}^n}{\tilde{\mathbf{Z}}}_i{\tilde{m}}_{add}\left({\mathbf{X}}_i\right)=o\left(\sqrt{n{2}^{\lambda \left(n\right)}}\right).\end{array}$$

(75)

Applying the same arguments, we also deduce

$$\begin{array}{c}\hfill {\displaystyle \sum _{i=1}^n}{\tilde{\mathbf{Z}}}_i{\displaystyle \sum _{j=1}^n}{W}_{nj}\left({\mathbf{X}}_i\right){\epsilon }_j=o\left(\sqrt{n{2}^{\lambda \left(n\right)}}\right),\end{array}$$

(76)

and

$$\begin{array}{c}\hfill {\displaystyle \sum _{i=1}^n}{\tilde{\mathbf{Z}}}_i{\epsilon }_i=o\left(\sqrt{n{2}^{\lambda \left(n\right)}}\right).\end{array}$$

(77)

Finally, combining (67), (75), (76), and (77) with the fact that

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n}{\tilde{\mathbf{Z}}}_i^{\top }{\tilde{\mathbf{Z}}}_i=B,$$

we conclude that

$$\begin{array}{c}\hfill \sqrt{{\displaystyle \frac{n}{2^{\lambda \left(n\right)}}}}\left(\widehat{\mathit{\beta }}-\mathit{\beta }\right)=o\left(1\right).\end{array}$$

(78)

Hence, from (65), (66), (67), (78), and hypothesis (H.3), we obtain

$$\begin{array}{c}\hfill {\displaystyle \sum _{\mathcal{l}=1}^d}\left({\widehat{\eta }}_{\mathcal{l}}^{\widehat{\mathit{\beta }}}\left(x_{\mathcal{l}}\right)-{\widehat{\eta }}_{\mathcal{l}}^{\mathit{\beta }}\left(x_{\mathcal{l}}\right)\right)=o{\left(\frac{logn}{n}\right)}^{a\left(s-\frac{1}{p}\right)}.\end{array}$$

(79)

Next, observe that

$$\begin{array}{l}{\displaystyle {\widehat{m}}_n^{\widehat{\mathit{\beta }}}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}-{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}}\\ =2^{\lambda \left(n\right)d}{\displaystyle \sum _{i=1}^n}{\displaystyle \frac{{\mathbf{Z}}_i^{\top }\left(\widehat{\mathit{\beta }}-\mathit{\beta }\right)}{n{f}_{\mathbf{X}}\left({\mathbf{X}}_i\right)}}K\left(2^{\lambda \left(n\right)}\mathbf{z},2^{\lambda \left(n\right)}{\mathbf{X}}_i\right).\hfill \end{array}$$

(80)

By (78) and hypothesis (H.3), it follows that

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^d}\left({\widehat{m}}_n^{\widehat{\mathit{\beta }}}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}-{\widehat{m}}_n^{\mathit{\beta }}\left(\mathbf{z}\right)q\left(\mathbf{z}\right)d\mathbf{z}\right)=o{\left(\frac{logn}{n}\right)}^{a\left(s-\frac{1}{p}\right)}.\end{array}$$

(81)

Therefore, by (64), (79), and (81), we conclude that

$$\begin{array}{c}\hfill \underset{\mathbf{x}\in \mathcal{C}}{sup}\left|{\widehat{m}}_{add}^{\widehat{\mathit{\beta }}}\left(\mathbf{x}\right)-{\widehat{m}}_{add}^{\mathit{\beta }}\left(\mathbf{x}\right)\right|=o{\left(\frac{logn}{n}\right)}^{a\left(s-\frac{1}{p}\right)}.\end{array}$$

(82)

Finally, combining (37), (48), (49), (50), 63, and (82) completes the proof of the theorem.

Observe that, up to this point, we have focused on the scenario in which the density function $f_{\mathbf{X}}(·)$ is known. When $f_{\mathbf{X}}(·)$ is unknown, however, it can be estimated by the empirical function $f_n(·)$. In this case, we use the following decomposition:

$${\displaystyle \frac{1}{f_n}}={\displaystyle \frac{1}{f_{\mathbf{X}}}}+{\displaystyle \frac{f_{\mathbf{X}}-f_n}{f_{\mathbf{X}}{f}_n}}.$$

According to [70], under conditions (G.1)–(G.3), it holds on every compact subset $I^d\subset {\mathbb{R}}^d$ that

$$\begin{array}{c}\hfill \underset{\mathbf{x}\in I^d}{sup}\left|f_n\left(\mathbf{x}\right)-f_{\mathbf{X}}\left(\mathbf{x}\right)\right|=\mathcal{O}\left({\left(\frac{logn}{n}\right)}^{\frac{s}{d+2s}}\right).\end{array}$$

(83)

By combining this result with the preceding findings, we can now complete the proof of the theorem. □