Pointwise Estimation of Anisotropic Regression Functions Using Wavelets with Data-Driven Selection Rule

Abstract

For nonparametric regression estimation, conventional research all focus on isotropic regression function. In this paper, a linear wavelet estimator of anisotropic regression function is constructed, the rate of convergence of this estimator is discussed in anisotropic Besov spaces. More importantly, in order to obtain an adaptive estimator, a regression estimator is proposed with scaling parameter data-driven selection rule. It turns out that our results attain the optimal convergence rate of nonparametric pointwise estimation.

1. Introduction

The classical regression estimation model considers independent and identically distributed (i.i.d.) random variables $(U_1,V_1),\cdots ,(U_n,V_n)$, which are defined on ${[0,1]}^d\times \mathbb{R}$, and a regression function given by

$$r(x):=E(\rho (V)|U=x)=\frac{{\int }_{\mathbb{R}}\rho (y)g(x,y)dy}{h(x)},x\in {[0,1]}^d.$$

(1)

In this setting, $\rho$ is a known function, g stands for the density function of the random variables $(U,V)$, and h is the density function of U. The aim of this classical model is to estimate the unknown regression function $r(x)$ from the data $\{(U_i,V_i),i=1,\dots ,n\}$. However, the data $(U_1,V_1),\cdots ,(U_n,V_n)$ are not directly observed in some situations that only exhibit a version of them contaminated by some noise. For this problem, this paper considers the biased sampling regression model as follows. Let $(X_1,Y_1),\cdots ,(X_n,Y_n)$ be i.i.d. random variables with the common density function

$$f(x,y)=\frac{\omega (x,y)g(x,y)}{\mu },(x,y)\in {[0,1]}^d\times \mathbb{R},$$

(2)

where $\omega$ is a known function, $\mu :=E(\omega (U,V))<+\infty$. We want to estimate the unknown regression function $r(x)$ based on the observed data $\{(X_i,Y_i),1\le i\le n\}$.

The above biased sampling regression model arises in many practical applications, for instance, in [1,2,3]. In order to illustrate an application, take the relationship of company business income V and research and development (R&D) investment U in a country. In order to raise working efficiency, the researchers narrow the sample range and obtain the data $(X,Y)$ from some special companies that have higher business income X and R&D investment Y. Because some special companies are chosen to provide data, the density f of $(X,Y)$ can be considered as given by $f(x,y)=\frac{\omega (x,y)g(x,y)}{\mu }$ with a known biased sampling function $\omega$ and the density g of $(U,V)$. Then, the researchers can estimate the relationship function $r(x)$ of the business income and R&D investment in the country using the observed data $\{(X_i,Y_i),1\le i\le n\}$.

For nonparametric estimation problems, the wavelet method has been widely used by the local property in both time and frequency domain; see [4,5,6,7,8]. For the above biased sampling regression model, the mean integrated square error ($L^2$ risk) of linear and nonlinear wavelet estimators is discussed by [9]. An optimal convergence rate of wavelet estimators over the $L^p(1\le p<+\infty )$ risk is considered by [10,11]. Ref. [12] studied the pointwise $l^p(1\le p<+\infty )$ risk of wavelet estimators under independent conditions. However, all those results require that the regression functions are isotropic functions (a multivariate function $f(x)$ is isotropic if its smoothing parameters are the same at each coordinate axis direction). Moreover, the key scaling parameters of the above conventional wavelet estimators are chosen based only on the sample size n of the observed data $\{(X_i,Y_i),1\le i\le n\}$. Any other important information about the observed data is neglected. More importantly, the definitions of the conventional wavelet estimators usually do not bring with them a rule to choose the smoothing parameter of the regression function, which means that the conventional wavelet estimators are not adaptive.

In this paper, we firstly construct a linear wavelet estimator for an anisotropic regression function (a multivariate function $f(x)$ is anisotropic if its smoothing parameters are different in the coordinate directions; see [13,14]) and study the pointwise $l^p(1\le p<+\infty )$ risk of the wavelet estimator in anisotropic Besov spaces. In order to overcome the shortage of conventional wavelet estimators, a data-driven selection of a wavelet estimator is proposed. The choice of the scaling parameter of our data-driven estimator depends on not only the sample size but also on other important information of the observed data. Furthermore, the definition of the data-driven estimator is determined by the observed data, which means that our data-driven estimator is completely adaptive. Finally, it should be pointed out that our results attain the optimal convergence rate of nonparametric pointwise estimation when the anisotropic regression function reduces to isotropic function.

The structure of this paper is organized as follows. The definitions and properties of wavelet and Besov space are given in Section 2. Section 3 will propose a linear wavelet estimator, and study the rate of convergence of this estimator in anisotropic Besov spaces. Furthermore, a data-driven estimator with scaling parameter data-driven selection rule is constructed in Section 4.

2. Anisotropic Wavelets and Besov Space

In order to construct wavelet estimators for anisotropic regression functions, some basic concepts of the anisotropic and orthonormal wavelet basis will be given in the following. More details can be found in [14,15,16,17,18,19,20,21,22,23].

Let $\{V_j,j\in \mathbb{Z}\}$ be a classical orthonormal multiresolution analysis of $L^2(\mathbb{R})$ with a compactly supported scaling function $\tilde{\varphi }$ and wavelet function $\tilde{\psi }$. A function space generated by the wavelet functions $\{2^{j/2}\tilde{\psi }(2^{j}y-l),l\in \mathbb{Z}\}$ is denoted as $W_j$. For $\tau =({\tau }_1,\cdots ,{\tau }_d)$ with ${\tau }_1,\cdots ,{\tau }_d>0$ and ${\tau }_1+\cdots +{\tau }_d=d$, we define

$$V_j^{{\tau }_i}:=V_{\lfloor j{\tau }_i\rfloor },$$

where $j\ge 0$, $i\in \{1,\cdots ,d\}$ and $\lfloor y\rfloor$ denotes the integer part of y. Then, for $x=(x_1,\cdots ,x_d)\in {\mathbb{R}}^d$ and $k=(k_1,\cdots ,k_d)\in {\mathbb{Z}}^d$ (the multiple integer set),

$${\varphi }_{j\tau ,k}(x):=\prod _{i=1}^d{\tilde{\varphi }}_{\lfloor j{\tau }_i\rfloor ,k_i}(x_i)=\prod _{i=1}^d{2}^{\lfloor j{\tau }_i\rfloor /2}\tilde{\varphi }(2^{\lfloor j{\tau }_i\rfloor }{x}_i-k_i)$$

constitutes an orthonormal basis of $V_j^{\tau }:={\displaystyle \underset{i=1}{\overset{d}{⨂}}}{V}_j^{{\tau }_i}$.

The notation $V_j^{{\tau }_i}$ is reasonable since it depends on $\lfloor j{\tau }_i\rfloor$ for all $i\in \{1,\cdots d\}$. It is easy to see that with $\mathsf{\Gamma }={\{0,1\}}^d\setminus {\{0\}}^d$,

$$V_{j+1}^{\tau }=\underset{i=1}{\overset{d}{⨂}}{V}_{j+1}^{{\tau }_i}=\underset{i=1}{\overset{d}{⨂}}(V_{\lfloor j{\tau }_i\rfloor }⨁\underset{l=\lfloor j{\tau }_i\rfloor }{\overset{\lfloor (j+1){\tau }_i\rfloor -1}{⨁}}{W}_l)=V_j^{\tau }⨁\underset{\gamma \in \mathsf{\Gamma }}{⨁}\underset{i=1}{\overset{d}{⨂}}{W}_{\lfloor j{\tau }_i\rfloor }^{{\gamma }_i},$$

where $\gamma =({\gamma }_1,\cdots ,{\gamma }_d)$, $W_{\lfloor j{\tau }_i\rfloor }^{{\gamma }_i}:=V_{\lfloor j{\tau }_i\rfloor }$ with ${\gamma }_i=0$, and $W_{\lfloor j{\tau }_i\rfloor }^{{\gamma }_i}:={\displaystyle \underset{l=\lfloor j{\tau }_i\rfloor }{\overset{\lfloor (j+1){\tau }_i\rfloor -1}{⨁}}}{W}_l$ for ${\gamma }_i=1$. Then, the above equation can be rewritten as

$$V_{j+1}^{\tau }=V_j^{\tau }⨁W_j^{\tau };W_j^{\tau }=\underset{\gamma \in \mathsf{\Gamma }}{⨁}\underset{i=1}{\overset{d}{⨂}}{W}_{\lfloor j{\tau }_i\rfloor }^{{\gamma }_i}.$$

Now, we give the anisotropic wavelet basis of $W_j^{\tau }$. Define the indicator set $I_{j\tau }:=\{(\gamma ,m):\gamma \in \mathsf{\Gamma },m\in {\mathbb{Z}}^d,j\in \mathbb{Z}\}$. For any $i\in \{1,\cdots ,d\}$, $m_i=\lfloor j{\tau }_i\rfloor$ with ${\gamma }_i=0$, $m_i\in [\lfloor j{\tau }_i\rfloor ,\lfloor (j+1){\tau }_i\rfloor ]$ in the case of ${\gamma }_i=1$.

Define

$${\psi }_{j\tau ,k}^{(\gamma ,m)}:=2^{\frac{|m|}{2}}\prod _{i=1}^d{\psi }^{{\gamma }_i}(2^{m_i}{x}_i-k_i),$$

where $(\gamma ,m)\in I_{j\tau }$ and $|m|=m_1+\cdots +m_d$. Moreover, ${\psi }^{{\gamma }_i}=\tilde{\psi }$ with ${\gamma }_i=1$ and ${\psi }^{{\gamma }_i}=\tilde{\varphi }$ when ${\gamma }_i=0$. For $j_0\ge 0$,

$$\{{\varphi }_{j_0\tau ,k},{\psi }_{j\tau ,k}^{(\gamma ,m)};j\ge j_0,(\gamma ,m)\in I_{j\tau },k\in {\mathbb{Z}}^d\}$$

constitutes an orthonormal basis of $L^2({\mathbb{R}}^d)$. Then, for each $f(x)\in L^2({\mathbb{R}}^d)$,

$$f(x)=\sum _{k\in {\mathbb{Z}}^d}{\alpha }_{j_0\tau ,k}{\varphi }_{j_0\tau ,k}(x)+\sum _{j=j_0}^{\infty }\sum _{(\gamma ,m)\in I_{j\tau }}\sum _{k\in {\mathbb{Z}}^d}{\beta }_{j\tau ,k}^{(\gamma ,m)}{\psi }_{j\tau ,k}^{(\gamma ,m)}(x)$$

with ${\alpha }_{j_0\tau ,k}={\int }_{{\mathbb{R}}^d}f(x){\varphi }_{j_0\tau ,k}(x)dx$ and ${\beta }_{j\tau ,k}^{(\gamma ,m)}={\int }_{{\mathbb{R}}^d}f(x){\psi }_{j\tau ,k}^{(\gamma ,m)}(x)dx$.

Let $P_{j\tau }$ be the orthogonal projection operator from $L^2({\mathbb{R}}^d)$ onto $V_j^{\tau }$ with the orthonormal basis $\{{\varphi }_{j\tau ,k}(x),k\in {\mathbb{Z}}^d\}$. In this paper, we choose the Daubechies scaling function as $\tilde{\varphi }$; then, the function $\varphi \in L({\mathbb{R}}^d)\cap L^{\infty }({\mathbb{R}}^d)$ and ${\displaystyle \sum _{k\in {\mathbb{Z}}^d}}\varphi (x-k)\overline{\varphi (y-k)}$ converges absolutely almost everywhere. It can be shown that for $f(x)\in L^p({\mathbb{R}}^d)(1\le p\le \infty )$,

$$P_{j\tau }f(x):=\sum _{k\in {\mathbb{Z}}^d}{\alpha }_{j\tau ,k}{\varphi }_{j\tau ,k}(x)$$

(3)

holds almost everywhere on ${\mathbb{R}}^d$.

Besov spaces are important in theory and applications that contain Hölder and $L^2$ Sobolev spaces as special cases. Now, we give the definition of the anisotropic Besov space $B_{p,q}^{\overrightarrow{s}}({\mathbb{R}}^d)$ ([14]). Let

$${\mathsf{\Delta }}_{t,i}f(x):=f(x+t{e}_i)-f(x),{\mathsf{\Delta }}_{t,i}^{M_i}f(x):=(\underset{M_i}{\underbrace{{\mathsf{\Delta }}_{t,i}\cdots {\mathsf{\Delta }}_{t,i}f}})(x)$$

with $e_i=(\underset{i-1}{\underbrace{0,\cdots ,0}},1,0,\cdots ,0)$, $t\in \mathbb{R}$, and $M_i\in {\mathbb{N}}^{+}(i=1,2,\dots ,d)$.

Definition 1.

Let $1\le p,q<+\infty$, $\overrightarrow{s}=(s_1,\dots ,s_d)$, $\overrightarrow{M}=(M_1,\dots ,M_d)$, and $0<s_i<M_i$. Then,

$$B_{p,q}^{\overrightarrow{s}}({\mathbb{R}}^d)=\{f\in L^p({\mathbb{R}}^d){:\parallel f\parallel }_{\overrightarrow{s}pq}^{\overrightarrow{M}}<+\infty \}.$$

${\parallel f\parallel }_{\overrightarrow{s}pq}^{\overrightarrow{M}}={\parallel f\parallel }_p+{\displaystyle \sum _{i=1}^d}({\int }_0^1{t}^{-s_{i}q}\parallel {\mathsf{\Delta }}_{t,i}^{M_i}f{{\parallel }_p^q\frac{dt}{t})}^{\frac{1}{q}}$ is the Besov norm of f.

The following lemma gives the wavelet description of the anisotropic Besov space.

Lemma 1.

Let $\overrightarrow{s}:=(s_1,\dots ,s_d),s_i>0$, $\frac{1}{s(d)}=\frac{1}{d}{\displaystyle \sum _{i=1}^d}\frac{1}{s_i}$, and ${\tau }_i=\frac{s(d)}{s_i}$. Then, the following assertions are equivalent:

(i) $f\in B_{p,q}^{\overrightarrow{s}}({\mathbb{R}}^d);$

(ii) ${\parallel f\parallel }_{B_{p,q}^{\overrightarrow{s}}}:=\parallel {\alpha }_{j_0\tau ·}{\parallel }_p+\{{\displaystyle \sum _{j=j_0}^{\infty }}{\displaystyle \sum _{(\gamma ,m)\in I_{j\tau }}}[2^{j(s(d)-\frac{d}{p})+\frac{|m|}{2}}\parallel {\beta }_{j\tau ·}^{(\gamma ,m)}{\parallel }_p{{]}^q\}}^{\frac{1}{q}}<+\infty ;$

(iii) $\parallel {\beta }_{j\tau ·}^{(\gamma ,m)}{\parallel }_p\lesssim 2^{-j(s(d)-\frac{d}{p})-\frac{|m|}{2}},$

• where $\parallel {\alpha }_{j_0\tau ·}{\parallel }_p^p={\displaystyle \sum _{k\in {\mathbb{Z}}^d}}{|{\alpha }_{j_0\tau ,k}|}^p$ and $\parallel {\beta }_{j\tau ·}^{(\gamma ,m)}{\parallel }_p^p={\displaystyle \sum _{k\in {\mathbb{Z}}^d}}{|{\beta }_{j\tau ,k}^{(\gamma ,m)}|}^p$.

Here, $A\lesssim B$ denotes $A\le cB$ for some constant $c>0$; $A\gtrsim B$ means $B\lesssim A$; and $A\backsim B$ stands for both $A\lesssim B$ and $A\gtrsim B$. In this paper, we assume $r(x)\in B_{p,q}^{\overrightarrow{s}}(H)$ with $H>0$, where

$$B_{p,q}^{\overrightarrow{s}}(H):=\{f\in B_{p,q}^{\overrightarrow{s}}({\mathbb{R}}^d){:\parallel f\parallel }_{B_{p,q}^{\overrightarrow{s}}}\le H\}.$$

3. Linear Wavelet Estimation

In this section, a linear wavelet estimator for an anisotropic regression function $r(x)$ is constructed, and a convergence rate of this wavelet estimator is proved under some mild assumptions. Now, we firstly give some assumptions about the regression models (1) and (2). Hereafter, general positive constants will be denoted by $c_1,c_2,\dots ,$ where they can be distinct or not.

Hypothesis 1.

The density function h of a random variable U has a positive lower bound,

$${\displaystyle \underset{x\in {[0,1]}^d}{inf}}h(x)\ge c_1>0.$$

Hypothesis 2.

The function ρ in the Equation (1) satisfies $\rho \in L(\mathbb{R})\cap L^{\infty }(\mathbb{R})$.

Hypothesis 3.

For any $(x,y)\in {[0,1]}^d\times \mathbb{R}$, there exist constants $0<c_2<c_3<\infty$ such that

$$0<c_2<\omega (x,y)<c_3<\infty .$$

Note that Hypotheses 1 and 3 and are standard conditions for a nonparametric regression model with biased data ([9,24,25]). Moreover, the condition $y\in [a,b]$ is required in [24,25]. However, this paper does not need this harsh condition. A new mild condition Hypothesis 2 is proposed, which is the same as in [11,12].

Now, a linear wavelet estimator is defined by

$${\widehat{r}}_{j\tau }(x)={\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j\tau }}}{\widehat{\alpha }}_{j\tau ,k}{\varphi }_{j\tau ,k}(x),$$

(4)

where the cardinality of ${\mathsf{\Omega }}_{j\tau }$ satisfies $|{\mathsf{\Omega }}_{j\tau }|\sim 2^{{\displaystyle \sum _{i=1}^d}\lfloor j{\tau }_i\rfloor }$,

$${\widehat{\mu }}_n={\left[\frac{1}{n}{\displaystyle \sum _{i=1}^n}\frac{1}{\omega (X_i,Y_i)}\right]}^{-1},$$

(5)

$${\widehat{\alpha }}_{j\tau ,k}=\frac{{\widehat{\mu }}_n}{n}{\displaystyle \sum _{i=1}^n}\frac{\rho (Y_i)}{\omega (X_i,Y_i)h(X_i)}{\varphi }_{j\tau ,k}(X_i).$$

(6)

Similar to the results of Lemma 2 in [11], one can easily obtain

$$E({\widehat{\mu }}_n^{-1})={\mu }^{-1},E(\frac{\mu \rho (Y_i)}{\omega (X_i,Y_i)h(X_i)}{\varphi }_{j\tau ,k}(X_i))={\alpha }_{j\tau ,k}.$$

On the other hand, the following lemma is obtained using Rosenthal’s inequality. Its proof is similar to Proposition 4.1 in [12].

Lemma 2.

If  Hypotheses 1–3  hold and $2^{|j\tau |}\le n$ with $|j\tau |:={\displaystyle \sum _{i=1}^d}\lfloor j{\tau }_i\rfloor$, then for $p\in [1,\infty )$,

$$E|{\widehat{\alpha }}_{j\tau ,k}-{\alpha }_{j\tau ,k}{|}^p\lesssim n^{-\frac{p}{2}}.$$

This paper considers nonparametric regression estimation in anisotropic Besov spaces. The following lemma for Besov space is crucial ([14,26]).

Lemma 3.

Let $r\in B_{\tilde{p},q}^{\overrightarrow{s}}(H)(\tilde{p},q\in [1,\infty ))$ and $s(d)>d/\tilde{p}$. Then,

$$\begin{array}{c}\hfill {\displaystyle \sum _{(\gamma ,m)\in I_{j\tau }}}{\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j\tau }}}|{\beta }_{j\tau ,k}^{(\gamma ,m)}||{\psi }_{j\tau ,k}^{(\gamma ,m)}(x)|\lesssim 2^{-j(s(d)-d/\tilde{p})}.\end{array}$$

Now, we are in this position to state the convergence rate of the linear wavelet estimator.

Theorem 1.

Consider the problem defined by (1) and (2) with Hypotheses 1–3, $r(x)\in B_{\tilde{p},q}^{\overrightarrow{s}}(H)(\tilde{p},q\in [1,\infty ))$, and $s(d)>d/\tilde{p}$. The linear wavelet estimator ${\widehat{r}}_{j\tau }(x)$ is defined by (4) with $2^{\lfloor j{\tau }_i\rfloor }\sim n^{\frac{s(d)}{2(s(d)-d/\tilde{p})+d}\frac{1}{s_i}}$. Then,

$$E|{\widehat{r}}_{j\tau }{(x)-r(x)|}^p\lesssim n^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}$$

(7)

for every $x\in {[0,1]}^d$.

Remark 1.

When $s_i$ is constant for every $i=1,2,\dots ,d$, the anisotropic function spaces reduce to isotropic spaces. Then, our result is the same as the optimal convergence rate of standard nonparametric pointwise estimation ([27,28]).

Proof. 

It is easy to see that

$$\begin{array}{c}\hfill E|{\widehat{r}}_{j\tau }{(x)-r(x)|}^p\lesssim E|{\widehat{r}}_{j\tau }(x)-P_{j\tau }{r(x)|}^p+{|P_{j\tau }r(x)-r(x)|}^p.\end{array}$$

(8)

For the upper bound of $|P_{j\tau }{r(x)-r(x)|}^p$, according to the Hölder inequality and Lemma 3,

$$\begin{array}{ccc}\hfill |P_{j\tau }{r(x)-r(x)|}^p& \le & ({\displaystyle \sum _{j^{\prime }=j}^{\infty }}{\displaystyle \sum _{(\gamma ,m)\in I_{j^{\prime }\tau }}}{\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j^{\prime }\tau }}}|{\beta }_{j^{\prime }\tau ,k}^{(\gamma ,m)}||{\psi }_{j^{\prime }\tau ,k}^{(\gamma ,m)}(x){|)}^p\hfill \\ & \lesssim & 2^{-jp(s(d)-d/\tilde{p})}.\hfill \end{array}$$

(9)

For the upper bound of $E|{\widehat{r}}_{j\tau }(x)-P_{j\tau }{r(x)|}^p$, it follows from the Hölder inequality ($\frac{1}{p}+\frac{1}{p^{\prime }}=1$) and Lemma 2 that

$$\begin{array}{ccc}\hfill E|{\widehat{r}}_{j\tau }(x)-P_{j\tau }{r(x)|}^p& & \le E\{{\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j\tau }}}|{\widehat{\alpha }}_{j\tau ,k}-{\alpha }_{j\tau ,k}||{\varphi }_{j\tau ,k}(x){|\}}^p\hfill \\ & & \le E({\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j\tau }}}|{\widehat{\alpha }}_{j\tau ,k}-{\alpha }_{j\tau ,k}{|}^p|{\varphi }_{j\tau ,k}(x)|)({\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j\tau }}}|{\varphi }_{j\tau ,k}(x){|)}^{\frac{p}{p^{\prime }}}\hfill \\ & & \lesssim 2^{|j\tau |p/2}{n}^{-\frac{p}{2}}.\hfill \end{array}$$

(10)

Note that $|j\tau |={\displaystyle \sum _{i=1}^d}\lfloor j{\tau }_i\rfloor \le {\displaystyle \sum _{i=1}^d}j{\tau }_i=jd$ and $2^{\lfloor j{\tau }_i\rfloor }\sim n^{\frac{s(d)}{2(s(d)-d/\tilde{p})+d}\frac{1}{s_i}}$. One thus obtains

$$2^{|j\tau |}\le 2^{jd},2^{|j\tau |}\sim n^{\frac{d}{2(s(d)-d/\tilde{p})+d}}.$$

Those with (9) and (10) show that

$$\begin{array}{c}\hfill E|{\widehat{r}}_{j\tau }{(x)-r(x)|}^p\lesssim 2^{-\frac{|j\tau |}{d}p(s(d)-d/\tilde{p})}+n^{\frac{\frac{1}{2}pd}{2(s(d)-d/\tilde{p})+d}-\frac{p}{2}}\lesssim n^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}.\end{array}$$

□

4. Data-Driven Estimation

In conventional wavelet regression estimators ([12,24,25]) and in the above anisotropic linear wavelet estimator, the choice of the wavelet scaling parameter $j(j\tau )$ of wavelet estimators are based only on the sample size n of the observed data $(X_1,Y_1),\dots ,(X_n,Y_n)$. Because the definitions of these wavelet estimators are related to the smoothing parameter of the regression function, those estimators are not adaptive. In this section, we will construct an adaptive data-driven regression estimator. The scaling parameter selection of this estimator is determined by $(X_1,Y_1),\dots ,(X_n,Y_n)$. Furthermore, the definition of this data-driven estimator does not depend on any information about the regression function.

Now, we define an auxiliary estimator

$${\widehat{r}}_{j\tau ,j^{*}{\tau }^{*}}(x)={\displaystyle \sum _k}{\widehat{\alpha }}_{j\tau \wedge j^{*}{\tau }^{*},k}{\varphi }_{j\tau \wedge j^{*}{\tau }^{*},k}(x),$$

where $j\tau \wedge j^{*}{\tau }^{*}:=j\tau$ for $|j\tau |\le |j^{*}{\tau }^{*}|$; otherwise, $j\tau \wedge j^{*}{\tau }^{*}:=j^{*}{\tau }^{*}$. For a constant $\lambda$ (specified at almost the end of the proof of Theorem 2),

$${\nu }_{j\tau }=\lambda \sqrt{(1+p/2)2^{|j\tau |}max\{1,(ln2)|j\tau |\}/n},$$

(11)

satisfies

$${\nu }_{j\tau }\lesssim \sqrt{n^{-1}|j\tau |2^{|j\tau |}}.$$

(12)

Define $|t|=\lfloor t_1\rfloor +\cdots +\lfloor t_d\rfloor$ and

$$\mathsf{\Theta }:=\{t=(t_1,\cdots ,t_d)\in {\mathbb{R}}^d;0\le \lfloor t_i\rfloor \le {log}_2\frac{n}{lnn},|t|2^{|t|}\le n\}.$$

Then, $j_0{\tau }_0\in \mathsf{\Theta }$ is determined by the following selection rule:

$$(I){\widehat{\xi }}_{j\tau }(x)={\displaystyle \underset{j^{*}{\tau }^{*}\in \mathsf{\Theta }}{max}}[|{\widehat{r}}_{j\tau ,j^{*}{\tau }^{*}}(x)-{\widehat{r}}_{j^{*}{\tau }^{*}}(x){|-{\nu }_{j^{*}{\tau }^{*}}-{\nu }_{j\tau }]}_{+}$$

$$(II){\widehat{\xi }}_{j_0{\tau }_0}(x)+2{\nu }_{j_0{\tau }_0}={\displaystyle \underset{j\tau }{min}}\{{\widehat{\xi }}_{j\tau }(x)+2{\nu }_{j\tau }\}.$$

According to the above selection rule, it is easy to see that the parameter $j_0{\tau }_0$ is decided by the observed data $(X_1,Y_1),\dots ,(X_n,Y_n)$. Now, we give a data-driven estimator ${\widehat{r}}_{j_0{\tau }_0}(x)$,

$${\widehat{r}}_{j_0{\tau }_0}(x)={\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j_0{\tau }_0}}}{\widehat{\alpha }}_{j_0{\tau }_0,k}{\varphi }_{j_0{\tau }_0,k}(x).$$

In order to discuss the convergence rate of this data-driven estimator, the following Lemma ([29]) and Bernstein’s inequality are essential.

Lemma 4.

Let (Ω,ϝ,χ) be a measurable space and let $f(y)\in L^p$(Ω,ϝ,χ) with $p\in (0,\infty )$. Then, with $\delta (t)=\chi \{y\in \mathsf{\Omega }:|f(y)|>t\}$,

$$\begin{array}{c}\hfill {\int }_{\mathsf{\Omega }}{|f(y)|}^{p}d\chi =p{\int }_0^{\infty }{t}^{p-1}\delta (t)dt.\end{array}$$

Bernstein’s inequality Let $X_1,\cdots ,X_n$ be independent random variables such that $E{X}_i=0$, $|X_i|\le M$ and $E{X}_i^2={\sigma }^2$. Then, for each $v\ge 0$,

$$\begin{array}{c}\hfill \mathbb{P}(\frac{1}{n}|{\displaystyle \sum _{i=1}^n}{X}_i|\ge v)\le 2exp\{-\frac{n{v}^2}{2({\sigma }^2+\frac{vM}{3})}\}.\end{array}$$

Theorem 2.

For Problems (1) and (2) with Hypotheses 1–3, $r(x)\in B_{\tilde{p},q}^{\overrightarrow{s}}(H)(\tilde{p},q\in [1,\infty ))$ and $s(d)>d/\tilde{p}$. When $j_0{\tau }_0$ is determined by Selection Rules $(I)$ and $(II)$, the data-driven estimator ${\widehat{r}}_{j_0{\tau }_0}(x)$ satisfies

$$E|{\widehat{r}}_{j_0{\tau }_0}{(x)-r(x)|}^p\lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}$$

(13)

for every $x\in {[0,1]}^d$.

Remark 2.

Note that the convergence rate of this data-driven estimator stays the same as that of the above linear wavelet estimator up to a logarithmic factor ${(lnn)}^{\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}$. These two regression estimators all can attain the optimal convergence rate over pointwise error ([28]). On the other hand, the scaling parameter $j_0{\tau }_0$ of the data-driven estimator ${\widehat{r}}_{j_0{\tau }_0}(x)$ depends on the observed data $(X_1,Y_1),\dots ,(X_n,Y_n)$, not only the sample size n. Additionally, the definition of the data-driven estimator ${\widehat{r}}_{j_0{\tau }_0}(x)$ is not related to any information about the regression function $r(x)$, which means that this data-driven estimator is completely adaptive.

Proof. 

Let $j_1{\tau }_1:=(j_1{\tau }_{1_1},\dots ,j_1{\tau }_{1_d})$ and $2^{\lfloor j_1{\tau }_{1_i}\rfloor }\sim {(\frac{n}{lnn})}^{\frac{s(d)}{2(s(d)-d/\tilde{p})+d}\frac{1}{s_i}}$. Then, one can easily obtain

$$\frac{s(d)}{2(s(d)-d/\tilde{p})+d}\frac{1}{s_i}\le {\displaystyle \sum _{i=1}^d}\frac{s(d)}{2(s(d)-d/\tilde{p})+d}\frac{1}{s_i}<1,$$

$\lfloor j_1{\tau }_{1_i}\rfloor \le {log}_2\frac{n}{lnn}$ and $j_1{\tau }_1\in \mathsf{\Theta }$.

According to Theorem 1 and $2^{\lfloor j_1{\tau }_{1_i}\rfloor }\sim {(\frac{n}{lnn})}^{\frac{s(d)}{2(s(d)-d/\tilde{p})+d}\frac{1}{s_i}}$,

$$\begin{array}{c}\hfill E|{\widehat{r}}_{j_1{\tau }_1}{(x)-r(x)|}^p\lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}.\end{array}$$

(14)

Note that

$$\begin{array}{c}\hfill E|{\widehat{r}}_{j_0{\tau }_0}{(x)-r(x)|}^p\lesssim E|{\widehat{r}}_{j_0{\tau }_0}(x)-{\widehat{r}}_{j_1{\tau }_1}{(x)|}^p+E{|{\widehat{r}}_{j_1{\tau }_1}(x)-r(x)|}^p.\end{array}$$

(15)

Hence, we only need to estimate the upper bound of $E|{\widehat{r}}_{j_0{\tau }_0}(x)-{\widehat{r}}_{j_1{\tau }_1}{(x)|}^p$.

It follows from the selection rule and the definition of an auxiliary estimator that

$$\begin{array}{ccc}\hfill |{\widehat{r}}_{j_0{\tau }_0}(x)-{\widehat{r}}_{j_1{\tau }_1}(x)|& & \le |{\widehat{r}}_{j_0{\tau }_0}(x)-{\widehat{r}}_{j_0{\tau }_0,j_1{\tau }_1}(x)|+|{\widehat{r}}_{j_0{\tau }_0,j_1{\tau }_1}(x)-{\widehat{r}}_{j_1{\tau }_1}(x)|\hfill \\ & & \le ({\widehat{\xi }}_{j_1{\tau }_1}(x)+{\nu }_{j_0{\tau }_0}+{\nu }_{j_1{\tau }_1})+({\widehat{\xi }}_{j_0{\tau }_0}(x)+{\nu }_{j_1{\tau }_1}+{\nu }_{j_0{\tau }_0})\hfill \\ & & \lesssim ({\widehat{\xi }}_{j_1{\tau }_1}(x)+2{\nu }_{j_1{\tau }_1}).\hfill \end{array}$$

Then, one can easily obtain that

$$E|{\widehat{r}}_{j_0{\tau }_0}(x)-{\widehat{r}}_{j_1{\tau }_1}{(x)|}^p\lesssim E{({\widehat{\xi }}_{j_1{\tau }_1}(x))}^p+E{\nu }_{j_1{\tau }_1}^p.$$

(16)

According to (12),

$$E{\nu }_{j_1{\tau }_1}^p\lesssim (n^{-1}|j_1{\tau }_1{|2^{|j_1{\tau }_1|})}^{\frac{p}{2}}\lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}.$$

(17)

for $E{({\widehat{\xi }}_{j_1{\tau }_1}(x))}^p$. Note that ${\widehat{r}}_{j_1{\tau }_1,j^{*}{\tau }^{*}}(x)={\widehat{r}}_{j^{*}{\tau }^{*}}(x)$ in the case of $|j^{*}{\tau }^{*}|\le |j_1{\tau }_1|$. Moreover,

$${\widehat{\xi }}_{j_1{\tau }_1}(x)\le {\displaystyle \underset{|j^{*}{\tau }^{*}|>|j_1{\tau }_1|}{max}}[|{\widehat{r}}_{j_1{\tau }_1}(x)-{\widehat{r}}_{j^{*}{\tau }^{*}}(x){|-{\nu }_{j_1{\tau }_1}-{\nu }_{j^{*}{\tau }^{*}}]}_{+}.$$

It is easy to see from the triangle inequality that $|{\widehat{r}}_{j_1{\tau }_1}(x)-{\widehat{r}}_{j^{*}{\tau }^{*}}(x)|\le |{\widehat{r}}_{j_1{\tau }_1}(x)-P_{j_1{\tau }_1}r(x)|+|P_{j_1{\tau }_1}r(x)-r(x)|+|r(x)-P_{j^{*}{\tau }^{*}}r(x)|+|P_{j^{*}{\tau }^{*}}r(x)-{\widehat{r}}_{j^{*}{\tau }^{*}}(x)|.$ Then, using $|j_1{\tau }_1|\le j_{1}d$, $|j^{*}{\tau }^{*}|>|j_1{\tau }_1|$ and (9), one obtains

$$\begin{array}{c}\hfill |P_{j_1{\tau }_1}{r(x)-r(x)|}^p\lesssim 2^{-j_{1}p(s(d)-d/\tilde{p})}\lesssim 2^{-|j_1{\tau }_1|\frac{p}{d}(s(d)-d/\tilde{p})}\lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}},\end{array}$$

$$\begin{array}{c}\hfill |r(x)-P_{j^{*}{\tau }^{*}}{r(x)|}^p\lesssim 2^{-|j^{*}{\tau }^{*}|\frac{p}{d}(s(d)-d/\tilde{p})}\lesssim 2^{-|j_1{\tau }_1|\frac{p}{d}(s(d)-d/\tilde{p})}\lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}.\end{array}$$

Hence, one knows

$$\begin{array}{ccc}\hfill E{({\widehat{\xi }}_{j_1{\tau }_1}(x))}^p& & \lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}+E[|{\widehat{r}}_{j_1{\tau }_1}(x)-P_{j_1{\tau }_1}r(x){|-{\nu }_{j_1{\tau }_1}]}_{+}^p\hfill \\ & & +{\displaystyle \underset{|j^{*}{\tau }^{*}|>|j_1{\tau }_1|}{max}}E[|{\widehat{r}}_{j^{*}{\tau }^{*}}(x)-P_{j^{*}{\tau }^{*}}r(x){|-{\nu }_{j^{*}{\tau }^{*}}]}_{+}^p\hfill \\ & & \lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}+{\displaystyle \sum _{\lfloor j{\tau }_1\rfloor =0}^{{log}_2\frac{n}{lnn}}}\cdots {\displaystyle \sum _{\lfloor j{\tau }_d\rfloor =0}^{{log}_2\frac{n}{lnn}}}E[|{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x){|-{\nu }_{j\tau }]}_{+}^p.\hfill \end{array}$$

For $t>0$, $\{(|{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x)|-{\nu }_{j\tau }{)}_{+}\ge t\}=\{|{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x)|-{\nu }_{j\tau }\ge t\}$. Then, using Lemma 4,

$$\begin{array}{ccc}\hfill E[|{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x){|-{\nu }_{j\tau }]}_{+}^p& & =p{\int }_0^{\infty }{t}^{p-1}\mathbb{P}(|{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x)|\ge {\nu }_{j\tau }+t)dt\hfill \\ & & =p{\nu }_{j\tau }^p{\int }_0^{\infty }{t}^{p-1}\mathbb{P}(|{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x)|\ge (1+t){\nu }_{j\tau })dt.\hfill \end{array}$$

According to the definition of ${\widehat{r}}_{j\tau }(x)$,

$$\begin{array}{ccc}\hfill |{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x)|& & \lesssim |\frac{1}{n}{\displaystyle \sum _{i=1}^n}\{{\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j\tau }}}[\frac{\mu \rho (Y_i)}{\omega (X_i,Y_i)h(X_i)}{\varphi }_{j\tau ,k}(X_i)-{\alpha }_{j\tau ,k}]{\varphi }_{j\tau ,k}(x)\}|\hfill \\ & & +|\frac{1}{n}{\displaystyle \sum _{i=1}^n}\{{\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j\tau }}}[\frac{1}{\omega (X_i,Y_i)}-\frac{1}{\mu }]{\varphi }_{j\tau ,k}(x)\}|\hfill \\ & & :=|\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\eta }_i|+|\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\zeta }_i|.\hfill \end{array}$$

Furthermore, the following probability inequality is true:

$$\begin{array}{ccc}\hfill \mathbb{P}(|{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x)|\ge (1+t){\nu }_{j\tau })& \le & \mathbb{P}(|\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\eta }_i|\ge \frac{1}{2}(1+t){\nu }_{j\tau })\hfill \\ & +& \mathbb{P}(|\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\zeta }_i|\ge \frac{1}{2}(1+t){\nu }_{j\tau }).\hfill \end{array}$$

(18)

Note that ${\eta }_1,\cdots ,{\eta }_d$ is independent and identically distributed and $E{\eta }_i=0$. The assumptions Hypotheses 1–3 and the property of ${\varphi }_{j\tau ,k}(x)$ imply $|{\eta }_i|\le C_1{2}^{|j\tau |}(C_1>0)$. Using the Hölder inequality,

$$\begin{array}{c}\hfill E|{\eta }_i{|}^2\le E[{\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j\tau }}}|\frac{\mu \rho (Y_i)}{\omega (X_i,Y_i)h(X_i)}{\varphi }_{j\tau ,k}(X_i)-{\alpha }_{j\tau ,k}{|}^2|{\varphi }_{j\tau ;k}(x)|][{\displaystyle \sum _{k\in {\mathsf{\Omega }}_{j\tau }}}|{\varphi }_{j\tau ,k}(x)|].\end{array}$$

According to the argument of (6), one has

$$\begin{array}{ccc}& & E|\frac{\mu \rho (Y_i)}{\omega (X_i,Y_i)h(X_i)}{\varphi }_{j\tau ,k}(X_i)-{\alpha }_{j\tau ,k}{|}^2\lesssim E{|\frac{\mu \rho (Y_i)}{\omega (X_i,Y_i)h(X_i)}{\varphi }_{j\tau ,k}(X_i)|}^2\hfill \\ & & ={\int }_{{[0,1]}^d\times \mathbb{R}}{|\frac{\mu \rho (y)}{\omega (x,y)h(x)}{\varphi }_{j\tau ,k}(x)|}^{2}f(x,y)dxdy.\hfill \end{array}$$

This with Hypotheses 1–3 and (2) shows that

$$E|\frac{\mu \rho (Y_i)}{\omega (X_i,Y_i)h(X_i)}{\varphi }_{j\tau ,k}(X_i)-{\alpha }_{j\tau ,k}{|}^2\lesssim 1.$$

Hence, there exists a constant $C_2>0$ such that

$$\begin{array}{c}\hfill E|{\eta }_i{|}^2\le C_2{2}^{|j\tau |}.\end{array}$$

According to ${\nu }_{j\tau }\lesssim 1$ and Bernstein’s inequality,

$$\begin{array}{ccc}\hfill \mathbb{P}(|\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\eta }_i|\ge \frac{1}{2}(1+t){\nu }_{j\tau })& & \le 2exp\{-\frac{n{(\frac{1}{2}(1+t){\nu }_{j\tau })}^2}{2(C_2{2}^{|j\tau |}+\frac{(1+t)C_1{2}^{|j\tau |}{\nu }_{j\tau }}{6})}\}\hfill \\ & & \le exp\{-\frac{n(1+t){\nu }_{j\tau }^2}{C_3{2}^{|j\tau |})}\}(C_3>0).\hfill \end{array}$$

(19)

Similar to the arguments of (19), one can obtain that

$$\begin{array}{c}\hfill \mathbb{P}(|\frac{1}{n}{\displaystyle \sum _{i=1}^n}{\zeta }_i|\ge \frac{1}{2}(1+t){\nu }_{j\tau })\le exp\{-\frac{n(1+t){\nu }_{j\tau }^2}{C_4{2}^{|j\tau |})}\}(C_4>0).\end{array}$$

(20)

Choosing $\lambda \ge max\{\sqrt{C_3},\sqrt{C_4}\}$, it follows from (11), (19) and (20) that the (18) reduces to

$$\begin{array}{ccc}\hfill \mathbb{P}(|{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x)|\ge (1+t){\nu }_{j\tau })& & \lesssim exp\{-(1+t)(1+\frac{p}{2})max(1,(ln2)|j\tau |)\}\hfill \\ & & \lesssim e^{-t}{2}^{-(1+\frac{p}{2})|j\tau |}.\hfill \end{array}$$

Furthermore, by the definition of ${\nu }_{j\tau }$,

$$\begin{array}{c}\hfill E[|{\widehat{r}}_{j\tau }(x)-P_{j\tau }r(x){|-{\nu }_{j\tau }]}_{+}^p\le p{\nu }_{j\tau }^p{\int }_0^{\infty }{t}^{p-1}{e}^{-t}{2}^{-(1+\frac{p}{2})|j\tau |}dt\lesssim 2^{-|j\tau |}{(\frac{lnn}{n})}^{\frac{p}{2}}.\end{array}$$

Finally, one obtains

$$\begin{array}{ccc}\hfill E{({\widehat{\xi }}_{j_1{\tau }_1}(x))}^p& & \lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}+{\displaystyle \sum _{\lfloor j{\tau }_1\rfloor =0}^{{log}_2\frac{n}{lnn}}}···{\displaystyle \sum _{\lfloor j{\tau }_d\rfloor =0}^{{log}_2\frac{n}{lnn}}}{2}^{-|j\tau |}{(\frac{lnn}{n})}^{\frac{p}{2}}\hfill \\ & & \lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}+{(\frac{lnn}{n})}^{\frac{p}{2}}\hfill \\ & & \lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}.\hfill \end{array}$$

(21)

Combining (14)–(17) and (21),

$$E|{\widehat{r}}_{j_0{\tau }_0}{(x)-r(x)|}^p\lesssim {(\frac{n}{lnn})}^{-\frac{p(s(d)-d/\tilde{p})}{2(s(d)-d/\tilde{p})+d}}.$$

□

5. Conclusions

In this paper, we consider nonparametric wavelet estimations of anisotropic regression function. Firstly, a linear wavelet estimator is proposed by wavelet projection operator. The rate of convergence over pointwise error of this wavelet estimator is proved under some mild conditions. Secondly, we construct a data-driven regression estimator with scaling parametric selection method. It should be point out that the definition of this data-driven only depends on the observed data. In other word, this data-driven estimator is an adaptive estimator. Finally, the convergence rate of the data-driven estimator is discussed by the selection rule. Compared with the conventional nonparametric regression estimations, those two regression estimators all can attain the optimal convergence rate. In addition, the data-driven estimator will be widely used in big data processing.