Pointwise Sharp Moderate Deviations for a Kernel Density Estimator

Abstract

Let $f_n$ be the non-parametric kernel density estimator based on a kernel function K and a sequence of independent and identically distributed random vectors taking values in ${\mathbb{R}}^d$. With some mild conditions, we establish sharp moderate deviations for the kernel density estimator. This means that we provide an equivalent for the tail probabilities of this estimator.

1. Introduction

Let $\{X_i;i⩾1\}$ be a sequence of independent and identically distributed (i.i.d.) random vectors taking values in ${\mathbb{R}}^d$ on probability space $(\Omega ,\mathcal{F},\mathbf{P})$ with density function f. Let $K:{\mathbb{R}}^d\mapsto \mathbb{R}$ be a kernel function. The kernel density estimator of f is defined by

$$f_n\left(x\right)={\displaystyle \frac{1}{n{a}_n^d}}\sum _{i=1}^{n}K\left({\displaystyle \frac{x-X_i}{a_n}}\right),x={(x_1,x_2,...,x_d)}^T\in {\mathbb{R}}^d,$$

where $\{a_n,n⩾1\}$ is a bandwidth sequence, that is, a sequence of positive numbers satisfying

$$a_n\to 0,n{a}_n^d\to +\infty asn\to +\infty .$$

A great and synthetic reference for such estimates is [1]. Among the huge number of applications of kernel density estimation, let us cite the elegant paper in [2] which makes use of this estimator for an important problem related with green algae: using our results may be used to derive a decision rule for this important ecological question.

In this paper, we are interested in the pointwise sharp moderate deviations for $\{f_n,n⩾1\}$ by the empirical process approach; the volume in [3] is a perfect overview for such questions. In order to present our main result, let us first introduce some notations and assumptions. Let $g:{\mathbb{R}}^d\mapsto \mathbb{R}$ be a real function. As usual, denote by

$${∥g∥}_p={\left({\int }_{x\in {\mathbb{R}}^d}{\left|g\left(x\right)\right|}^{p}dx\right)}^{{\displaystyle \frac{1}{p}}},1\le p<\infty \mathrm{and}{∥g∥}_{\infty }=\underset{x\in {\mathbb{R}}^d}{sup}\left|g\left(x\right)\right|$$

the $L_p$-norm of g and the supermum norm, respectively.

The consistency for the kernel density estimator has been studied widely. Let f be continuously differentiable on $\mathbb{R}$ such that ${\parallel f\parallel }_{\infty }<\infty$ and $\parallel f^{\prime }{\parallel }_{\infty }<\infty$ ($f^{\prime }$ is the derivative of f). In addition, suppose that ${lim}_{N\to \infty }n{a}_n=\infty$ and ${lim}_{N\to \infty }n{a}_n^2=c\ge 0$, where c is a constant. With some mild conditions, Joutard [4] proved the following pointwise sharp large deviation: for any $\alpha >0$ and $n\to \infty$,

$$\mathbf{P}\left(f_n\left(x\right)-f\left(x\right)>\alpha \right)={\displaystyle \frac{exp\{-n{a}_n{\Lambda }^{*}\left(\alpha \right)+cH\left(\tau \right)\}}{\tau {\left(2\pi n{a}_{n}f\left(x\right)I^{″}\left(\tau \right)\right)}^{\frac{1}{2}}}}\left(1+O\left(1\right)\right),$$

where ${\Lambda }^{*}\left(\alpha \right)=\tau (\alpha +f\left(x\right))-f\left(x\right)I\left(\tau \right),\tau \in [0,\alpha ],$ is such that $\alpha +f\left(x\right)=f\left(x\right)I^{\prime }\left(\tau \right)$ and $H\left(\tau \right)=-(f^2\left(x\right)I^2\left(\tau \right)/2+f^{\prime }\left(x\right)J\left(\tau \right))$, with $J\left(t\right)={\int }_{\mathbb{R}}zexp\left\{tK\left(z\right)\right\}dz$. For uniform consistency, with some mild conditions, Gao [5] proved the following moderate deviation principle (MDP) result. Let $\{b_n,n\ge 1\}$ be a sequence of positive real numbers satisfying

$${\displaystyle \frac{n{a}_n^d}{b_n}}\to +\infty ,{\displaystyle \frac{n{a}_n^{d}log{a}_n^{-1}}{b_n^2}}\to +\infty asn\to +\infty .$$

Gao [5] proved that for any $\lambda >0$,

$$\underset{n\to \infty }{lim}{\displaystyle \frac{n{a}_n^d}{b_n}}ln\mathbf{P}\left({\displaystyle \frac{n{a}_n^d}{b_n}}{\parallel f_n-\mathbf{E}{f}_n\parallel }_{\infty }>\lambda \right)=-I\left(\lambda \right),$$

where

$$I\left(\lambda \right)={\displaystyle \frac{{\lambda }^2}{{2\parallel f\parallel }_{\infty }{\parallel K\parallel }_2^2}}.$$

A pointwise MDP is also established in Gao [5]. A class of refinements of pointwise MDP is called sharp moderate deviations. Sharp moderate deviations are also known as Cramér moderate deviations, and have attracted a lot of interest. We refer to Cramér [6], Petrov [7], Beknazaryan et al. [8] and Fan et al. [9] for such type results. In this paper, we are interested in establishing sharp moderate deviations for the kernel density estimator.

The paper is organized as follows. Our main result is stated and discussed in Section 2. The proof of our theorem is given in Section 3.

2. Main Results

The following assumptions will be used in this paper.

(A)

Assume that the kernel function K satisfies

$$\begin{array}{ccc}& & {\int }_{x\in {\mathbb{R}}^d}K\left(x\right)dx=1\mathrm{and}{∥K∥}_{\infty }<+\infty .\hfill \end{array}$$

(B)

There exist a constant $\beta \in (0,1]$ and a non-negative integer s such that for any $x,y,z\in {\mathbb{R}}^d,$

$$\left|{(z·\nabla )}^{s}f\left(x\right)-{(z·\nabla )}^{s}f\left(y\right)\right|⩽{A\parallel x-y\parallel }_2^{\beta }{\left|\right|z\left|\right|}_2^s,$$

where A is a positive constant, ${\parallel ·\parallel }_2$ is the Euclidean distance and

$$z·\nabla =z_1{\displaystyle \frac{\partial }{\partial x_1}}+z_2{\displaystyle \frac{\partial }{\partial x_2}}+\cdots +z_d{\displaystyle \frac{\partial }{\partial x_d}}.$$

(C)

Assume

$${\int }_{x\in {\mathbb{R}}^d}{K}^2\left(x\right){∥x∥}_2^{s+\beta }dx<+\infty \mathrm{and}{\int }_{x\in {\mathbb{R}}^d}K\left(x\right)\prod _{i=1}^d{x}_i^{j_i}dx=0,0<\sum _{i=1}^d{j}_i\le s.$$

Remark 1.

After [1], we recall that the previous assumptions are pretty standard and we restate them in the current multivariate setting:

1.

The first condition in Assumption (A) is necessary to ensure that the estimate remains a function with integral 1;

the second one is not necessary but no striking or useful unbounded kernel was used in the frame of density estimation. Moreover, this condition makes it useless to assume that ${\int }_{x\in {\mathbb{R}}^d}{K}^2\left(x\right)dx<\infty$, for example.

2.

Assumption (B) is a regularity condition on f with order $s+\beta$, with $\beta >0$ and $s\in \mathbb{N}$ as considered before.

3.

The first condition in Assumption (C) is useful to prove that the involved expressions are square-integrable. The second part of this condition is more tricky and ensures that the Taylor expansion up to order s provides the relation

$${\int }_{z\in {\mathbb{R}}^d}K\left(z\right)f(x+z)dz=f\left(x\right)+{\mathcal{O}(\parallel x\parallel }^{s+\beta }),$$

(1)

since all the intermediate terms simply vanish. It is important to also quote that such kernels K exist. A very simple and usual case is $s=\beta =1$ (second-order regularity), which holds in case K is symmetric with respect to each of its coordinates; it this case, it is possible to obtain $K\left(z\right)\ge 0$ and then the estimator $f_n$ is still a density (since it is non-negative). For the general case $s\in \mathbb{N}$, a standard procedure to prove the existence of such kernels is to define $K\left(z\right)=P\left(z\right)\delta \left(z\right)$ for a fixed bounded density function and $d^{\circ }P=s$, where $d^{\circ }P$ is the degree of the polynomial P. Then, it is easy to prove that the system of equations in (C) together with the first part of (A) is invertible and linear because the matrix with coefficients

$$a_{j,k}={\int }_{z\in {\mathbb{R}}^d}K\left(x\right)\prod _{i=1}^d{x}_i^{j_i+k_i}dx,0<\sum _{i=1}^d{j}_i\le s,0<\sum _{i=1}^d{k}_i\le s,$$

is symmetric non-negative definite; this point is a straightforward extension of Lemma 3.3.1 in [10] to our multidimensional setting.

Assume that $f\left(x\right)>0$ for some $x\in {\mathbb{R}}^d.$ Denote

$$D_n\left(x\right)={\displaystyle \frac{\sqrt{n{a}_n^d}\left(f_n\left(x\right)-\mathbf{E}{f}_n\left(x\right)\right)}{\sqrt{{f\left(x\right)\parallel K\parallel }_2^2+{\sum }_{t=1}^s{a}_n^t\frac{1}{t!}{\int }_{z\in {\mathbb{R}}^d}{(z·\nabla )}^{t}f\left(x\right)K^2\left(z\right)dz}}}.$$

We have the following pointwise sharp moderate deviations for the kernel density estimator.

Theorem 1.

Assume that Conditions (A)–(C) are satisfied. Assume $f\left(x\right)>0$ for some $x\in {\mathbb{R}}^d.$ Then, it holds that

$$\begin{array}{c}\hfill ln{\displaystyle \frac{\mathbf{P}\left(D_n\left(x\right)\ge t\right)}{1-\mathrm{\Phi }\left(t\right)}}=O\left({\displaystyle \frac{1+t^3}{\sqrt{n{a}_n^d}}}+t^2{a}_n^{(s+\beta )\wedge d}\right)\end{array}$$

(2)

uniformly for $0\le t=o\left(\sqrt{n{a}_n^d}\right)$ as $n\to \infty .$ Moreover, the same equality remains valid when ${\displaystyle ln\frac{\mathbf{P}(D_n\left(x\right)\ge t)}{1-\mathrm{\Phi }\left(t\right)}}$ is replaced by ${\displaystyle ln\frac{\mathbf{P}(D_n\left(x\right)\le -t)}{1-\mathrm{\Phi }\left(t\right)}}$.

For the non-centered case, we have the following pointwise sharp moderate deviations for the kernel density estimator. Denote

$${\widehat{D}}_n\left(x\right)={\displaystyle \frac{\sqrt{n{a}_n^d}}{{\parallel K\parallel }_2\sqrt{f\left(x\right)}}}\left(f_n\left(x\right)-f\left(x\right)-\sum _{t=1}^s{\displaystyle \frac{1}{t!}}{\int }_{x\in {\mathbb{R}}^d}K\left(z\right){(z·\nabla )}^{t}f\left(x\right)dz{a}_n^t\right).$$

Theorem 2.

Assume that conditions (A)–(C) are satisfied. Assume $f\left(x\right)>0$ for some $x\in {\mathbb{R}}^d$. Then, it holds that

$$\begin{array}{c}\hfill ln{\displaystyle \frac{\mathbf{P}\left({\widehat{D}}_n\left(x\right)\ge t\right)}{1-\mathrm{\Phi }\left(t\right)}}=O\left({\displaystyle \frac{1+t^3}{\sqrt{n{a}_n^d}}}+t^2{a}_n^{(s+\beta )\wedge d}+(1+t)\sqrt{n}{a}_n^{s+\beta +d/2}\right)\end{array}$$

(3)

uniformly for $0\le t=o\left(\sqrt{n{a}_n^d}\right)$ as $n\to \infty .$ Moreover, the same equality remains valid when ${\displaystyle ln\frac{\mathbf{P}({\widehat{D}}_n\left(x\right)\ge t)}{1-\mathrm{\Phi }\left(t\right)}}$ is replaced by ${\displaystyle ln\frac{\mathbf{P}({\widehat{D}}_n\left(x\right)\le -t)}{1-\mathrm{\Phi }\left(t\right)}}$.

Remark 2.

Let us comment on Theorem 2.

1.

In the expression of $D_n$, recall that (1) in Remark 1 entails that $\mathbf{E}{f}_n\left(x\right)-f\left(x\right)=O\left(a_n^{s+\beta }\right).$

2.

This result makes it possible to provide a practitioner with precise confidence intervals that are easy to compute in the case of hypothesis testing. Explicit asymptotic p-values can thus be straightforwardly obtained. For instance, consider the following hypothesis testing:

$$H_0:f\left(x_0\right)=t_0\mathit{versus}{H}_1:f\left(x_0\right)\ne t_0,$$

with $t_0>0.$ Denote

$$\begin{array}{c}\hfill z_0={\displaystyle \frac{\sqrt{n{a}_n^d}}{{\parallel K\parallel }_2\sqrt{f\left(x_0\right)}}}\left(f_n\left(x_0\right)-t_0\right).\end{array}$$

Then, by Theorem 2, the p-value is asymptotically equal to $2(1-\mathrm{\Phi }\left(z_0\right))$, provided that $z_0$ satisfies

$${\displaystyle \frac{1+z_0^3}{\sqrt{n{a}_n^d}}}+z_0^2{a}_n^{(s+\beta )\wedge d}+(1+z_0)\sqrt{n}{a}_n^{s+\beta +d/2}\to 0.$$

as $n\to \infty$.

3.

Cases of other non parametric estimators, such as the Nadaraya–Watson kernel regression estimator (cf. El Machkouri et al. [11] for instance), non-linear regression estimates or conditional expectations, for predictions issues or estimates of derivatives or even quantile regression estimators, see Rosenblatt [1], will be derived in further subsequent papers.

4.

Even if a non-independent version of this result is accessible, we prefer to give a simple result in the current i.i.d. case.

By Theorem 2, we have the following Berry–Esseen bound for ${\widehat{D}}_n\left(x\right),$ that is,

$$\begin{array}{c}\hfill \underset{t\in \mathbb{R}}{sup}|\mathbf{P}\left({\widehat{D}}_n\left(x\right)\le t\right)-\mathrm{\Phi }\left(t\right)|=O\left({\displaystyle \frac{1}{\sqrt{n{a}_n^d}}}+a_n^{(s+\beta )\wedge d}+\sqrt{n}{a}_n^{s+\beta +d/2}\right).\end{array}$$

(4)

In particular, by taking $a_n=n^{-1/(s+\beta +d)}$, we obtain

$$\begin{array}{c}\hfill \underset{t\in \mathbb{R}}{sup}|\mathbf{P}\left({\widehat{D}}_n\left(x\right)\le t\right)-\mathrm{\Phi }\left(t\right)|=O\left(n^{-(s+\beta )/(2s+2\beta +d)}\right).\end{array}$$

Moreover, if $s=0$ and $\beta =d=1$, i.e., f is 1-Hölder-continuous, then it holds that

$$\begin{array}{c}\hfill \underset{t\in \mathbb{R}}{sup}|\mathbf{P}\left({\displaystyle \frac{n^{\frac{1}{4}}}{{\parallel K\parallel }_2\sqrt{f\left(x\right)}}}\left(f_n\left(x\right)-f\left(x\right)\right)\le t\right)-\mathrm{\Phi }\left(t\right)|=O\left(n^{-1/3}\right).\end{array}$$

Conclusions. When $f\in C^1\left({\mathbb{R}}^d\right)$ and $K\left(z\right)$ is symmetric with respect to 0, which implies that ${\int }_{z\in {\mathbb{R}}^d}{z}_i{K}^2\left(z\right)dz=0$ for all $1\le i\le d,$ by taking $s=1$ in Assumptions (C), then we have

$${\int }_{x\in {\mathbb{R}}^d}K\left(z\right)z·\nabla f\left(x\right)dz=0,$$

which implies

$$D_n\left(x\right)={\displaystyle \frac{\sqrt{n{a}_n^d}}{{\parallel K\parallel }_2\sqrt{f\left(x\right)}}}\left(f_n\left(x\right)-\mathbf{E}{f}_n\left(x\right)\right)\mathrm{and}{\widehat{D}}_n\left(x\right)={\displaystyle \frac{\sqrt{n{a}_n^d}}{{\parallel K\parallel }_2\sqrt{f\left(x\right)}}}\left(f_n\left(x\right)-f\left(x\right)\right).$$

Then, Theorems 1 and 2 hold with $s=1.$ Theorems 1 and 2 provide moderate deviations for the expressions $D_n$ and ${\widehat{D}}_n$, which are related through the expression of $f_n$’s bias; see Remark 2. Remarks 1 and 2 provide a detailed description of the calculation of the bias essential here.

3. Proof of Theorem 1

For $n\ge 1,$ let $\{Y_i,1\le i\le n\}$ be i.i.d. and centered random variables. Denote ${\sigma }^2=\mathbf{E}{Y}_1^2$ and $T_n={\sum }_{i=1}^n{Y}_i.$ Assume that $\sigma >0.$ Fan et al. [12] (see also Cramér [6]) established the following asymptotic expansion on the tail probabilities of moderate deviations for $T_n$.

Lemma 1.

Assume that there exists a constant ${\alpha }_n$ such that for all $1\le i\le n,$

$$\mathbf{E}|Y_i{|}^k\le {\displaystyle \frac{1}{2}}k!{\left({\displaystyle \frac{1}{{\alpha }_n}}\right)}^{k-2}\mathbf{E}{Y}_i^2,k\ge 2.$$

(5)

Then,

$$ln{\displaystyle \frac{\mathbf{P}(T_n\ge t\sigma \sqrt{n})}{1-\mathrm{\Phi }\left(t\right)}}=O\left({\displaystyle \frac{1+t^3}{\sqrt{n}{\alpha }_n}}\right)asn\to \infty$$

(6)

holds uniformly for $0\le t=o\left(\sqrt{n}{\alpha }_n\right).$

Proof. 

Lemma 1 is a simple consequence of Fan et al. [12]. □

With the preliminary lemma above, we are in the position to begin the proof of Theorem 1. It is easy to see that

$$\begin{array}{c}\hfill \sqrt{n{a}_n^d}\left(f_n\left(x\right)-\mathbf{E}{f}_n\left(x\right)\right)={\displaystyle \frac{1}{\sqrt{n{a}_n^d}}}\sum _{k=1}^n\left[K\left({\displaystyle \frac{x-X_i}{a_n}}\right)-\mathbf{E}K\left({\displaystyle \frac{x-X_i}{a_n}}\right)\right].\end{array}$$

(7)

In the sequel, we give an estimate for the right-hand side of the last equality. Notice that

$$\begin{array}{ccc}& & {\displaystyle \frac{1}{\sqrt{n{a}_n^d}}}\sum _{k=1}^n\left[K\left({\displaystyle \frac{x-X_i}{a_n}}\right)-\mathbf{E}K\left({\displaystyle \frac{x-X_i}{a_n}}\right)\right]\hfill \\ & & ={\displaystyle \frac{1}{\sqrt{n}}}\sum _{k=1}^n{\displaystyle \frac{1}{\sqrt{a_n^d}}}\left[K\left({\displaystyle \frac{x-X_i}{a_n}}\right)-\mathbf{E}K\left({\displaystyle \frac{x-X_i}{a_n}}\right)\right].\hfill \end{array}$$

(8)

Denote

$$Y_i={\displaystyle \frac{1}{\sqrt{a_n^d}}}\left[K\left({\displaystyle \frac{x-X_i}{a_n}}\right)-\mathbf{E}K\left({\displaystyle \frac{x-X_i}{a_n}}\right)\right],1\le i\le n.$$

We can prove that $Y_i$ satisfies the Bernstein condition Equation (5). Indeed, we can deduce that for all $k\ge 2,$

$$\begin{array}{cc}\hfill \mathbf{E}{\left|Y_i\right|}^k& ⩽{\left({\displaystyle \frac{1}{\sqrt{a_n^d}}}\right)}^k\mathbf{E}{\left|K\left({\displaystyle \frac{x-X_i}{a_n}}\right)-\mathbf{E}K\left({\displaystyle \frac{x-X_i}{a_n}}\right)\right|}^k\hfill \\ \hfill & ⩽{\left({\displaystyle \frac{1}{\sqrt{a_n^d}}}\right)}^k{∥K∥}_{\infty }^{k-2}\mathbf{E}{\left|K\left({\displaystyle \frac{x-X_i}{a_n}}\right)-\mathbf{E}K\left({\displaystyle \frac{x-X_i}{a_n}}\right)\right|}^2\hfill \\ \hfill & ⩽{\left({\displaystyle \frac{{∥K∥}_{\infty }}{\sqrt{a_n^d}}}\right)}^{k-2}\mathbf{E}{\left|Y_i\right|}^2\hfill \\ \hfill & ⩽{\displaystyle \frac{1}{2}}k!{\left({\displaystyle \frac{{∥K∥}_{\infty }}{\sqrt{a_n^d}}}\right)}^{k-2}\mathbf{E}{\left|Y_i\right|}^2.\hfill \end{array}$$

For the variance of $Y_i$, we have the following estimation:

$$\begin{array}{cc}\hfill \mathrm{Var}\left(Y_i\right)& ={\displaystyle \frac{1}{a_n^d}}\mathrm{Var}\left(K\left({\displaystyle \frac{x-X_i}{a_n}}\right)\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{a_n^d}}\left(\mathbf{E}{K}^2\left({\displaystyle \frac{x-X_i}{a_n}}\right)-{\left[\mathbf{E}K\left({\displaystyle \frac{x-X_i}{a_n}}\right)\right]}^2\right).\hfill \end{array}$$

It is easy to see that

$$\begin{array}{cc}\hfill \mathbf{E}{K}^2\left({\displaystyle \frac{x-X_i}{a_n}}\right)& =a_n^d\left[{\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right)f(x-z{a}_n)dz\right]\hfill \\ \hfill & =a_n^d\left[{\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right)f\left(x\right)dz+{\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right)(f(x-z{a}_n)-f\left(x\right))dz\right].\hfill \end{array}$$

(9)

By Assumption (B), it is easy to see that

$$\begin{array}{ccc}& & |f(x-z{a}_n)-f\left(x\right)-\sum _{t=1}^s{\displaystyle \frac{1}{t!}}{(z·\nabla )}^{t}f\left(x\right)a_n^t|\hfill \\ & & =\left|{\displaystyle \frac{1}{s!}}\left({(z·\nabla )}^{s}f(x+\theta z{a}_n)-{(z·\nabla )}^{s}f\left(x\right)\right)a_n^s\right|\hfill \\ & & \le C_{d}A{a}_n^{s+\beta }{\parallel z\parallel }_2^{s+\beta },\hfill \end{array}$$

(10)

where $\left|\theta \right|\le 1$ and A is given by Assumption (B). Again by Condition (C), we can deduce that

$$\begin{array}{ccc}\hfill \mathbf{E}{K}^2\left({\displaystyle \frac{x-X_i}{a_n}}\right)& =& a_n^d\left[{\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right)f\left(x\right)dz+{\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right)(f(x-z{a}_n)-f\left(x\right))dz\right]\hfill \\ & =& a_n^{d}f\left(x\right){\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right)dz+\sum _{t=1}^s{\displaystyle \frac{1}{t!}}{\int }_{z\in {\mathbb{R}}^d}{(z·\nabla )}^{t}f\left(x\right)K^2\left(z\right)dz{a}_n^{d+t}\hfill \\ & & +O\left(1\right)a_n^{d+s+\beta }{\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right){\parallel z\parallel }_2^{s+\beta }dz\hfill \\ & =& a_n^{d}f\left(x\right){\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right)dz+\sum _{t=1}^s{a}_n^{d+t}{\displaystyle \frac{1}{t!}}{\int }_{z\in {\mathbb{R}}^d}{(z·\nabla )}^{t}f\left(x\right)K^2\left(z\right)dz\hfill \\ & & +O\left(a_n^{d+s+\beta }\right).\hfill \end{array}$$

(11)

By Condition (B), it is easy to see that

$$\begin{array}{ccc}\hfill \mathbf{E}K\left({\displaystyle \frac{x-X_i}{a_n}}\right)& =& a_n^d\left[{\int }_{z\in {\mathbb{R}}^d}K\left(z\right)f\left(x\right)dz+{\int }_{z\in {\mathbb{R}}^d}K\left(z\right)[f(x-z{a}_n)-f\left(x\right)]dz\right]\hfill \\ & =& a_n^{d}f\left(x\right){\int }_{z\in {\mathbb{R}}^d}K\left(z\right)dz+o\left(a_n^d\right)\hfill \\ & =& a_n^{d}f\left(x\right)+o\left(a_n^d\right).\hfill \end{array}$$

(12)

From Equations (10) and (11), we have

$$\begin{array}{ccc}\hfill \mathrm{Var}\left(Y_i\right)& =& {\displaystyle \frac{1}{a_n^d}}(a_n^{d}f\left(x\right){\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right)dz+\sum _{t=1}^s{a}_n^{d+t}{\displaystyle \frac{1}{t!}}{\int }_{z\in {\mathbb{R}}^d}{(z·\nabla )}^{t}f\left(x\right)K^2\left(z\right)dz\hfill \\ & & +O\left(a_n^{d+s+\beta }\right)-{\left(a_n^{d}f\left(x\right)+o\left(a_n^d\right)\right)}^2)\hfill \\ & =& f\left(x\right){\int }_{z\in {\mathbb{R}}^d}{K}^2\left(z\right)dz+\sum _{t=1}^s{a}_n^t{\displaystyle \frac{1}{t!}}{\int }_{z\in {\mathbb{R}}^d}{(z·\nabla )}^{t}f\left(x\right)K^2\left(z\right)dz+O(a_n^{(s+\beta )\wedge d}).\hfill \end{array}$$

When $f\left(x\right)>0,$ we obtain

$${\gamma }_n:={\displaystyle \frac{{∥K∥}_{\infty }/\sqrt{a_n^d}}{\sqrt{n\mathrm{Var}\left(Y_1\right)}}}=O\left({\displaystyle \frac{1}{\sqrt{n{a}_n^d}}}\right)$$

and

$$\begin{array}{c}\hfill {\displaystyle \frac{\mathrm{Var}\left(Y_i\right)}{{\displaystyle {f\left(x\right)\parallel K\parallel }_2^2+\sum _{t=1}^s{a}_n^t\frac{1}{t!}{\int }_{z\in {\mathbb{R}}^d}{(z·\nabla )}^{t}f\left(x\right)K^2\left(z\right)dz}}}=1+O\left(a_n^{(s+\beta )\wedge d}\right).\end{array}$$

(13)

Therefore, by Lemma 1, we can deduce that for all $0\le t=o\left(\sqrt{n{a}_n^d}\right)$

$$\begin{array}{cc}\hfill \mathbf{P}\left(\sqrt{n{a}_n^d}\left(f_n\left(x\right)-\mathbf{E}{f}_n\left(x\right)\right)⩾t\sqrt{\mathrm{Var}\left(Y_1\right)}\right)& =\mathbf{P}\left({\displaystyle \frac{1}{\sqrt{n\mathrm{Var}\left(Y_1\right)}}}\sum _{k=1}^n{Y}_i⩾t\right)\hfill \\ \hfill & =\left(1-\mathrm{\Phi }\left(t\right)\right)exp\left\{O\left(1\right){\displaystyle \frac{1+t^3}{\sqrt{n{a}_n^d}}}\right\}.\hfill \end{array}$$

Applying inequality Equation (12) to the last inequality, we deduce that for all $0\le t=o\left(\sqrt{n{a}_n^d}\right)$

$$\begin{array}{ccc}& & \mathbf{P}\left(D_n\left(x\right)\ge t\right)\hfill \\ & & =\mathbf{P}\left(\sqrt{n{a}_n^d}\left(f_n\left(x\right)-\mathbf{E}{f}_n\left(x\right)\right)⩾t\sqrt{{f\left(x\right)\parallel K\parallel }_2^2+\sum _{t=1}^s{a}_n^t{\displaystyle \frac{1}{t!}}{\int }_{z\in {\mathbb{R}}^d}{(z·\nabla )}^{t}f\left(x\right)K^2\left(z\right)dz}\right)\hfill \\ & & =\mathbf{P}\left(\sqrt{n{a}_n^d}\left(f_n\left(x\right)-\mathbf{E}{f}_n\left(x\right)\right)⩾t\sqrt{\mathrm{Var}\left(Y_1\right)}\sqrt{1+O\left(a_n^{(s+\beta )\wedge d}\right)}\right)\hfill \\ & & =\left(1-\mathrm{\Phi }\left(t(1+O\left(a_n^{(s+\beta )\wedge d}\right))\right)\right)exp\left\{O\left(1\right){\displaystyle \frac{1+t^3}{\sqrt{n{a}_n^d}}}\right\}.\hfill \end{array}$$

Because of

$${\displaystyle \frac{1}{1+\lambda }}{e}^{-{\lambda }^2/2}\le \sqrt{2\pi }\left(1-\mathrm{\Phi }\left(\lambda \right)\right),\lambda \ge 0,$$

it is easy to see that for all $0\le \lambda \le x$,

$$\begin{array}{ccc}\hfill 1\le {\displaystyle \frac{{\displaystyle {\int }_{\lambda }^{\infty }exp\{-t^2/2\}dt}}{{\displaystyle {\int }_x^{\infty }exp\{-t^2/2\}dt}}}& \le & 1+{\displaystyle \frac{{\displaystyle {\int }_{\lambda }^{x}exp\{-t^2/2\}dt}}{{\displaystyle {\int }_x^{\infty }exp\{-t^2/2\}dt}}}\hfill \\ & \le & 1+c_{1}x(x-\lambda )exp\left\{(x^2-{\lambda }^2)/2\right\}\hfill \\ & \le & exp\left\{c_{2}x|x-\lambda |\right\}.\hfill \end{array}$$

Hence, we obtain for any $\lambda ,x\ge 0,$

$$1-\mathrm{\Phi }\left(\lambda \right)=\left(1-\mathrm{\Phi }\left(x\right)\right)exp\left\{O\left(1\right)(x+\lambda )|x-\lambda |\right\}.$$

By the last equality, it follows that

$$\begin{array}{ccc}\hfill 1-\mathrm{\Phi }\left(x(1+O\left(a_n^{(s+\beta )\wedge d}\right))\right)& =& \left(1-\mathrm{\Phi }\left(x\right)\right)exp\left\{O\left(1\right)x^2{a}_n^{(s+\beta )\wedge d}\right\}.\hfill \end{array}$$

Therefore, we have, for all $0\le t=o\left(\sqrt{n{a}_n^d}\right),$

$$\begin{array}{c}\hfill \mathbf{P}\left(D_n\left(x\right)\ge t\right)=\left(1-\mathrm{\Phi }\left(t\right)\right)exp\left\{O\left(1\right)\left({\displaystyle \frac{1+t^3}{\sqrt{n{a}_n^d}}}+t^2{a}_n^{(s+\beta )\wedge d}\right)\right\}.\end{array}$$

(14)

This completes the proof of Theorem 1.

4. Proof of Theorem 2

It is easy to see that

$$\begin{array}{cc}\hfill \mathbf{E}{f}_n\left(x\right)-f\left(x\right)& ={\displaystyle \frac{1}{n{a}_n^d}}\sum _{k=1}^n{\int }_{x\in {\mathbb{R}}^d}K\left({\displaystyle \frac{x-t}{a_n}}\right)f\left(t\right)dt-f\left(x\right)\hfill \\ \hfill & ={\int }_{x\in {\mathbb{R}}^d}K\left(z\right)f(x-a_{n}z)dz-f\left(x\right)\hfill \\ \hfill & ={\int }_{x\in {\mathbb{R}}^d}K\left(z\right)[f(x-a_{n}z)-f\left(x\right)]dz.\hfill \end{array}$$

By inequality Equation (9), we deduce that

$$\begin{array}{ccc}\hfill \mathbf{E}{f}_n\left(x\right)-f\left(x\right)& =& \sum _{t=1}^s{\displaystyle \frac{1}{t!}}{\int }_{x\in {\mathbb{R}}^d}K\left(z\right){(z·\nabla )}^{t}f\left(x\right)dz{a}_n^t\hfill \\ & & +C_{d}A{\int }_{x\in {\mathbb{R}}^d}K\left(z\right){\parallel z\parallel }_2^{s+\beta }dz{a}_n^{s+\beta }\hfill \\ & =& \sum _{t=1}^s{\displaystyle \frac{1}{t!}}{\int }_{x\in {\mathbb{R}}^d}K\left(z\right){(z·\nabla )}^{t}f\left(x\right)dz{a}_n^t+O\left(a_n^{s+\beta }\right).\hfill \end{array}$$

Applying the last line to Equation (13), we obtain, for all $0\le t=o\left(\sqrt{n{a}_n^d}\right),$

$$\begin{array}{ccc}\hfill \mathbf{P}\left({\widehat{D}}_n\left(x\right)\ge t\right)& =& \mathbf{P}\left(D_n\left(x\right)\ge t-{\displaystyle \frac{\sqrt{n{a}_n^d}O\left(a_n^{s+\beta }\right)}{{\parallel K\parallel }_2\sqrt{f\left(x\right)}}}\right)\hfill \\ & =& \left(1-\mathrm{\Phi }\left(t\right)\right)exp\left\{O\left(1\right)\left({\displaystyle \frac{1+t^3}{\sqrt{n{a}_n^d}}}+t^2{a}_n^{(s+\beta )\wedge d}+(1+t)\sqrt{n}{a}_n^{s+\beta +d/2}\right)\right\}.\hfill \end{array}$$

This completes the proof of Theorem 2.