The Law of the Iterated Logarithm for L_p-Norms of Kernel Estimators of Cumulative Distribution Functions

Abstract

In this paper, we consider the strong convergence of $L_p$-norms ($p\ge 1$) of a kernel estimator of a cumulative distribution function (CDF). Under some mild conditions, the law of the iterated logarithm (LIL) for the $L_p$-norms of empirical processes is extended to the kernel estimator of the CDF.

1. Introduction

Consider an independent identically distributed random sample $X_1,X_2,\cdots ,X_n$ from a population with an unknown cumulative distribution function (CDF). For the empirical distribution function $F_n$, defined as follows:

$$\begin{array}{c}\hfill F_n\left(x\right)=\frac{1}{n}{\displaystyle \sum _{i=1}^n}I(X_i\le x),\forall x\in R^1,\end{array}$$

with I denoting the indicator function, the classical Glivenko–Cantelli theorem states that $F_n\left(x\right)$ converges almost surely (a.s.) to $F\left(x\right)$ uniformly in $x\in R^1$, i.e.,

$$\begin{array}{c}\hfill {\displaystyle \underset{x\in R^1}{sup}}|F_n\left(x\right)-F\left(x\right)|\to 0,\mathrm{a}.\mathrm{s}.\end{array}$$

The extended Glivenko–Cantelli lemma (in Fabian and Hannan 1985, pp. 80–83 [1]) provides the strong uniform convergence rate as follows:

$$\begin{array}{c}\hfill {\displaystyle \underset{x\in R}{sup}}{n}^{\alpha }|F_n\left(x\right)-F\left(x\right)|\to 0a.s.,\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{y}0<\alpha <1/2.\end{array}$$

(1)

The law of the iterated logarithm (LIL) for $F_n\left(t\right)$, i.e.,

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{2log(logn)}}{\displaystyle \underset{x}{sup}}|F_n\left(x\right)-F\left(x\right)|=\frac{1}{2}a.s.\end{array}$$

(2)

was proven by Smirnov (1944) [2] and, independently, Chung (1949) [3].

Finkelstein (1971) [4] obtained the $L_2$-version of the law of iterated logarithm,

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{2log(logn)}}{\left[{\int }_{-\infty }^{\infty }{(F_n\left(x\right)-F\left(x\right))}^{2}dF\left(x\right)\right]}^{1/2}=\frac{1}{\pi }a.s.\end{array}$$

(3)

For any $p\ge 1$, setting

$$\begin{array}{c}\hfill C\left(p\right)=\frac{1}{2}{\left(\frac{p(p+2)}{\pi }\right)}^{1/2}{\left(\frac{2}{p+2}\right)}^{1/p}\frac{\Gamma (1/p+\frac{1}{2})}{\Gamma (1/p)},\end{array}$$

(4)

the law of the iterated logarithm for $L_p$-norm of $F_n\left(x\right)$,

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{2log(logn)}}{\left[{\int }_{-\infty }^{\infty }{|F_n\left(x\right)-F\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}=C\left(p\right)a.s.\end{array}$$

(5)

was developped by Gajek, Kahszka, and Lenic (1996) [5]. It is easy to verify that

$$\begin{array}{c}\hfill C\left(1\right)=\frac{\sqrt{3}}{6}\mathrm{a}\mathrm{n}\mathrm{d}C\left(2\right)=\frac{1}{\pi }.\end{array}$$

And (3) is a special case of (5) corresponding to $p=2$.

Notice that there is one serious discontinuity drawback of $F_n$, regardless of F being continuous or discrete. To treat this deficiency of $F_n$, Yamato (1973) [6] proposed the following kernel distribution estimator:

$$\widehat{F}\left(x\right)={\int }_{-\infty }^x{n}^{-1}{\sum }_{i=1}^n{k}_h\left(u-X_i\right)du,x\in \mathbb{R},$$

(6)

in which $h=h_n$ is the usual band width sequence of positive numbers tending to zero, k is a probability density function(PDF) called kernel, and $k_h\left(u\right)=k\left(u/h\right)/h$.

The aim of this paper is to provide certain conditions to guarantee the LIL of $L_p$-norm of $\widehat{F}$. Some asymptotic properties of the smooth estimator $\widehat{F}$ have been established. For example, in Yamato (1973) [6], the asymptotic normality and uniform strong consistency of $\widehat{F}$ were obtained. In more general contexts, Winter (1979) [7] considered the convergence rate of perturbed empirical distribution functions. Wang, Cheng, and Yang (2013) [8] developed simultaneous confidence bands for F based on $\widehat{F}$. The strong convergence rate of $\widehat{F}$ was considered by Cheng (2017) [9], which extended the extended Glivenko–Cantelli Lemma (1) to the kernel estimator $\widehat{F}$.

Here, we shall continue to consider the strong convergence of a smooth estimator $\widehat{F}$ for F. More specifically, we are interested in extending the LIL of $L_p$-norm in (5) for $F_n\left(t\right)$ to the kernel estimator $\widehat{F}$.

The outline of this paper is as follows: Section 2 describes the basic assumptions and main results: the strong uniform closeness between $F_n$ and $\widehat{F}$, and the LIL of $L_p$-norm of $\widehat{F}$. Detailed proofs are provided in Section 3.

Note that for the proof of the strong uniform closeness between $F_n$ and $\widehat{F}$, we use the Kiefer type approximation for the empirical process (see Csörgő and Révész (1981) [10]).

Throughout the following all limits are taken as the sample size n tending to ∞.

2. Assumptions and the Main Results

In this section, we start with the assumptions for the kernel function k.

Assumption A1.

k: Functions $k\left(x\right)$, $xk\left(x\right)$ and $x^{2}k\left(x\right)$ are integrable on the whole real line and satisfy the following properties:

$$\begin{array}{c}\hfill k\left(x\right)\ge 0,{\int }_{-\infty }^{+\infty }k\left(x\right)dx=1,{\int }_{-\infty }^{+\infty }xk\left(x\right)dx=0and{\int }_{-\infty }^{+\infty }{x}^{2}k\left(x\right)dx<\infty .\end{array}$$

About the band width h, we assume

$$\begin{array}{c}\hfill h^{3/2}log(logn)\to 0\mathrm{a}\mathrm{n}\mathrm{d}n{h}^4/log(logn)\to 0,\end{array}$$

(7)

which are stronger than the assumption $n{h}^4\to 0$ used in Cheng (2017) [9].

Under the above assumptions, we first state the result for evaluating the uniform closeness between $\widehat{F}$ and $F_n$, which improves Theorem 2.1 in Cheng (2017) [9].

Theorem 1.

Assume thatAssumption kand (7) hold. Then, for the continuous CDF F with bounded second order derivative, we have

$$\begin{array}{c}\hfill {\displaystyle \underset{x\in R}{sup}}\sqrt{\frac{n}{log(logn)}}|\widehat{F}\left(x\right)-F_n\left(x\right)|\to 0,a.s.\end{array}$$

(8)

Together with LIL in (2), the LIL can be extended to $\widehat{F}$, as follows:

Corollary 1.

Under the assumptions of Theorem 1, for the continuous CDF F with bounded second order derivative, we have

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{2log(logn)}}{\displaystyle \underset{x}{sup}}|\widehat{F}\left(x\right)-F\left(x\right)|=\frac{1}{2}a.s.\end{array}$$

(9)

Remark 1.

Using a different approach, (9) was verified in Winter (1979) [7].

Combining (8) with (5), the LIL for $L_p$-norm of $F_n$ can be extended to $\widehat{F}$.

Theorem 2.

Under the assumptions of Theorem 1, for any $p\ge 1$ and the continuous CDF F with bounded second order derivative, we have

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{2log(logn)}}{\left[{\int }_{-\infty }^{\infty }{|\widehat{F}\left(x\right)-F\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}=C\left(p\right)a.s.,\end{array}$$

(10)

where $C\left(p\right)$ is defined in (4).

Remark 2.

Applying the facts $C\left(1\right)=\frac{\sqrt{3}}{6}$ and $C\left(2\right)=\frac{1}{\pi }$, Theorem 2 can result in the following corollary:

Corollary 2.

Under the assumptions of Theorem 1, for the continuous CDF F with bounded second order derivative, we have

$$lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{2log(logn)}}{\int }_{-\infty }^{\infty }|{\widehat{F}}_n\left(x\right)-F\left(x\right)|dF\left(x\right)=\frac{\sqrt{3}}{6}a.s.$$

and

$$lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{2log(logn)}}{\left[{\int }_{-\infty }^{\infty }{|{\widehat{F}}_n\left(x\right)-F\left(x\right)|}^{2}dF\left(x\right)\right]}^{1/2}=\frac{1}{\pi }a.s.$$

Detailed proofs of the above results are given below.

3. Proof

Set

$$\begin{array}{ccc}\hfill U_n\left(x\right)& :=& \frac{1}{n}{\displaystyle \sum _{i=1}^n}\{I(X_i\le x)-F\left(x\right)\},x\in R.\hfill \end{array}$$

Therefore, (2) guarantees that

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{log(logn)}}{\displaystyle \underset{x}{sup}}\left|U_n\left(x\right)\right|=O\left(1\right)a.s.\end{array}$$

(11)

For independent uniform $[0,1]$ random variables: ${\xi }_1,{\xi }_2,\cdots ,{\xi }_n$, we define

$$V_n\left(v\right):=\frac{1}{n}\sum _{i=1}^n\left[I({\xi }_i\le v)-v\right],\forall v\in [0,1].$$

Then, $V_n\left(v\right)$ is a standardized uniform $[0,1]$ empirical process, and $U_n\left(x\right)$ has the same distribution as $V_n\left(F\left(x\right)\right)$. Using Theorem 4.4.3 and Theorem 1.15.2 in Csörgő and Révész (1981) [10], applying the Kiefer type approximation of the empirical process, there exists a Kiefer process $\left\{K\right(s;t):0\le s\le 1,0\le t<\infty \}$ such that

$$\begin{array}{c}\hfill {\displaystyle \underset{x}{sup}}|n{U}_n\left(x\right)-K\left(F\left(x\right),n\right)|=O\left({(logn)}^2\right)\mathrm{a}.\mathrm{s}.,\end{array}$$

(12)

with $B_n\left(v\right)=K(v,n)/\sqrt{n},0\le v\le 1$ being a Brownian bridge.

The Proof of Theorem 1 involves three parts: (i) applying the the triangular inequality to the distribution functions, (ii) using the the Kiefer approximation of the empirical process, and (iii) applying the the Taylor expansion. See below for details.

Proof of Theorem 1. 

Rewrite $\widehat{F}\left(x\right)$,

$$\begin{array}{c}\hfill \widehat{F}\left(x\right)=n^{-1}{\displaystyle \sum _{i=1}^n}{\int }_{-\infty }^x{k}_h\left(u-{\epsilon }_i\right)du=n^{-1}{\displaystyle \sum _{i=1}^n}G\left(\frac{x-{\epsilon }_i}{h}\right),\end{array}$$

(13)

where $G\left(x\right)={\int }_{-\infty }^{x}k\left(u\right)du.$ By the definition of $F_n\left(x\right)$, performing integration by parts and a change of variable $u=\frac{x-t}{h}$, we can continue to rewrite $\widehat{F}\left(x\right)$, as follows:

$$\begin{array}{ccc}\hfill \widehat{F}\left(x\right)& =& {\int }_{-\infty }^{+\infty }G\left(\frac{x-t}{h}\right)d{F}_n\left(t\right)\hfill \\ & =& G\left(\frac{x-t}{h}\right)F_n\left(t\right){|}_{-\infty }^{+\infty }+{\int }_{-\infty }^{+\infty }{F}_n\left(t\right)k\left(\frac{x-t}{h}\right)\frac{1}{h}dt\hfill \\ & =& {\int }_{-\infty }^{+\infty }{F}_n\left(t\right)\frac{1}{h}k\left(\frac{x-t}{h}\right)dt={\int }_{-\infty }^{+\infty }{F}_n\left(x-hu\right)k\left(u\right)du.\hfill \end{array}$$

(14)

Combining (14) with the properties $k\left(u\right)\ge 0$ and ${\int }_{-\infty }^{+\infty }k\left(u\right)du=1$, and applying the triangular inequality, we have that

$$\begin{array}{ccc}& & |\widehat{F}\left(x\right)-F_n\left(x\right)|=\left|{\int }_{-\infty }^{+\infty }\left[F_n\left(x-hu\right)-F_n\left(x\right)\right]k\left(u\right)du\right|\hfill \\ & & \le \left|{\int }_{-\infty }^{+\infty }\left\{[F_n\left(x-hu\right)-F\left(x-hu\right)]-[F_n\left(x\right)-F\left(x\right)]\right\}k\left(u\right)du\right|\hfill \\ & & +\left|{\int }_{-\infty }^{+\infty }\left[F(x-hu)-F\left(x\right)\right]k\left(u\right)du\right|\hfill \\ & & =\left|{\int }_{-\infty }^{+\infty }[U_n\left(x-hu\right)-U_n\left(x\right)]k\left(u\right)du\right|+\left|{\int }_{-\infty }^{+\infty }\left[F(x-hu)-F\left(x\right)\right]k\left(u\right)du\right|.\hfill \end{array}$$

Moreover, this results in

$$\begin{array}{ccc}& & {\displaystyle \underset{x}{sup}}|\widehat{F}\left(x\right)-F_n\left(x\right)|\hfill \\ & & \le {\displaystyle \underset{x}{sup}}\left|{\int }_{-\infty }^{+\infty }[U_n\left(x-hu\right)-U_n\left(x\right)]k\left(u\right)du\right|+{\displaystyle \underset{x}{sup}}\left|{\int }_{-\infty }^{+\infty }\left[F(x-hu)-F\left(x\right)\right]k\left(u\right)du\right|\hfill \\ & & =:D_{1n}+D_{2n},say.\hfill \end{array}$$

Thus, to show Theorem 1, it is sufficient to verify that

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{log(logn)}}{D}_{1n}=0\mathrm{a}\mathrm{n}\mathrm{d}{\displaystyle \underset{n\to \infty }{lim}}\sqrt{\frac{n}{log(logn)}}{D}_{2n}=0\mathrm{a}.\mathrm{s}.\end{array}$$

(15)

By $log\left(logn\right)\to \infty$ and the integrability of $k\left(u\right)$, it follows that

$$\begin{array}{c}\hfill \left({\int }_{log\left(logn\right)}^{+\infty }+{\int }_{-\infty }^{-log\left(logn\right)}\right)k\left(u\right)du=o\left(1\right).\end{array}$$

(16)

Partitioning the integral in $D_{1n}$ into three parts, and using the triangular inequality, we can obtain

$$\begin{array}{ccc}& & D_{1n}={\displaystyle \underset{x}{sup}}\left|\left({\int }_{-\infty }^{-log\left(logn\right)}+{\int }_{log\left(logn\right)}^{+\infty }+{\int }_{-log\left(logn\right)}^{-log\left(logn\right)}\right)[U_n\left(x-hu\right)-U_n\left(x\right)]k\left(u\right)du\right|\hfill \\ & \le & {\displaystyle \underset{x}{sup}}\left({\int }_{-\infty }^{-log\left(logn\right)}+{\int }_{log\left(logn\right)}^{+\infty }+{\int }_{-log\left(logn\right)}^{log\left(logn\right)}\right)|U_n\left(x-hu\right)-U_n\left(t\right)|k\left(u\right)du\hfill \\ & \le & 2\left({\displaystyle \underset{x}{sup}}\left|U_n\left(x\right)\right|\right)\left({\int }_{-\infty }^{-log\left(logn\right)}+{\int }_{log\left(logn\right)}^{+\infty }\right)k\left(u\right)du+{\displaystyle \underset{x}{sup}}{\int }_{-log\left(logn\right)}^{log\left(logn\right)}|U_n\left(x-hu\right)-U_n\left(x\right)|k\left(u\right)du\hfill \\ & =:& D_{11n}+D_{12n},say.\hfill \end{array}$$

(17)

It is easy to see that (11) and (16) imply that

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{log(logn)}}{D}_{11n}=0\mathrm{a}.\mathrm{s}.\end{array}$$

(18)

As for $D_{12n}$, with the triangular inequality, (12), ${\int }_{-log\left(logn\right)}^{log\left(logn\right)}k\left(u\right)du\le 1$ and the continuity of modulus of $B_n\left(F\left(x\right)\right)=K\left(F\right(x),n)/\sqrt{n}$, we have

$$\begin{array}{ccc}& & D_{12n}\hfill \\ & \le & {\displaystyle \underset{x}{sup}}\frac{1}{n}\left|{\int }_{-log\left(logn\right)}^{log\left(logn\right)}\left\{[n{U}_n\left(x-hu\right)-K(F(x-hu),n)]-[n{U}_n\left(x\right)-K(F\left(x\right),n)]\right\}k\left(u\right)du\right|\hfill \\ & & +{\displaystyle \underset{x}{sup}}\frac{1}{n}\left|{\int }_{-log\left(logn\right)}^{log\left(logn\right)}[K(F(x-hu),n)-K(F\left(x\right),n)]k\left(u\right)du\right|\hfill \\ & \le & \frac{2}{n}\left({\displaystyle \underset{x}{sup}}|n{U}_n\left(x\right)-K(F\left(x\right),n)|\right)+O(hlog(logn)\sqrt{hlog(logn)}/\sqrt{n}\hfill \\ & =& O\left(\frac{{(logn)}^2}{n}\right)+O\left(hlog(logn)\sqrt{hlog(logn)}/\sqrt{n}\right)\mathrm{a}.\mathrm{s}.\hfill \end{array}$$

Hence, combining the above bound with $\frac{{(logn)}^2}{\sqrt{nlog\left(logn\right)}}\to 0$ and the assumption $h^{3/2}log(logn)\to 0$, it follows that

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{log(logn)}}{D}_{12n}=0.\mathrm{a}.\mathrm{s}.\end{array}$$

(19)

Next, we proceed to evaluate $D_{2n}$. Using the Taylor expansion with integral remainder, the properties ${\int }_{-\infty }^{+\infty }uk\left(u\right)du=0$, ${\int }_{-\infty }^{+\infty }{u}^{2}k\left(u\right)du<\infty$, ${sup}_t\left|f^{\prime }\left(t\right)\right|<\infty$ and $n{h}^4/log(logn)\to 0$, we obtain

$$\begin{array}{ccc}\hfill F(x-hu)-F\left(x\right)& =& -huf\left(x\right)+{\int }_x^{x-hu}{f}^{\prime }\left(s\right)(x-hu-s)ds\hfill \\ & =& -huf\left(x\right)+{\int }_0^{-hu}{f}^{\prime }(x-hu-t)tdt,\hfill \\ \hfill D_{2n}& =& {\displaystyle \underset{x}{sup}}\left|{\int }_{-\infty }^{+\infty }\left[{\int }_0^{-hu}{f}^{\prime }(x-hu-t)tdt\right]k\left(u\right)du\right|\hfill \\ & \le & {\displaystyle \underset{x}{sup}}\left|f^{\prime }\left(x\right)\right|{\int }_{-\infty }^{+\infty }\frac{1}{2}{h}^2{u}^{2}k\left(u\right)du=O\left(h^2\right),\hfill \\ \hfill \sqrt{\frac{n}{log(logn)}}{D}_{2n}& =& O\left(\sqrt{\frac{n{h}^4}{log(logn)}}\right)\to 0.\hfill \end{array}$$

(20)

Therefore, (15) can be produced from (17)–(20). We have completed the proof of Theorem 1. □

Proof of Corollary 1. 

Decomposing $\widehat{F}\left(x\right)-F\left(x\right)$ into two parts and using the triangular inequality, we have

$$\begin{array}{ccc}\hfill |\widehat{F}\left(x\right)-F\left(x\right)|& =& |\widehat{F}\left(x\right)-F_n\left(x\right)+F_n\left(x\right)-F\left(x\right)|\hfill \\ & \le & |\widehat{F}\left(x\right)-F_n\left(x\right)|+|F_n\left(x\right)-F\left(x\right)|,\hfill \\ \hfill |\widehat{F}\left(x\right)-F\left(x\right)|& \ge & -|\widehat{F}\left(x\right)-F_n\left(x\right)|+|F_n\left(x\right)-F\left(x\right)|.\hfill \end{array}$$

Then, it follows that

$$\begin{array}{ccc}\hfill {\displaystyle \underset{x\in R}{sup}}|\widehat{F}\left(x\right)-F\left(x\right)|& \le & {\displaystyle \underset{x\in R}{sup}}|\widehat{F}\left(x\right)-F_n\left(x\right)|+{\displaystyle \underset{x\in R}{sup}}|F_n\left(x\right)-F\left(x\right)|\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill {\displaystyle \underset{x\in R}{sup}}|\widehat{F}\left(x\right)-F\left(x\right)|& \ge & -{\displaystyle \underset{x\in R}{sup}}|\widehat{F}\left(x\right)-F_n\left(x\right)|+{\displaystyle \underset{x\in R}{sup}}|F_n\left(x\right)-F\left(x\right)|.\hfill \end{array}$$

Combining the above inequalities with (8) and (2), this guarantees that

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{2log(logn)}}{\displaystyle \underset{x}{sup}}|\widehat{F}\left(x\right)-F\left(x\right)|=\frac{1}{2}a.s.\end{array}$$

Thus, we have finished the proof of Theorem 1. □

Proof of Theorem 2. 

For any $p\ge 1$, using the triangular inequality and the fact that ${\int }_{-\infty }^{\infty }1dF\left(x\right)=1$, we have that

$$\begin{array}{ccc}& & {\left[{\int }_{-\infty }^{\infty }{|\widehat{F}\left(x\right)-F\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}\hfill \\ & \le & {\left[{\int }_{-\infty }^{\infty }{|\widehat{F}\left(x\right)-F_n\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}+{\left[{\int }_{-\infty }^{\infty }{|F_n\left(x\right)-F\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}\hfill \\ & \le & {\displaystyle \underset{x\in R}{sup}}|\widehat{F}\left(x\right)-F_n\left(x\right)|+{\left[{\int }_{-\infty }^{\infty }{|F_n\left(x\right)-F\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}\hfill \end{array}$$

(21)

and

$$\begin{array}{ccc}& & {\left[{\int }_{-\infty }^{\infty }{|\widehat{F}\left(x\right)-F\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}\hfill \\ & \ge & -{\left[{\int }_{-\infty }^{\infty }{|\widehat{F}\left(x\right)-F_n\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}+{\left[{\int }_{-\infty }^{\infty }{|F_n\left(x\right)-F\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}\hfill \\ & \ge & -{\displaystyle \underset{x\in R}{sup}}|\widehat{F}\left(x\right)-F_n\left(x\right)|+{\left[{\int }_{-\infty }^{\infty }{|F_n\left(x\right)-F\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}.\hfill \end{array}$$

(22)

Note that we have proven

$${\displaystyle \underset{x\in R}{sup}}\sqrt{\frac{n}{log(logn)}}|\widehat{F}\left(x\right)-F_n\left(x\right)|\to 0,\mathrm{a}.\mathrm{s}.$$

in Theorem 1. Thus, combining it with (21), (22) and the law of the iterated logarithm for $L_p$-norm of $F_n\left(x\right)$ in (5), it follows that

$$\begin{array}{c}\hfill lim{\displaystyle \underset{n\to \infty }{sup}}\sqrt{\frac{n}{2log(logn)}}{\left[{\int }_{-\infty }^{\infty }{|\widehat{F}\left(x\right)-F\left(x\right)|}^{p}dF\left(x\right)\right]}^{1/p}=C\left(p\right)a.s.\end{array}$$

Therefore, we have finished the proof of Theorem 2. □

Proof of Corollary 2. 

Corollary 2 is the special case of results of Theorem 2 with $p=1,2$. □