Invariance Principle and Berry–Esseen Bound for Error Variance Estimator for Pth-Order Nonlinear Autoregressive Models

Abstract

The invariance principle and Berry–Esseen bound for an error variance estimator based on the residuals are established by using a Taylor expansion and the classical invariance principle and Berry–Esseen bound for independent random variables. Some examples are given to illustrate their applications.

1. Introduction and Main Results

Let $\{X_k,k\in \mathbb{Z}\}$ be a sequence of strictly stationary random variables that fulfills the following nonlinear autoregressive models of order p (NAR(p), for short):

$$\begin{array}{c}\hfill X_k=g_{\mathit{\theta }}\left(X_{k-1},\dots ,X_{k-p}\right)+{\epsilon }_k,\end{array}$$

(1)

where $g_{\mathit{\theta }}:{\mathbb{R}}^p\to \mathbb{R}$ is a collection of known measurable functions, $\mathit{\theta }={\left({\theta }_1,\dots ,{\theta }_q\right)}^{\prime }\in \mathsf{\Theta }\subset {\mathbb{R}}^q$, $\{{\epsilon }_k,k\in \mathbb{Z}\}$ is a collection of mean zero i.i.d. random variables with finite variance ${\sigma }^2$.

In recent years, an increasing number of scholars have been studying the error sequence in Model (1). One of the research directions is an error density estimator, and some classic results can be found in Liebscher [1], Cheng and Sun [2], Fu and Yang [3], Cheng [4], Hilgert and Portier [5], Kim et al. [6], Li [7], Cheng [8], Liu and Zhang [9], and Wu et al. [10]. Another research direction is an error variance estimator. There are fewer results for the error variance estimator compared to the error density estimator. Cheng [11] and Liang and Zhang [12] discussed asymptotic normality and almost sure central limit theorem for the error variance estimator from NAR(p) with i.i.d. errors, respectively.

The invariance principle can be seen mathematically as the functional central limit theorem, which is more effective than the classical central limit theorem. By combining the invariance principle with the continuity of convergence according to the distribution, one can obtain the asymptotic distribution of various statistics. The Berry–Esseen bound is commonly used to represent the speed at which the distribution function $F_n\left(x\right)$ of the regularized sum of the first n terms of a random variable sequence $\{X_n,n\ge 1\}$ converges to zero compared to its limit distribution $\mathsf{\Phi }\left(x\right)$. They have a wide range of applications in disciplines such as mathematical statistics, econometrics, and financial mathematics.

The purpose of this article is to establish the invariance principle and Berry–Esseen bound of the estimator of the error variance ${\sigma }^2$ by using the observation value $\{X_1,X_2,\cdots ,X_n\}$ from Model (1). The biggest inconvenience is that the error sequence and the residual sequence cannot be observed directly. Taylor’s expansion and some technical methods are needed to solve it. Firstly, we provide an estimator $\widehat{\mathit{\theta }}={({\widehat{\theta }}_1,\dots ,{\widehat{\theta }}_q)}^{\prime }$ of $\mathit{\theta }$. Secondly, on the basis of $\widehat{\mathit{\theta }}$ and Model (1) above, the residuals below are established.

$$\begin{array}{c}\hfill {\widehat{\epsilon }}_k=X_k-g_{\widehat{\mathit{\theta }}}\left(X_{k-1},\dots ,X_{k-p}\right),k=1,2,\dots ,n.\end{array}$$

(2)

Finally, by (2), the estimator of ${\sigma }^2$ is given by

$$\begin{array}{c}\hfill {\widehat{\sigma }}_{\left[nt\right]}^2={\displaystyle \frac{1}{n}}\sum _{k=1}^{\left[nt\right]}{\widehat{\epsilon }}_k^2,0\le t\le 1.\end{array}$$

(3)

The underlying assumptions below for Model (1) are needed throughout this paper.

Assumption 1A. 

For arbitrary $z\in {\mathbb{R}}^p$, $\mathit{\theta }={({\theta }_1,\dots ,{\theta }_q)}^{\prime }\in \mathbb{V}\subset \mathsf{\Theta }\subset {\mathbb{R}}^q$, suppose that

$$\left|{\displaystyle \frac{\partial }{\partial {\theta }_j}}{g}_{\mathit{\theta }}\left(z\right)\right|\le N_1\left(z\right),\left|{\displaystyle \frac{{\partial }^2}{\partial {\theta }_j\partial {\theta }_l}}{g}_{\mathit{\theta }}\left(z\right)\right|\le N_2\left(z\right),$$

where $E\left[N_1^2(X_{k-1},\dots ,X_{k-p})\right]<\infty$ and $E\left[N_2^2(X_{k-1},\dots ,X_{k-p})\right]<\infty$ for every $k\ge 1$, $1\le j,l\le q$.

Assumption 1B. 

$$\begin{array}{c}\hfill {\left|\widehat{\mathit{\theta }}-\mathit{\theta }\right|}^2=\sum _{j=1}^q{({\widehat{\theta }}_j-{\theta }_j)}^2\le C{\displaystyle \frac{loglogn}{n}},a.s.\end{array}$$

(4)

where $\widehat{\mathit{\theta }}$ is a strong consistent estimator of θ and $C>0$ is a constant.

Remark 1. 

Assumption 1B can be satisfied for some models. Please see the following corollaries for details.

We will state the first result concerning the invariance principle for error variance estimator ${\widehat{\sigma }}_{\left[nt\right]}^2$.

Theorem 1. 

For Model (1), under Assumptions 1A and 1B , if $E{\epsilon }_1=0$, $E{\epsilon }_1^2={\sigma }^2$ and $E{\epsilon }_1^4<\infty$, then

$$\begin{array}{c}\hfill \underset{0\le t\le 1}{sup}{\displaystyle \frac{n({\widehat{\sigma }}_{\left[nt\right]}^2-\frac{\left[nt\right]}{n}{\sigma }^2)}{\sqrt{nVar\left({\epsilon }_1^2\right)}}}\stackrel{\mathrm{d}}{\to }\underset{0\le t\le 1}{sup}W\left(t\right),n\to \infty .\end{array}$$

(5)

particularly for $t=1$.

$$\begin{array}{c}\hfill \sqrt{n}({\widehat{\sigma }}_n^2-{\sigma }^2)\stackrel{\mathrm{d}}{\to }\mathcal{N}(0,Var\left({\epsilon }_1^2\right)),n\to \infty .\end{array}$$

The second result is the Berry–Esseen bound for error variance estimator ${\widehat{\sigma }}_n^2$.

Theorem 2. 

For Model (1), under Assumptions A and B, if $E{\epsilon }_1=0$, $E{\epsilon }_1^2={\sigma }^2$ and $E{\epsilon }_1^{4+2\delta }<\infty$, for $0<\delta \le 1$, we have

$$\begin{array}{c}\hfill \underset{z\in \mathbb{R}}{sup}\left|P\left\{{\displaystyle \frac{\sqrt{n}}{\sqrt{Var\left({\epsilon }_1^2\right)}}}\left({\widehat{\sigma }}_n^2-{\sigma }^2\right)\le z\right\}-\mathsf{\Phi }\left(z\right)\right|\le C\left({\displaystyle \frac{1}{n^{\delta /2}}}+{\displaystyle \frac{{(loglogn)}^{2/3}}{n^{1/3}}}\right),\end{array}$$

(6)

where $\mathsf{\Phi }(·)$ stands for the distribution function of the standard normal distribution.

Remark 2. 

Theorem 1 generalizes Theorem 2.2 of Cheng [11] from the central limit theorem to the invariance principle for error variance estimator ${\widehat{\sigma }}_{\left[nt\right]}^2$.

Some typical examples are shown to test the invariance principle and Berry–Esseen bound for the error variance estimator for some special NAR(p). The first one is a degenerate case, which concerns AR(1) models.

Example 1. 

Consider the following AR(1) model:

$$X_k=\theta X_{k-1}+{\epsilon }_k,k\ge 1,$$

where $\left|\theta \right|<1$.

Assumption 1A obviously holds for AR(1) model. Assumption 1B can be satisfied for the least squares estimator $\widehat{\theta }$ from Theorem 1 of Wang et al. [13] under the condition $Eexp\left\{\alpha \right|{\epsilon }_k{\epsilon }_l\left|\right\}<\infty$. Thus, Theorems 1 and 2 hold for the AR(1) model. We summarize it as the following corollary:

Corollary 1. 

For the AR(1) model, if $Eexp\left\{\alpha \right|{\epsilon }_k{\epsilon }_l\left|\right\}<\infty$ for some $\alpha >0$ and arbitrary $k,l\ge 1$, one can obtain

$$\begin{array}{c}\hfill \underset{0\le t\le 1}{sup}{\displaystyle \frac{n({\widehat{\sigma }}_{\left[nt\right]}^2-\frac{\left[nt\right]}{n}{\sigma }^2)}{\sqrt{nVar\left({\epsilon }_1^2\right)}}}\stackrel{\mathrm{d}}{\to }\underset{0\le t\le 1}{sup}W\left(t\right),n\to \infty ,\end{array}$$

and

$$\begin{array}{c}\hfill \underset{z\in \mathbb{R}}{sup}\left|P\left\{{\displaystyle \frac{\sqrt{n}}{\sqrt{Var\left({\epsilon }_1^2\right)}}}\left({\widehat{\sigma }}_n^2-{\sigma }^2\right)\le z\right\}-\mathsf{\Phi }\left(z\right)\right|\le {\displaystyle \frac{C}{n^{\delta /2}}}+{\displaystyle \frac{C{(loglogn)}^{2/3}}{n^{1/3}}},\end{array}$$

where $0<\delta \le 1$.

The second example encompasses the self-exciting threshold autoregressive (SETAR) models.

Example 2. 

Let $\{X_k,k\ge p\}$ satisfy the continuous SETAR($p,l,d$) models, as follows:

$$X_k=\left\{\begin{array}{c}{a}_0+\sum _{m=1}^p{a}_m{X}_{k-m}+{\epsilon }_k\mathit{if}{X}_{k-d}\in R_1,\hfill \\ a_0+\sum _{m=1}^p{a}_m{X}_{k-m}+\sum _{n=2}^j{b}_n(X_{k-d}-r_{n-1})+{\epsilon }_k\mathit{if}{X}_{k-d}\in R_j,j=2,\dots ,l\hfill \end{array}\right.$$

where $\{R_n=(r_{n-1},r_n],1\le n\le l\}$ are the disjoint intervals, and $-\infty =r_0<r_1<r_2<\cdots <r_{l-1}<r_l=+\infty$ are the thresholds. ${\mathit{\theta }}_0={(a_0,\dots ,a_p,b_2,\dots ,b_l,r_1,\dots ,r_{l-1})}^{\prime }\in \mathsf{\Theta }\subset {\mathbb{R}}^q$ denote the true parameters of the models, and $\mathit{\theta }={({\overline{a}}_0,\dots ,{\overline{a}}_p,{\overline{b}}_2,\dots ,{\overline{b}}_l,{\overline{r}}_1,\dots ,{\overline{r}}_{l-1})}^{\prime }\in \mathsf{\Theta }$ denote the unknown parameters, ${\tilde{\mathit{X}}}_k={(X_k,\dots ,X_{k-p+1})}^{\prime }$, $q=2l+p-1$. $\{X_{k-d},k\ge p\}$ is the threshold process (sometimes referred to as a switching variable), which controls the switching or jump between the regime $R_j$. The integer $0<d\le p$ is called a delay parameter. The integer $0<d\le p$ is called a delay parameter. The SETAR($p,l,d$) is an AR(p) model in each of the several regimes. As such, it is a piecewise model which is linear in each regime $R_j$, but the overall time series model is nonlinear. The piecewise nature of the SETAR model is able to capture some important nonlinear phenomena, such as sudden jumps, sub and higher harmonics, asymmetric limit cycles and chaos, and amplitude-dependent frequency.

Assumption 1C. 

Assume that $\left\{{\epsilon }_i\right\}$ has the density function h and that the continuous density function f of $X_i$ has a support including the interval $[r_{min}-\eta ,r_{max}+\eta ]$, where $r_{min}=min\{{\overline{r}}_1:\mathit{\theta }\in \mathsf{\Theta }\}$, $r_{max}=max\{{\overline{r}}_{l-1}:\mathit{\theta }\in \mathsf{\Theta }\}$, $\eta >0$. ${\overline{r}}_{n-1}\le {\overline{r}}_n-ϵ$ for every $\mathit{\theta }\in \mathsf{\Theta }$ and $n=2,\dots ,l$ and some $ϵ>0$.

Assumption 1B can be satisfied by using Corollary 3.1 of Liebscher [1], under $E|{\epsilon }_k{|}^{\beta }<\infty$, $E\left|\right|{\tilde{\mathit{X}}}_k{\left|\right|}^{\beta }<\infty$ and Assumption 1C, $\beta >4$.

Thus, Theorems 1 and 2 hold for the SETAR($p,l,d$) model. We summarize it as the following corollary:

Corollary 2. 

For SETAR($p,l,d$) models, under Assumption 1A and Assumption 1C, if $E\left|\right|{\tilde{\mathit{X}}}_k{\left|\right|}^{\beta }<\infty$, $E|{\epsilon }_k{|}^{(4+2\delta )\vee \beta }<\infty$, $\beta >4$, $0<\delta \le 1$; thus,

$$\begin{array}{c}\hfill \underset{0\le t\le 1}{sup}{\displaystyle \frac{n({\widehat{\sigma }}_{\left[nt\right]}^2-\frac{\left[nt\right]}{n}{\sigma }^2)}{\sqrt{nVar\left({\epsilon }_1^2\right)}}}\stackrel{\mathrm{d}}{\to }\underset{0\le t\le 1}{sup}W\left(t\right),n\to \infty ,\end{array}$$

and

$$\begin{array}{c}\hfill \underset{z\in \mathbb{R}}{sup}\left|P\left\{{\displaystyle \frac{\sqrt{n}}{\sqrt{Var\left({\epsilon }_1^2\right)}}}\left({\widehat{\sigma }}_n^2-{\sigma }^2\right)\le z\right\}-\mathsf{\Phi }\left(z\right)\right|\le {\displaystyle \frac{C}{n^{\delta /2}}}+{\displaystyle \frac{C{(loglogn)}^{2/3}}{n^{1/3}}}.\end{array}$$

The third example encompasses the threshold-exponential AR models.

Example 3. 

The threshold-exponential AR models are defined below

$$X_k=\sum _{l=1}^K({\alpha }_l+{\beta }_l{X}_{k-1})I\{X_{k-1}\in R_l\}+c{e}^{-\gamma X_{k-1}^2}{X}_{k-1}+{\epsilon }_k,$$

with $X_0=x_0$ and $\{R_l,1\le l\le K\}$ being non-empty and non-overlapping intervals of $\mathbb{R}$ with ${\bigcup }_l{R}_l=\mathbb{R}$. ${\mathit{\theta }}_0={({\alpha }_1,\cdots ,{\alpha }_K,{\beta }_1,\cdots ,{\beta }_K,c,\gamma )}^{\prime }\in \mathsf{\Theta }\subset {\mathbb{R}}^q$ denote the true parameters, $\mathit{\theta }={({\overline{\alpha }}_1,\cdots ,{\overline{\alpha }}_K,{\overline{\beta }}_1,\cdots ,{\overline{\beta }}_K,\overline{c},\overline{\gamma })}^{\prime }$ denote the unknown parameters, and $\left\{X_{k-1}\right\}$ is the threshold variable, $q=2K+2$.

By Theorem 4 of Yao [14], Assumption 1B can be satisfied if $c\ne 0$, $\gamma >0$, $|{\beta }_l|<1$, $l=1,\cdots ,K$ and $E|{\epsilon }_k{|}^{2+\eta }<\infty$ for some $\eta >0$. Thus, Theorems 1 and 2 hold for threshold-exponential AR progresses. We summarize it as the following corollary:

Corollary 3. 

For threshold-exponential AR models, if $c\ne 0$, $\gamma >0$, $|{\beta }_l|<1$, $l=1,\cdots ,K$, then under Assumption 1A, for $E|{\epsilon }_1{|}^{(2+\eta )\vee 4}<\infty$ for some $\eta >0$, one can obtain

$$\begin{array}{c}\hfill \underset{0\le t\le 1}{sup}{\displaystyle \frac{n({\widehat{\sigma }}_{\left[nt\right]}^2-\frac{\left[nt\right]}{n}{\sigma }^2)}{\sqrt{nVar\left({\epsilon }_1^2\right)}}}\stackrel{\mathrm{d}}{\to }\underset{0\le t\le 1}{sup}W\left(t\right),n\to \infty ,\end{array}$$

for $E|{\epsilon }_1{|}^{(2+\eta )\vee (4+2\delta )}<\infty$ for some $\eta >0$ and $0<\delta \le 1$, one can obtain

$$\begin{array}{c}\hfill \underset{z\in \mathbb{R}}{sup}\left|P\left\{{\displaystyle \frac{\sqrt{n}}{\sqrt{Var\left({\epsilon }_1^2\right)}}}\left({\widehat{\sigma }}_n^2-{\sigma }^2\right)\le z\right\}-\mathsf{\Phi }\left(z\right)\right|\le {\displaystyle \frac{C}{n^{\delta /2}}}+{\displaystyle \frac{C{(loglogn)}^{2/3}}{n^{1/3}}}.\end{array}$$

The last example is the multilayer perceptron model.

Example 4. 

Consider the following multilayer perceptrons model:

$$X_k=\sum _{j=1}^K{\alpha }_j\psi \left(\sum _{l=1}^p{\beta }_{lj}{X}_{k-l}+{\beta }_{0j}\right)+{\alpha }_0+{\epsilon }_k,$$

where ${\mathit{\theta }}_0={({\alpha }_0,\cdots ,{\alpha }_K,{\beta }_{lj},0\le l\le p,1\le j\le K)}^{\prime }\in \mathsf{\Theta }\subset {\mathbb{R}}^q$ denote the the true parameters, $\mathit{\theta }={({\overline{\alpha }}_0,\cdots ,{\overline{\alpha }}_K,{\overline{\beta }}_{lj},0\le l\le p,1\le j\le K)}^{\prime }$ denote the unknown parameters, and $\psi (·)$ is called an activation function, $q=1+K(p+1)$. The above models can describe that it has one output unit which provides the variable $X_k$, p input units being fed by variables $X_{k-1},\cdots ,X_{k-p}$ at time k and a hidden layer with K units. Due to its universal approximation ability, it has become popular in nonlinear models. We will take $\psi \left(x\right)=tanh\left(x\right)={\displaystyle \frac{e^x-e^{-x}}{e^x+e^{-x}}}$ in this example as in Yao [14].

By Theorem 5 of Yao [14], Assumption 1B can be satisfied if $E|{\epsilon }_k{|}^{6+\eta }<\infty$ for some $\eta >0$, $r_{\mathit{\theta }}\left(x\right)\ne r_{{\theta }_0}\left(x\right)$ for every $\mathit{\theta }$ different from ${\mathit{\theta }}_0$ and some $x\in {\mathbb{R}}^p$, $\psi \left(x\right)=tanh\left(x\right)={\displaystyle \frac{e^x-e^{-x}}{e^x+e^{-x}}}$ and the matrix $I_0$ is regular, where

$$I_0=2{\int }_{{\mathbb{R}}^p}{M}_{{\mathit{\theta }}_0}\left(x\right){\mu }_{{\mathit{\theta }}_0}\left(dx\right),M_{\mathit{\theta }}\left(x\right)={({\displaystyle \frac{\partial r_{\mathit{\theta }}\left(x\right)}{\partial {\theta }_i}}·{\displaystyle \frac{\partial r_{\mathit{\theta }}\left(x\right)}{\partial {\theta }_j}})}_{1\le i,j\le q}.$$

Thus Theorems 1 and 2 hold for multilayer perceptrons models. We summarize it as the following corollary:

Corollary 4. 

For multilayer perceptrons models with $E|{\epsilon }_k{|}^{6+\eta }<\infty$ for some $\eta >0$, $r_{\mathit{\theta }}\left(x\right)\ne r_{{\mathit{\theta }}_0}\left(x\right)$ for every θ different from ${\mathit{\theta }}_0$ and some $x\in {\mathbb{R}}^p$, $\psi \left(x\right)=tanh\left(x\right)={\displaystyle \frac{e^x-e^{-x}}{e^x+e^{-x}}}$, and with the matrix $I_0$ being regular, if Assumption 1A holds, then one can obtain

$$\begin{array}{c}\hfill \underset{0\le t\le 1}{sup}{\displaystyle \frac{n({\widehat{\sigma }}_{\left[nt\right]}^2-\frac{\left[nt\right]}{n}{\sigma }^2)}{\sqrt{nVar\left({\epsilon }_1^2\right)}}}\stackrel{\mathrm{d}}{\to }\underset{0\le t\le 1}{sup}W\left(t\right),n\to \infty .\end{array}$$

and

$$\begin{array}{c}\hfill \underset{z\in \mathbb{R}}{sup}\left|P\left\{{\displaystyle \frac{\sqrt{n}}{\sqrt{Var\left({\epsilon }_1^2\right)}}}\left({\widehat{\sigma }}_n^2-{\sigma }^2\right)\le z\right\}-\mathsf{\Phi }\left(z\right)\right|\le {\displaystyle \frac{C}{n^{\delta /2}}}+{\displaystyle \frac{C{(loglogn)}^{2/3}}{n^{1/3}}},\end{array}$$

where $0<\delta \le 1$.

The structure of this article is as follows. The background and main results of this paper are declared in Section 1. Auxiliary lemmas are stated in Section 2. The proofs are given in Section 3. Throughout this paper, C denotes a constant that may be distinctly subjected to its locations. Let $I\left\{A\right\}$ be the indicator function of the collection A.

2. Preliminary Lemmas

We will use the following lemmas:

Lemma 1 

(Hall and Heyde [15]). Let $Y_1=T_1$ and $Y_j=T_j-T_{j-1}$, $2\le j\le n$. If $\{T_j,{\mathcal{F}}_j,1\le j\le n\}$ is a martingale and $q>0$, then

$$\begin{array}{c}\hfill E\left(\underset{1\le j\le n}{max}{\left|T_j\right|}^q\right)\le C_q\left\{E\left[{\left(\sum _{j=1}^{n}E\left(Y_j^2\right|{\mathcal{F}}_{j-1})\right)}^{q/2}\right]+E\left(\underset{1\le j\le n}{max}{\left|Y_i\right|}^q\right)\right\}.\end{array}$$

where $C_q$ is a constant depending only on q.

Lemma 2 

(Liang and Zhang [12]). Under the assumptions of Theorem 1 or 2, for $1\le i\le n$, $1\le j$, $l\le q$; then, for any $2\le s\le 4$, one can obtain

$$\begin{array}{cc}\hfill & E|Y_{kj}{|}^s\le E{N}_1^s(X_{k-1},\dots ,X_{k-p})\le {\left(E{N}_1^4(X_{k-1},\dots ,X_{k-p})\right)}^{s/4}<\infty ,\hfill \\ \hfill & E|Z_{kjl}{|}^s\le E{N}_2^s(X_{k-1},\dots ,X_{k-p})\le {\left(E{N}_2^4(X_{k-1},\dots ,X_{k-p})\right)}^{s/4}<\infty ,\hfill \end{array}$$

where

$$Y_{kj}={\displaystyle \frac{\partial }{\partial {\theta }_j}}{g}_{\mathit{\theta }}(X_{k-1},\dots ,X_{k-p}),Z_{kjl}={\displaystyle \frac{{\partial }^2}{\partial {\theta }_j\partial {\theta }_l}}{g}_{{\mathit{\theta }}^{*}}(X_{k-1},\dots ,X_{k-p}),$$

${\mathit{\theta }}^{*}=\mathit{\theta }+\mu (\widehat{\mathit{\theta }}-\mathit{\theta })$ for some $\mu \in (0,1)$.

Lemma 3 

(Chao and Rao [16]). Let $X_1,X_2$ be two random variables; then, for any $z\in \mathbb{R}$ and $b>0$,

$$\underset{z\in \mathbb{R}}{sup}|P(X_1+X_2\le z)-\mathsf{\Phi }\left(z\right)|\le \underset{z\in \mathbb{R}}{sup}|P(X_1\le z)-\mathsf{\Phi }\left(z\right)|+{\displaystyle \frac{b}{\sqrt{2\pi }}}+P\left(\right|X_2|>b).$$

Lemma 4 

(Petrov [17]). Let $X_1,\cdots ,X_n$ be independent random variables with zero means and $E|X_k{|}^{2+\delta }<\infty$ for $0<\delta \le 1$, $1\le k\le n$. Then, there exists a positive constant A such that

$$\begin{array}{c}\hfill \underset{z\in \mathbb{R}}{sup}\left|P\left\{{\displaystyle \frac{1}{B_n^{1/2}}}\sum _{k=1}^n{X}_k\le z\right\}-\mathsf{\Phi }\left(z\right)\right|\le {\displaystyle \frac{A}{B_n^{1+\delta /2}}}\sum _{k=1}^{n}E{\left|X_k\right|}^{2+\delta },\end{array}$$

where $B_n={\sum }_{k=1}^{n}E{\left|X_k\right|}^2$.

3. Proofs

Proof of Theorem 1. 

In order to prove Theorem 1, it suffices to prove

$$\begin{array}{c}\hfill \underset{0\le t\le 1}{sup}\left|{\displaystyle \frac{n({\widehat{\sigma }}_{\left[nt\right]}^2-\frac{\left[nt\right]}{n}{\sigma }^2)}{B_n}}-{\displaystyle \frac{W\left(nt\right)}{\sqrt{n}}}\right|\stackrel{\mathrm{P}}{\to }0,n\to \infty ,\end{array}$$

(7)

where $B_n=\sqrt{nVar\left({\epsilon }_1^2\right)}$.

Noting (1) and (2), via Taylor’s expansion, one can obtain

$$\begin{array}{cc}\hfill {\widehat{\epsilon }}_k=& {\epsilon }_k-[g_{\widehat{\mathit{\theta }}}(X_{k-1},\dots ,X_{k-p})-g_{\mathit{\theta }}(X_{k-1},\dots ,X_{k-p})]\hfill \\ \hfill =& {\epsilon }_j-\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}-{\displaystyle \frac{1}{2}}\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl},\hfill \end{array}$$

(8)

where $Y_{kj}$ and $Z_{kjl}$ are defined as in Lemma 2. Using (8), one can conclude the following:

$$\begin{array}{cc}& \underset{0\le t\le 1}{sup}\left|{\displaystyle \frac{n({\widehat{\sigma }}_{\left[nt\right]}^2-\frac{\left[nt\right]}{n}{\sigma }^2)}{B_n}}-{\displaystyle \frac{W\left(nt\right)}{\sqrt{n}}}\right|\hfill \\ \hfill =& \underset{0\le t\le 1}{sup}\left|{\displaystyle \frac{1}{B_n}}\sum _{k=1}^{\left[nt\right]}({\widehat{\epsilon }}_k^2-{\sigma }^2)-{\displaystyle \frac{W\left(nt\right)}{\sqrt{n}}}\right|\hfill \\ \hfill =& \underset{0\le t\le 1}{sup}\left|{\displaystyle \frac{1}{B_n}}\sum _{k=1}^{\left[nt\right]}({\widehat{\epsilon }}_k^2-{\epsilon }_k^2)-{\displaystyle \frac{1}{B_n}}\sum _{k=1}^{\left[nt\right]}({\epsilon }_k^2-E{\epsilon }_k^2)-{\displaystyle \frac{W\left(nt\right)}{\sqrt{n}}}\right|\hfill \\ \hfill \le & \underset{0\le t\le 1}{sup}{\displaystyle \frac{1}{B_n}}\sum _{k=1}^{\left[nt\right]}{\left(\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}\right)}^2\hfill \\ \hfill +& \underset{0\le t\le 1}{sup}{\displaystyle \frac{1}{B_n}}\sum _{k=1}^{\left[nt\right]}{\left(\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}\right)}^2\hfill \\ \hfill +& \underset{0\le t\le 1}{sup}{\displaystyle \frac{2}{B_n}}\left|\sum _{k=1}^{\left[nt\right]}\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}{\epsilon }_k\right|\hfill \\ \hfill +& \underset{0\le t\le 1}{sup}{\displaystyle \frac{1}{B_n}}\left|\sum _{k=1}^{\left[nt\right]}\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}{\epsilon }_k\right|\hfill \\ \hfill +& \underset{0\le t\le 1}{sup}{\displaystyle \frac{1}{B_n}}\left|\sum _{k=1}^{\left[nt\right]}\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)({\widehat{\theta }}_m-{\theta }_m)Y_{kj}{Z}_{klm}\right|\hfill \\ \hfill +& \underset{0\le t\le 1}{sup}{\displaystyle \frac{1}{B_n}}\left|\sum _{k=1}^{\left[nt\right]}({\epsilon }_k^2-E{\epsilon }_k^2)-{\displaystyle \frac{W\left(nt\right)}{\sqrt{n}}}\right|\hfill \\ \hfill =:& \sum _{j=1}^6{G}_{nj}.\hfill \end{array}$$

(9)

By (9), in order to obtain (7), it is enough to show

$$G_{nj}\stackrel{\mathrm{P}}{\to }0,n\to \infty ,1\le j\le 6.$$

Noticing the fundamental inequality

$$\begin{array}{c}\hfill {\left(\sum _{j=1}^q{c}_j{d}_j\right)}^2\le \left(\sum _{j=1}^q{c}_j^2\right)\left(\sum _{j=1}^q{d}_j^2\right).\end{array}$$

(10)

For $G_{n1}$, via the Markov inequality, (4) and (10), Lemma 2, and $B_n=\sqrt{nVar\left({\epsilon }_1^2\right)}$, we obtain

$$\begin{array}{cc}& P\left(\right|G_{n1}|>\epsilon )=P\left(\underset{0\le t\le 1}{sup}{\displaystyle \frac{1}{B_n}}\sum _{k=1}^{\left[nt\right]}{\left(\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}\right)}^2>\epsilon \right)\hfill \\ \le & {\displaystyle \frac{1}{\epsilon }}{\displaystyle \frac{1}{B_n}}E\underset{0\le t\le 1}{sup}\left[\sum _{k=1}^{\left[nt\right]}{\left(\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}\right)}^2\right]\hfill \\ \le & {\displaystyle \frac{1}{\epsilon }}{\displaystyle \frac{1}{B_n}}E\left[\sum _{j=1}^q{({\widehat{\theta }}_j-{\theta }_j)}^2·\sum _{j=1}^q\sum _{k=1}^n{Y}_{kj}^2\right]\hfill \\ \le & {\displaystyle \frac{C}{\epsilon }}{\displaystyle \frac{1}{B_n}}·{\displaystyle \frac{loglogn}{n}}·\sum _{j=1}^q\sum _{k=1}^{n}E{Y}_{kj}^2\hfill \\ \le & {\displaystyle \frac{C}{\epsilon }}{\displaystyle \frac{1}{B_n}}·{\displaystyle \frac{loglogn}{n}}·n\hfill \\ \le & {\displaystyle \frac{C}{\epsilon }}{\displaystyle \frac{1}{\sqrt{nVar\left({\epsilon }_1^2\right)}}}·loglogn\hfill \\ \to & 0,n\to \infty .\hfill \end{array}$$

(11)

For $G_{n2}$, via the Markov inequality, (4) and (10), Lemma 2, and $B_n=\sqrt{nVar\left({\epsilon }_1^2\right)}$, we have

$$\begin{array}{cc}& P\left(\right|G_{n2}|>\epsilon )=P\left(\underset{0\le t\le 1}{sup}{\displaystyle \frac{1}{B_n}}\sum _{k=1}^{\left[nt\right]}{\left(\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}\right)}^2>\epsilon \right)\hfill \\ \le & {\displaystyle \frac{1}{\epsilon }}{\displaystyle \frac{1}{B_n}}E\underset{0\le t\le 1}{sup}\left[\sum _{k=1}^{\left[nt\right]}{\left(\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}\right)}^2\right]\hfill \\ \le & {\displaystyle \frac{1}{\epsilon }}{\displaystyle \frac{1}{B_n}}E\left[\sum _{j=1}^q{({\widehat{\theta }}_j-{\theta }_j)}^2·\sum _{l=1}^q{({\widehat{\theta }}_l-{\theta }_l)}^2·\sum _{j=1}^q\sum _{l=1}^q\sum _{k=1}^n{Z}_{kjl}^2\right]\hfill \\ \le & {\displaystyle \frac{C}{\epsilon }}{\displaystyle \frac{1}{B_n}}·{\displaystyle \frac{{(loglogn)}^2}{n^2}}·\sum _{j=1}^q\sum _{l=1}^q\sum _{k=1}^{n}E{Z}_{kjl}^2\hfill \\ \le & {\displaystyle \frac{C}{\epsilon }}{\displaystyle \frac{1}{B_n}}·{\displaystyle \frac{{(loglogn)}^2}{n^2}}·n\hfill \\ \le & {\displaystyle \frac{C}{\epsilon }}{\displaystyle \frac{1}{\sqrt{nVar\left({\epsilon }_1^2\right)}}}·{\displaystyle \frac{{(loglogn)}^2}{n}}\hfill \\ \to & 0,n\to \infty .\hfill \end{array}$$

(12)

For $G_{n3}$, let

$$\begin{array}{c}\hfill Y_m=\sum _{k=1}^m{Y}_{kj}{\epsilon }_k,{\mathcal{F}}_m=\sigma \left(\left\{{\epsilon }_k,1\le k\le m\right\}\right),1\le m\le n.\end{array}$$

By (1) and the independence of $\left\{{\epsilon }_k\right\}$, one can conclude that $Y_{kj}$ and ${\epsilon }_k$ are independent; thus, $\left\{Y_m,{\mathcal{F}}_m,1\le m\le n\right\}$ is a martingale. Using Lemmas 1 and 2, we know

$$\begin{array}{cc}\hfill E\left(\underset{1\le m\le n}{max}{\left|Y_m\right|}^2\right)=& E\underset{0\le t\le 1}{sup}{\left|\sum _{k=1}^{\left[nt\right]}{Y}_{kj}{\epsilon }_k\right|}^2\hfill \\ \hfill \le & CE\left[\left(\sum _{k=1}^{n}E\left(Y_{kj}^2{\epsilon }_k^2|{\mathcal{F}}_{k-1}\right)\right)\right]+CE\left(\underset{1\le k\le n}{max}{\left|Y_{kj}{\epsilon }_k\right|}^2\right)\hfill \\ \hfill \le & CE\left[\left(\sum _{k=1}^{n}E{Y}_{kj}^2\right)\right]·\left[E{\epsilon }_1^2\right]+C\sum _{k=1}^{n}E|Y_{kj}{|}^2·E{\left|{\epsilon }_1\right|}^2\hfill \\ \hfill \le & Cn.\hfill \end{array}$$

(13)

Then, via the Markov inequality, (4), (10), and (13), and $B_n=\sqrt{nVar\left({\epsilon }_1^2\right)}$, we know that

$$\begin{array}{cc}& P\left(\right|G_{n3}|>\epsilon )=P\left(\underset{0\le t\le 1}{sup}{\displaystyle \frac{2}{B_n}}\left|\sum _{k=1}^{\left[nt\right]}\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}{\epsilon }_k\right|>\epsilon \right)\hfill \\ \le & {\displaystyle \frac{1}{{\epsilon }^2}}{\displaystyle \frac{4}{B_n^2}}E\underset{0\le t\le 1}{sup}{\left[\sum _{k=1}^{\left[nt\right]}\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}{\epsilon }_k\right]}^2\hfill \\ \le & {\displaystyle \frac{1}{{\epsilon }^2}}{\displaystyle \frac{4}{B_n^2}}E\left[\sum _{j=1}^q{({\widehat{\theta }}_j-{\theta }_j)}^2·\sum _{j=1}^q\underset{0\le t\le 1}{sup}{\left(\sum _{k=1}^{\left[nt\right]}{Y}_{kj}{\epsilon }_k\right)}^2\right]\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}·{\displaystyle \frac{loglogn}{n}}·\sum _{j=1}^{q}E\underset{0\le t\le 1}{sup}{\left(\sum _{k=1}^{\left[nt\right]}{Y}_{kj}{\epsilon }_k\right)}^2\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}·{\displaystyle \frac{loglogn}{n}}·n\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{nVar\left({\epsilon }_1^2\right)}}·loglogn\hfill \\ \to & 0,n\to \infty .\hfill \end{array}$$

(14)

For $G_{n4}$, via the Markov inequality, (4) and (10), $C_r$ inequality, Cauchy–Schwarz inequality, Lemma 2, and $B_n=\sqrt{nVar\left({\epsilon }_1^2\right)}$, one can obtain

$$\begin{array}{cc}& P\left(\right|G_{n4}|>\epsilon )=P\left(\underset{0\le t\le 1}{sup}{\displaystyle \frac{1}{B_n}}\left|\sum _{k=1}^{\left[nt\right]}\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}{\epsilon }_k\right|>\epsilon \right)\hfill \\ \le & {\displaystyle \frac{1}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}E\underset{0\le t\le 1}{sup}{\left[\sum _{k=1}^{\left[nt\right]}\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}{\epsilon }_k\right]}^2\hfill \\ \le & {\displaystyle \frac{1}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}E\underset{0\le t\le 1}{sup}\left[\sum _{j=1}^q{({\widehat{\theta }}_j-{\theta }_j)}^2·\sum _{l=1}^q{({\widehat{\theta }}_l-{\theta }_l)}^2·\sum _{j=1}^q\sum _{l=1}^q{\left(\sum _{k=1}^{\left[nt\right]}{Z}_{kjl}{\epsilon }_k\right)}^2\right]\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^2}{n^2}}·\sum _{j=1}^q\sum _{l=1}^{q}E\underset{0\le t\le 1}{sup}{\left(\sum _{k=1}^{\left[nt\right]}{Z}_{kjl}{\epsilon }_k\right)}^2\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^2}{n^2}}·\sum _{j=1}^q\sum _{l=1}^{q}E\underset{0\le t\le 1}{sup}\left[nt\right]·\sum _{k=1}^{\left[nt\right]}{Z}_{kjl}^2{\epsilon }_k^2\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^2}{n^2}}·\sum _{j=1}^q\sum _{l=1}^{q}n·\sum _{k=1}^{n}E{Z}_{kjl}^2{\epsilon }_k^2\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^2}{n}}·\sum _{j=1}^q\sum _{l=1}^q\sum _{k=1}^n{\left(E{Z}_{kjl}^4\right)}^{1/2}{\left(E{\epsilon }_k^4\right)}^{1/2}\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^2}{n}}·n\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{nVar\left({\epsilon }_1^2\right)}}·{(loglogn)}^2\hfill \\ \to & 0,n\to \infty .\hfill \end{array}$$

(15)

For $G_{n5}$, via the Markov inequality, (4) and (10), $C_r$ inequality, Cauchy–Schwarz inequality, Lemma 2, and $B_n=\sqrt{nVar\left({\epsilon }_1^2\right)}$, it follows that

$$\begin{array}{cc}& P\left(\right|G_{n5}|>\epsilon )=P\left(\underset{0\le t\le 1}{sup}{\displaystyle \frac{1}{B_n}}\left|\sum _{k=1}^{\left[nt\right]}\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)({\widehat{\theta }}_m-{\theta }_m)Y_{kj}{Z}_{klm}\right|>\epsilon \right)\hfill \\ \le & {\displaystyle \frac{1}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}E\underset{0\le t\le 1}{sup}{\left[\sum _{k=1}^{\left[nt\right]}\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)({\widehat{\theta }}_m-{\theta }_m)Y_{kj}{Z}_{klm}\right]}^2\hfill \\ \le & {\displaystyle \frac{1}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}E\underset{0\le t\le 1}{sup}\left[\sum _{j=1}^q{({\widehat{\theta }}_j-{\theta }_j)}^2·\sum _{l=1}^q{({\widehat{\theta }}_l-{\theta }_l)}^2·\sum _{m=1}^q{({\widehat{\theta }}_m-{\theta }_m)}^2·\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^q{\left(\sum _{k=1}^{\left[nt\right]}{Y}_{kj}{Z}_{klm}\right)}^2\right]\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^3}{n^3}}·\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^{q}E\underset{0\le t\le 1}{sup}{\left(\sum _{k=1}^{\left[nt\right]}{Y}_{kj}{Z}_{klm}\right)}^2\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^3}{n^3}}·\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^{q}E\underset{0\le t\le 1}{sup}\left[nt\right]·\sum _{k=1}^{\left[nt\right]}{Y}_{kj}^2{Z}_{klm}^2\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^3}{n^3}}·\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^{q}n·\sum _{k=1}^{n}E{Y}_{kj}^2{Z}_{klm}^2\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^3}{n^2}}·\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^q\sum _{k=1}^n{\left(E{Y}_{kj}^4\right)}^{1/2}{\left(E{Z}_{kjm}^4\right)}^{1/2}\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{B_n^2}}{\displaystyle \frac{{(loglogn)}^3}{n^2}}·n\hfill \\ \le & {\displaystyle \frac{C}{{\epsilon }^2}}{\displaystyle \frac{1}{nVar\left({\epsilon }_1^2\right)}}·{\displaystyle \frac{{(loglogn)}^3}{n}}\hfill \\ \to & 0,n\to \infty .\hfill \end{array}$$

(16)

By the invariance principle for i.i.d. random variable (see Lu [18]), we know that

$$\begin{array}{c}\hfill P\left(\right|G_{n6}|>\epsilon )\to 0,n\to \infty .\end{array}$$

(17)

In view of (9)–(17), we obtain that (9) holds. Hence, the proof of Theorem 1 is completed. □

Proof of Theorem 2. 

By (8), we can obtain

$$\begin{array}{cc}& \sqrt{{\displaystyle \frac{n}{Var\left({\epsilon }_1^2\right)}}}\left({\widehat{\sigma }}^2-{\sigma }^2\right)\hfill \\ \hfill =& \sqrt{{\displaystyle \frac{n}{Var\left({\epsilon }_1^2\right)}}}\left({\displaystyle \frac{1}{n}}\sum _{ki=1}^n{\widehat{\epsilon }}_k^2-{\displaystyle \frac{1}{n}}\sum _{k=1}^n{\epsilon }_k^2+{\displaystyle \frac{1}{n}}\sum _{k=1}^n{\epsilon }_k^2-{\displaystyle \frac{1}{n}}\sum _{k=1}^{n}E{\epsilon }_k^2\right)\hfill \\ \hfill =& {\displaystyle \frac{1}{B_n}}\left[\sum _{k=1}^n({\widehat{\epsilon }}_k^2-{\epsilon }_k^2)\right]+{\displaystyle \frac{1}{B_n}}\left[\sum _{k=1}^n({\epsilon }_k^2-E{\epsilon }_k^2)\right]\hfill \\ \hfill =& {\displaystyle \frac{1}{B_n}}\left[\sum _{k=1}^n{\left(\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}\right)}^2\right]+{\displaystyle \frac{1}{B_n}}\left[\sum _{k=1}^n{\left(\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}\right)}^2\right]\hfill \\ & -{\displaystyle \frac{1}{B_n}}\left[\sum _{k=1}^n\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}{\epsilon }_k\right]-{\displaystyle \frac{1}{B_n}}\left[\sum _{k=1}^n\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}{\epsilon }_k\right]\hfill \\ & -{\displaystyle \frac{1}{B_n}}\left[\sum _{k=1}^n\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)({\widehat{\theta }}_m-{\theta }_m)Y_{kj}{Z}_{klm}\right]\hfill \\ & +{\displaystyle \frac{1}{B_n}}\left[\sum _{k=1}^n({\epsilon }_k^2-E{\epsilon }_k^2)\right]\hfill \\ \hfill =:& L_{n1}+L_{n2}-L_{n3}-L_{n4}-L_{n5}+L_{n6},\hfill \end{array}$$

(18)

where $B_n=\sqrt{nVar\left({\epsilon }_1^2\right)}$.

Then, by Lemma 3 and (18), one can obtain

$$\begin{array}{cc}& \underset{z\in \mathbb{R}}{sup}\left|P\left\{{\displaystyle \frac{\sqrt{n}}{\sqrt{Var\left({\epsilon }_1^2\right)}}}\left({\widehat{\sigma }}_n^2-{\sigma }^2\right)\le z\right\}-\mathsf{\Phi }\left(z\right)\right|\hfill \\ \le & \underset{z\in \mathbb{R}}{sup}\left|P\left\{|L_{n6}|\le z\right\}-\mathsf{\Phi }\left(z\right)\right|+{\displaystyle \frac{a}{\sqrt{2\pi }}}+P\left(\right|L_{n1}+L_{n2}-L_{n3}-L_{n4}-L_{n5}|>a)\hfill \\ \le & \underset{z\in \mathbb{R}}{sup}\left|P\left\{|L_{n6}|\le z\right\}-\mathsf{\Phi }\left(z\right)\right|+{\displaystyle \frac{a}{\sqrt{2\pi }}}+\sum _{j=1}^{5}P\left(\right|L_{nj}|>a/5).\hfill \end{array}$$

(19)

Using a similar proof to that of (11) with $t=1$ and the Markov inequality, we obtain

$$\begin{array}{cc}& P\left(\right|L_{n1}|>a/5)\le {\displaystyle \frac{5}{a}}E\left|L_{n1}\right|\hfill \\ =& {\displaystyle \frac{5}{a}}\sqrt{{\displaystyle \frac{1}{nVar\left({\epsilon }_1^2\right)}}}E\left[\sum _{k=1}^n{\left(\sum _{j=1}^q({\widehat{\theta }}_j-{\theta }_j)Y_{kj}\right)}^2\right]\hfill \\ \le & {\displaystyle \frac{C}{a}}·{\displaystyle \frac{loglogn}{n^{1/2}}}.\hfill \end{array}$$

(20)

Using a similar method to that of (12) with $t=1$ and the Markov inequality, we know that

$$\begin{array}{cc}& P\left(\right|L_{n2}|>a/5)\le {\displaystyle \frac{5}{a}}E\left|L_{n2}\right|\hfill \\ =& \sqrt{{\displaystyle \frac{1}{4nVar\left({\epsilon }_1^2\right)}}}E\left[\sum _{k=1}^n{\left(\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}\right)}^2\right]\hfill \\ \le & {\displaystyle \frac{C}{a}}·{\displaystyle \frac{{(loglogn)}^2}{n^{3/2}}}.\hfill \end{array}$$

(21)

Using a similar argument to that of (14) with $t=1$ and the Markov inequality, we have

$$\begin{array}{cc}& P\left(\right|L_{n3}|>a/5)\le {\displaystyle \frac{25}{a^2}}E{\left|L_{n3}\right|}^2\hfill \\ =& {\displaystyle \frac{25}{a^2}}{\displaystyle \frac{1}{nVar\left({\epsilon }_1^2\right)}}E{\left[\sum _{k=1}^n\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}{\epsilon }_i\right]}^2\hfill \\ \le & {\displaystyle \frac{C}{a^2}}·{\displaystyle \frac{loglogn}{n}}.\hfill \end{array}$$

(22)

By the similar proof to that of (15) with $t=1$ and Markov inequality, it follows that

$$\begin{array}{cc}& P\left(\right|L_{n4}|>a/5)\le {\displaystyle \frac{25}{a^2}}E{\left|L_{n4}\right|}^2\hfill \\ =& {\displaystyle \frac{25}{a^2}}{\displaystyle \frac{1}{nVar\left({\epsilon }_1^2\right)}}E{\left[\sum _{k=1}^n\sum _{j=1}^q\sum _{l=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)Z_{kjl}{\epsilon }_k\right]}^2\hfill \\ \le & {\displaystyle \frac{C}{a^2}}·{\displaystyle \frac{{(loglogn)}^2}{n}}.\hfill \end{array}$$

(23)

Using a similar method to that of (16) with $t=1$ and the Markov inequality, one can conclude that

$$\begin{array}{cc}& P\left(\right|L_{n5}|>a/5)\le {\displaystyle \frac{25}{a^2}}E\left|L_{n5}\right|\hfill \\ =& {\displaystyle \frac{25}{a^2}}{\displaystyle \frac{1}{nVar\left({\epsilon }_1^2\right)}}E{\left[\sum _{k=1}^n\sum _{j=1}^q\sum _{l=1}^q\sum _{m=1}^q({\widehat{\theta }}_j-{\theta }_j)({\widehat{\theta }}_l-{\theta }_l)({\widehat{\theta }}_m-{\theta }_m)Y_{kj}{Z}_{klm}\right]}^2\hfill \\ \le & {\displaystyle \frac{C}{a^2}}·{\displaystyle \frac{{(loglogn)}^3}{n^2}}.\hfill \end{array}$$

(24)

Using Lemma 4 with $X_k={\epsilon }_k^2-E{\epsilon }_k^2$, we can easily see that

$$\begin{array}{cc}& \underset{z\in \mathbb{R}}{sup}\left|P\left\{|L_{n6}|\le z\right\}-\mathsf{\Phi }\left(z\right)\right|\hfill \\ \le & {\displaystyle \frac{A}{{\left(nVar\left({\epsilon }_1^2\right)\right)}^{1+\delta /2}}}\sum _{k=1}^{n}E{|{\epsilon }_k^2-E{\epsilon }_k^2|}^{2+\delta }\hfill \\ \le & {\displaystyle \frac{C}{n^{\delta /2}}}.\hfill \end{array}$$

(25)

Then, by combining (19)–(25), it follows that

$$\begin{array}{cc}& \underset{z\in \mathbb{R}}{sup}\left|P\left\{{\displaystyle \frac{\sqrt{n}}{\sqrt{Var\left({\epsilon }_1^2\right)}}}\left({\widehat{\sigma }}_n^2-{\sigma }^2\right)\le z\right\}-\mathsf{\Phi }\left(z\right)\right|\hfill \\ \le & C\left({\displaystyle \frac{1}{n^{\delta /2}}}+a+{\displaystyle \frac{1}{a}}·{\displaystyle \frac{loglogn}{n^{1/2}}}+{\displaystyle \frac{1}{a}}·{\displaystyle \frac{{(loglogn)}^2}{n^{3/2}}}\right)\hfill \\ & +C\left({\displaystyle \frac{1}{a^2}}·{\displaystyle \frac{loglogn}{n}}+{\displaystyle \frac{1}{a^2}}·{\displaystyle \frac{{(loglogn)}^2}{n}}+{\displaystyle \frac{1}{a^2}}·{\displaystyle \frac{{(loglogn)}^3}{n^2}}\right)\hfill \\ \le & C\left({\displaystyle \frac{1}{n^{\delta /2}}}+a+{\displaystyle \frac{1}{a}}·{\displaystyle \frac{loglogn}{n^{1/2}}}+{\displaystyle \frac{1}{a^2}}·{\displaystyle \frac{{(loglogn)}^2}{n}}\right)\hfill \\ \le & C\left({\displaystyle \frac{1}{n^{\delta /2}}}+{\displaystyle \frac{{(loglogn)}^{2/3}}{n^{1/3}}}\right),\hfill \end{array}$$

and thus the last inequality is obtained by taking $a={\displaystyle \frac{{(loglogn)}^{2/3}}{n^{1/3}}}$. Therefore, (6) is proved. □

4. Conclusions

In this article, via Taylor’s expansion, the classical invariance principle, and Berry–Esseen bound for independent random variables, the authors established the invariance principle and Berry–Esseen bound for an error variance estimator for NAR with i.i.d. errors. The central limit theorem for the error variance estimator was extended to the invariance principle. Four corollaries were given to test the results. In the future, the moderate deviation principle and large deviation principle for the error variance estimator for NAR with independent errors will be considered.