Almost Sure Central Limit Theorem for Error Variance Estimator in Pth-Order Nonlinear Autoregressive Processes

Abstract

In this paper, under some suitable assumptions, using the Taylor expansion, Borel–Cantelli lemma and the almost sure central limit theorem for independent random variables, the almost sure central limit theorem for error variance estimator in the pth-order nonlinear autoregressive processes with independent and identical distributed errors was established. Four examples, first-order autoregressive processes, self-exciting threshold autoregressive processes, threshold-exponential AR progresses and multilayer perceptrons progress, are given to verify the results.

1. Introduction

Over the past twenty years, there has been an increasing interest in the nonlinear time series literature, for example, the monograph by Tong [1] represents a good account of nonlinear time series models. Compared to linear models, studying the properties of estimators in nonlinear time series models is technically more complex and difficult. In this paper, we will investigate the properties of estimators in nonlinear autoregressive processes.

Throughout this paper, we always assume that $\{{\epsilon }_i,i\in \mathbb{Z}\}$ is a sequence of independent and identically distributed random variables with mean zero, finite variance ${\sigma }^2$. $\{X_i,i\in \mathbb{Z}\}$ is a sequence of strictly stationary real random variables which satisfies nonlinear autoregressive processes of order p

$$\begin{array}{c}\hfill X_i=r_{\theta }\left(X_{i-1},\dots ,X_{i-p}\right)+{\epsilon }_i,\end{array}$$

(1)

for some $\mathit{\theta }={\left({\theta }_1,\dots ,{\theta }_q\right)}^{\prime }\in \Theta \subset {\mathbb{R}}^q$, where $r_{\theta }$, $\mathit{\theta }\in \Theta$, is a family of known measurable functions from ${\mathbb{R}}^p\to \mathbb{R}$. Obviously, $X_{i-1},\dots ,X_{i-s}$ are independent of $\{{\epsilon }_j,j\ge i\}$.

In recent years, many authors have studied the properties of estimators for the error sequence. One research interest is the error density estimator, for example, Liebscher [2] proved the law of logarithm and the law of iterated logarithm of the M-estimator in the nonlinear autoregressive processes of order p with independent errors. Cheng and Sun [3] studied the goodness-of-fit test of the errors in the nonlinear autoregressive processes of order p with independent and identical distributed errors. Fu and Yang [4] obtained the asymptotic normality of error kernel density estimators in the pth-order nonlinear autoregressive processes with independent and identical distributed errors. Cheng [5] obtained the asymptotic distribution of the maximum of a suitably normalized deviation of the density estimator from the expectation of the kernel error density. Li [6] established the asymptotic normality of the $L_p$-norms of error density estimators in the pth-order nonlinear autoregressive processes with independent and identical distributed errors. Kim et al. [7] considered the goodness-of-fit test of the errors in the nonlinear autoregressive processes of order p with a stationary $\alpha$-mixing error. Cheng [8] considered the uniform strong consistency of the classical Glivenko–Cantelli Theorem for the residual-based empirical error in the pth-order nonlinear autoregressive processes with independent and identical distributed errors. Liu and Zhang [9] established the law of the iterated logarithm for error density estimators in the pth-order nonlinear autoregressive processes with independent and identical distributed errors.

The other research interest is the error variance estimator. Cheng [10] obtained the consistency and asymptotic normality of the variance estimator in the pth-order nonlinear autoregressive processes with independent and identical distributed errors. As we know, there are few results about the error variance estimators except for Cheng [10], and there are no results for the almost sure central limit theorem for the error variance estimator, and therefore, we will study the almost sure central limit theorem for the error variance estimator in this paper.

The almost sure central limit theorem (ASCLT, for short) has been first introduced independently by Brosamler [11] and Schatte [12]. Since then many interesting results have been discovered in this field. The classical ASCLT for a sequence $\{X,X_n;n\ge 1\}$ of i.i.d. random variables with zero means states that when $\mathrm{Var}(X)={\sigma }^2$,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\frac{S_k}{\sqrt{k}\sigma }\le x\right\}=\Phi (x)a.s.\end{array}$$

(2)

for all $x\in \mathbb{R}$ with the logarithmic averages $d_k=1/k$ and $D_n={\sum }_{k=1}^n{d}_k$, $S_k={\sum }_{j=1}^k{X}_j$. However, logarithmic averaging is not the only one providing a.s. convergence for partial sums of i.i.d. random variables. Peligrad and Révész [13] showed that (2) holds with $d_k={(\log k)}^{\alpha }/k$, $\alpha >-1$. Berkes and Csáki [14] showed that (2) holds also if $d_k=\exp \left\{{(\log k)}^{\alpha }\right\}/k$, $0\le \alpha <1/2$. To compare these results, Hörmann [15], Tong et al. [16], Miao [17], Li [18], Zhang [19,20], Wu and Jiang [21], and Li and Zhang [22,23,24] showed that the a.s. limit (2) holds for any weight sequence $\left\{d_k\right\}$ satisfying a mild growth condition similar to Kolmogorov’s condition on the law of iterated logarithm.

The paper is organized as follows. In Section 1, the significance and background of research is introduced. Some assumptions and main results are stated in Section 2. Several useful lemmas are listed in Section 3. The proofs are listed in Section 4. Examples are stated in Section 5. In the sequel, we denote with $C,C_1,C_2,\cdots$ generic constants that may be different in each of its appearances. $I\left\{A\right\}$ denotes the indicator function of the set A. $\Phi (·)$ denotes the distribution function of the standard normal random variable $\mathcal{N}$.

2. Main Results

The goal of this paper is to study the properties of the estimator of the error variance ${\sigma }^2$ by means of the observations $\{X_1,X_2,\cdots ,X_n\}$ in model (1). The main difficulty is that we do not observe the error $\{{\epsilon }_1,{\epsilon }_2,\cdots ,{\epsilon }_n\}$, the structure of estimation of parameters is complex, and the residuals are unknown. We need to use Taylor’s expansion and many other techniques to deal with it. This is the greatest contribution of this paper. We will follow the following steps. Firstly, we compute an estimator $\hat{\mathit{\theta }}={({\hat{\theta }}_1,\dots ,{\hat{\theta }}_q)}^{\prime }$ of unknown parameter $\mathit{\theta }$. Secondly, based on the estimator $\hat{\mathit{\theta }}$ and model (1), we calculate the following residuals

$$\begin{array}{c}\hfill {\hat{\epsilon }}_i=X_i-r_{\hat{\mathit{\theta }}}\left(X_{i-1},\dots ,X_{i-p}\right),i=1,2,\dots ,n.\end{array}$$

(3)

Finally, using the above residuals, we estimate the error variance ${\sigma }^2$ by using the following equation

$$\begin{array}{c}\hfill {\hat{\sigma }}_n^2=\frac{1}{n}\sum _{i=1}^n{\hat{\epsilon }}_i^2.\end{array}$$

(4)

Before giving the main results, we need the following basic assumptions for model (1) which will be used throughout the paper. For $1\le i\le n$ and $1\le j\le q$, let

$$Y_{ij}\triangleq \frac{\partial }{\partial {\theta }_j}{r}_{\mathit{\theta }}(X_{i-1},\dots ,X_{i-p}),Z_{ijl}\triangleq \frac{{\partial }^2}{\partial {\theta }_j\partial {\theta }_l}{r}_{{\mathit{\theta }}^{*}}(X_{i-1},\dots ,X_{i-p}),$$

where ${\mathit{\theta }}^{*}=\mathit{\theta }+\lambda (\hat{\mathit{\theta }}-\mathit{\theta })$ for some $\lambda \in (0,1)$. By the fact that $X_{i-1},\dots ,X_{i-s}$ are independent of $\{{\epsilon }_j,j\ge i\}$, we conclude that ${\epsilon }_i$ is independent of $Y_{ij}$.

Assumption 1.

Let $\mathbb{U}\subset \Theta \subset {\mathbb{R}}^q$ be an open neighborhood of θ. For any $y\in {\mathbb{R}}^p$, $\mathit{\theta }=({\theta }_1,\dots ,{\theta }_q)\in \mathbb{U}$, $j,l=1,\dots ,q$, assume that

$$\left|\frac{\partial }{\partial {\theta }_j}{r}_{\mathit{\theta }}(y)\right|\le M_1(y),\left|\frac{{\partial }^2}{\partial {\theta }_j\partial {\theta }_l}{r}_{\mathit{\theta }}(y)\right|\le M_2(y),$$

where $E\left[M_1^4(X_{i-1},\dots ,X_{i-p})\right]<\infty$ and $E\left[M_2^4(X_{i-1},\dots ,X_{i-p})\right]<\infty$ for each $i\ge 1$.

Assumption 2.

Let $\hat{\mathit{\theta }}={({\hat{\theta }}_1,\dots ,{\hat{\theta }}_q)}^{\prime }$ be a strong consistent estimator for θ satisfying the following law of iterated logarithm

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim \sup }\sqrt{\frac{n}{\log \log n}}\left|\hat{\mathit{\theta }}-\mathit{\theta }\right|\le C,a.s.,\end{array}$$

(5)

where $|\hat{\mathit{\theta }}-\mathit{\theta }|=\sqrt{{\displaystyle \sum _{j=1}^q}{({\hat{\theta }}_j-{\theta }_j)}^2}$ and C is a positive constant.

Remark 1.

By Corollary 2.2 of Klimko and Nelson [25], we know that the least square estimator for the stochastic process under some suitable conditions satisfies (5). For the first-order autoregressive progresses, Wang et al. [26] proved that the least square estimator of the unknown parameters meets (5). For smooth threshold autoregressive progresses, Chan and Tong [27] obtained the conditional least square estimators of the unknown parameters that satisfy (5). For general nonlinear autoregressive progresses of order p, Liebscher [2] established M-estimators for the unknown parameters that satisfy (5), Yao [28] obtained (5) for least square estimators of nonlinear autoregressive progresses.

Now, we will state the main result for the almost sure central limit theorem of the error variance estimator ${\hat{\sigma }}^2$.

Theorem 1.

Suppose that $\left\{d_k\right\}$ is a sequence of positive numbers satisfying the following conditions:

(C1) 

${\lim \sup }_{k\to \infty }k{d}_k{(\log D_k)}^{\rho }/D_k<\infty$ for some $\rho >1$, where $D_n={\sum }_{k=1}^n{d}_k$.

(C2) 

$D_n\to \infty$, $D_n=o(n^{ϵ})$, for any $ϵ>0$.

For model (1), under the Assumptions 1 and 2, if $E{\epsilon }_1^4<\infty$, for all $x\in \mathbb{R}$, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\frac{\sqrt{k}}{\sqrt{Var({\epsilon }_1^2)}}\left({\hat{\sigma }}_k^2-{\sigma }^2\right)\le x\right\}=\Phi (x)a.s.\end{array}$$

(6)

Corollary 1.

Let $c_n>0$ with $c_n\uparrow \infty$ and $\underset{n\to \infty }{\lim }\frac{c_{n+1}}{c_n}=1$ and $\frac{k}{l}\le {(\frac{c_k}{c_l})}^{\gamma },k<l$ for some constant $\gamma >0$. Denote

$$d_k=\log \frac{c_{k+1}}{c_k}\exp \left({\log }^{\beta }{c}_k\right),D_n=\sum _{k=1}^n{d}_k,0\le \beta <1/2.$$

Then, under the assumptions of Theorem 1, (6) also holds.

Remark 2.

If the conditions (C1) and (C2) of Theorem 1 is satisfied for some sequence $\left\{D_n\right\}$, then it is also satisfied for any other sequence $D_n^{*}=\Psi (D_n)$, provided that $\Psi :{\mathbb{R}}^{+}\to {\mathbb{R}}^{+}$ is differentiable, ${\Psi }^{\prime }(x)=O(\Psi (x)/x)$ and $\log {\Psi }^{\prime }(x)$ is uniformly continuous on $(B,\infty )$ for some $B>0$. Typical examples are $\Psi (x)=x^{\gamma }$, $\Psi (x)={(\log x)}^{\gamma }$, $\Psi (x)={(\log \log x)}^{\gamma }$ with some suitable $\gamma >0$.

Remark 3.

It is easy to show that $d_k=l(k)/k$, where $l(x)$ is slowly varying at infinity and $D_n\to \infty$, satisfies the conditions $(C1)$ and $(C2)$. So typical examples including $d_k=1/k$; $d_k={\log }^{\theta }k/k,\theta >-1$; $d_k=\exp ({\log }^{\gamma }k)/k,0\le \gamma <1/2,1<\rho <(1-\gamma )/\gamma$.

3. Preliminary Lemmas

Some useful lemmas which are needed to prove the main result are given in the following section.

Lemma 1

(Hall and Heyde [29], Theorem 2.11, P.23). Let $X_1=S_1$ and $X_i=S_i-S_{i-1}$, $2\le i\le n$ denote the differences of the sequences $\{S_i,1\le i\le n\}$. If $\{S_i,{\mathcal{F}}_i,1\le i\le n\}$ is a martingale and $p>0$, then there exists constant C depending only on p such that

$$\begin{array}{c}\hfill E\left(\underset{1\le i\le n}{\max }{\left|S_i\right|}^p\right)\le C\left\{E\left[{\left(\sum _{i=1}^{n}E(X_i^2|{\mathcal{F}}_{i-1})\right)}^{p/2}\right]+E\left(\underset{1\le i\le n}{\max }{\left|X_i\right|}^p\right)\right\}.\end{array}$$

Lemma 2.

For $1\le i\le n$, $1\le j$, $l\le q$, then for any $2\le t\le 4$, one can obtain

$$\begin{array}{cc}\hfill & E{\left|Y_{ij}\right|}^t\le E{M}_1^t(X_{i-1},\dots ,X_{i-p})\le {\left(E{M}_1^4(X_{i-1},\dots ,X_{i-p})\right)}^{t/4}<\infty ,\hfill \\ \hfill & E{\left|Z_{ijl}\right|}^t\le E{M}_2^t(X_{i-1},\dots ,X_{i-p})\le {\left(E{M}_2^4(X_{i-1},\dots ,X_{i-p})\right)}^{t/4}<\infty .\hfill \end{array}$$

Proof. 

The proof of Lemma 2 is obvious by Assumption 1 and the Hölder inequality. □

Lemma 3.

Assume that $\{G_n,n\ge 1\}$ is a sequence of random variables satisfying the ASCLT with the weight $\left\{d_k\right\}$ defined as in Theorem 1, that is

$$\begin{array}{c}\hfill \forall x\in \mathbb{R},\underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\{G_k\le x\}=\Phi (x)a.s.\end{array}$$

Let $\{R_n,n\ge 1\}$ be a sequence of random variables converging almost surely to zero. Then, $\{G_n+R_n,n\ge 1\}$ also satisfies the ASCLT. That is

$$\begin{array}{c}\hfill \forall x\in \mathbb{R},\underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\{G_k+R_k\le x\}=\Phi (x)a.s.\end{array}$$

Proof. 

For fixed $x\in \mathbb{R}$ and $\eta >0$, recall that $\{G_n,n\ge 1\}$ satisfies the ASCLT, then we have

$$\begin{array}{c}\hfill T_{n,\eta }:=\left|\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\{G_k\le x+\eta \}-\Phi (x+\eta )\right|\to 0,a.s.\end{array}$$

and

$$\begin{array}{c}\hfill W_{n,\eta }:=\left|\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\{G_k\le x-\eta \}-\Phi (x-\eta )\right|\to 0,a.s.\end{array}$$

Remark that

$$\begin{array}{c}\hfill \{G_n+R_n\le x\}\subset \{G_n\le x+\eta \}\cup \left\{\right|R_n|>\eta \},\end{array}$$

$$\begin{array}{c}\hfill \{G_n\le x-\eta \}\subset \{G_n+R_n\le x\}\cup \left\{\right|R_n|>\eta \}.\end{array}$$

Then, we can conclude that

$$\begin{array}{cc}\hfill & \frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\{G_k+R_k\le x\}-\Phi (x)\hfill \\ \hfill \le & \frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\{G_k\le x+\eta \}-\Phi (x+\eta )+\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\right|R_k|>\eta \}+|\Phi (x+\eta )-\Phi (x)|\hfill \\ \hfill \le & T_{n,\eta }+\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\right|R_k|>\eta \}+{\int }_x^{x+\eta }\frac{1}{\sqrt{2\pi }}{e}^{-\frac{t^2}{2}}dt\hfill \\ \hfill \le & T_{n,\eta }+\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\right|R_k|>\eta \}+\frac{\eta }{\sqrt{2\pi }},\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill & \frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\{G_k+R_k\le x\}-\Phi (x)\hfill \\ \hfill \ge & \frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\{G_k\le x-\eta \}-\Phi (x-\eta )-\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\right|R_k|>\eta \}+\Phi (x-\eta )-\Phi (x)\hfill \\ \hfill \ge & -W_{n,\eta }-\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\right|R_k|>\eta \}-\frac{\eta }{\sqrt{2\pi }}.\hfill \end{array}$$

Noting that $\{R_n,n\ge 1\}$ is a sequence of random variables converging almost surely to zero and the arbitrariness of $\eta$, the desired conclusion follows from above discussion. □

Lemma 4

(Zhang [30], Lemma 2.10, P.391). Let $\{{\zeta }_n,n\ge 1\}$ be a sequence of uniformly bounded random variables and $\left\{d_n\right\}$, $\left\{D_n\right\}$ be defined as in Theorem 1. If there exist constants $C>0$ and $\delta >0$ and a sequence of positive numbers $\{a(k)\}$ such that ${\sum }_{n=1}^{\infty }a(2^n)<\infty$ and

$$E|{\zeta }_k{\zeta }_j|\le C({(k/j)}^{\delta }+a(k)),\mathrm{for}j/k>b_n={(\log D_n)}^{\rho /\delta },$$

then

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_k{\zeta }_k=0a.s.\end{array}$$

Lemma 5.

Let $\{{\epsilon }_i,i\ge 1\}$ be a sequence of independent and identically distributed random variables with mean zero, finite variance ${\sigma }^2$ and $E{\epsilon }_1^4<\infty$. Let $\left\{d_n\right\}$, $\left\{D_n\right\}$ be defined as in Theorem 1. Then for all $x\in \mathbb{R}$,

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\frac{1}{\sqrt{kVar({\epsilon }_1^2)}}\sum _{i=1}^k\left({\epsilon }_i^2-{\sigma }^2\right)\le x\right\}=\Phi (x)a.s.\end{array}$$

(7)

Proof. 

Denote $T_k={\sum }_{i=1}^k\left({\epsilon }_i^2-{\sigma }^2\right)$. Suppose that f is a bounded Lipschitz function. By classical central limit theorem, we have

$$\begin{array}{c}\hfill Ef\left(\frac{T_k}{\sqrt{kVar({\epsilon }_1^2)}}\right)\to Ef(\mathcal{N})\mathrm{as}k\to \infty .\end{array}$$

By the conclusions in Section 2 of Peligrad and Shao [31] and Theorem 7.1 of Billingsley [32], we know that (7) is equivalent to

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}f\left(\frac{T_k}{\sqrt{kVar({\epsilon }_1^2)}}\right)=Ef(\mathcal{N}),a.s.\end{array}$$

Hence, to prove (7), it suffices to show that

$$\begin{array}{c}\hfill \frac{1}{D_n}\sum _{k=1}^n{d}_k\left[f\left(\frac{T_k}{\sqrt{kVar({\epsilon }_1^2)}}\right)-Ef\left(\frac{T_k}{\sqrt{kVar({\epsilon }_1^2)}}\right)\right]\to 0,a.s.n\to \infty .\end{array}$$

(8)

For convenience, let $W_k=f(\frac{T_k}{\sqrt{kVar({\epsilon }_1^2)}})-Ef(\frac{T_k}{\sqrt{kVar({\epsilon }_1^2)}})$. Notice that $\{{\epsilon }_i,i\in \mathbb{Z}\}$ are independent, both f and $f^{\prime }$ are bounded, then we conclude that for $1\le k<j\le n$,

$$\begin{array}{ccc}\hfill & & |E{W}_k{W}_j|=\left|\mathrm{Cov}\left(f\left(\frac{T_k}{\sqrt{kVar({\epsilon }_1^2)}}\right),f\left(\frac{T_j}{\sqrt{jVar({\epsilon }_1^2)}}\right)\right)\right|\hfill \\ \hfill & =& \left|\mathrm{Cov}\left(f\left(\frac{T_k}{\sqrt{kVar({\epsilon }_1^2)}}\right),f\left(\frac{T_j}{\sqrt{jVar({\epsilon }_1^2)}}\right)-f\left(\frac{T_j-T_k}{\sqrt{jVar({\epsilon }_1^2)}}\right)\right)\right|\hfill \\ \hfill & \le & C_{1}E\left|f\left(\frac{T_j}{\sqrt{jVar({\epsilon }_1^2)}}\right)-f\left(\frac{T_j-T_k}{\sqrt{jVar({\epsilon }_1^2)}}\right)\right|\hfill \\ \hfill & \le & C_2\frac{E|T_k|}{\sqrt{jVar({\epsilon }_1^2)}}\le C_3\frac{{(E{T}_k^2)}^{1/2}}{\sqrt{jVar({\epsilon }_1^2)}}\hfill \\ & \le & C_4\frac{\sqrt{kVar({\epsilon }_1^2)}}{\sqrt{jVar({\epsilon }_1^2)}}\le C_5{(\frac{k}{j})}^{1/2},\hfill \end{array}$$

then by Lemma 4 with $\delta =1/2$ and $a(k)\equiv 0$, (8) holds, and therefore, the proof of (7) is completed. □

Lemma 6.

Under the assumptions of Theorem 1, for any $1\le j\le q$, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim \sup }\frac{\log \log n}{n^{3/2}}\sum _{i=1}^n{Y}_{ij}^2=0a.s.\end{array}$$

Proof. 

Let $n_k=\left[k^{\alpha }\right]$, $\alpha >2$. By Lemma 2 and the Markov inequality, for any $ϵ>0$, it is easy to known that

$$\begin{array}{ccc}& & \sum _{k=1}^{\infty }P\left(\frac{\log \log n_{k+1}}{n_k^{3/2}}\sum _{i=1}^{n_k}{Y}_{ij}^2>ϵ\right)\hfill \\ & \le & C_1\sum _{k=1}^{\infty }\frac{\log \log n_{k+1}}{n_k^{3/2}}\sum _{i=1}^{n_k}E{Y}_{ij}^2\hfill \\ & \le & C_2\sum _{k=1}^{\infty }\frac{\log \log n_{k+1}}{n_k^{3/2}}·n_k\hfill \\ & \le & C_3\sum _{k=1}^{\infty }\frac{\log \log (k+1)}{k^{\alpha /2}}<\infty .\hfill \end{array}$$

Then by the Borel–Cantelli lemma, we obtain

$$\begin{array}{c}\hfill \frac{\log \log n_{k+1}}{n_k^{3/2}}\sum _{i=1}^{n_k}{Y}_{ij}^2\to 0a.s.ask\to \infty .\end{array}$$

(9)

Similarly, by Lemma 2 and the Markov inequality, for any $ϵ>0$, one can obtain

$$\begin{array}{ccc}& & \sum _{k=1}^{\infty }P\left(\frac{\log \log n_{k+1}}{n_k^{3/2}}\underset{n_k<n\le n_{k+1}}{\max }\sum _{i=n_k+1}^n{Y}_{ij}^2>ϵ\right)\hfill \\ & \le & C_1\sum _{k=1}^{\infty }\frac{\log \log n_{k+1}}{n_k^{3/2}}E\left[\underset{n_k<n\le n_{k+1}}{\max }\sum _{i=n_k+1}^n{Y}_{ij}^2\right]\hfill \\ & \le & C_2\sum _{k=1}^{\infty }\frac{\log \log n_{k+1}}{n_k^{3/2}}\sum _{i=n_k+1}^{n_{k+1}}E{Y}_{ij}^2\hfill \\ & \le & C_3\sum _{k=1}^{\infty }\frac{\log \log n_{k+1}}{n_k^{3/2}}·[n_{k+1}-n_k]\hfill \\ & \le & C_4\sum _{k=1}^{\infty }\frac{\log \log (k+1)}{k^{3\alpha /2}}·\left[{(k+1)}^{\alpha }-k^{\alpha }\right]\hfill \\ & \le & C_5\sum _{k=1}^{\infty }\frac{\log \log (k+1)}{k^{\alpha /2+1}}<\infty .\hfill \end{array}$$

By the Borel–Cantelli lemma, it follows that

$$\begin{array}{c}\hfill \frac{\log \log n_{k+1}}{n_k^{3/2}}\underset{n_k<n\le n_{k+1}}{\max }\sum _{i=n_k+1}^n{Y}_{ij}^2\to 0a.s.ask\to \infty .\end{array}$$

(10)

Then combining (9) with (10), for $n_k<n\le n_{k+1}$, one can obtain

$$\begin{array}{ccc}& & \underset{n\to \infty }{\lim \sup }\frac{\log \log n}{n^{3/2}}\sum _{i=1}^n{Y}_{ij}^2\hfill \\ & \le & \underset{k\to \infty }{\lim \sup }\frac{\log \log n_{k+1}}{n_k^{3/2}}\sum _{i=1}^{n_k}{Y}_{ij}^2+\underset{k\to \infty }{\lim \sup }\frac{\log \log n_{k+1}}{n_k^{3/2}}\underset{n_k<n\le n_{k+1}}{\max }\sum _{i=n_k+1}^n{Y}_{ij}^2\hfill \\ & \to & 0a.s.ask\to \infty .\hfill \end{array}$$

Thus, the proof of Lemma 6 is completed. □

Lemma 7.

Under the assumptions of Theorem 1, for any $1\le j,l\le q$, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim \sup }\frac{{(\log \log n)}^2}{n^{5/2}}\sum _{i=1}^n{Z}_{ijl}^2=0a.s.\end{array}$$

Proof. 

By Lemma 2 and the Markov inequality, for any $ϵ>0$, it is easy to see that

$$\begin{array}{ccc}& & \sum _{n=1}^{\infty }P\left(\frac{{(\log \log n)}^2}{n^{5/2}}\sum _{i=1}^n{Z}_{ijl}^2>ϵ\right)\hfill \\ & \le & C_1\sum _{n=1}^{\infty }\frac{{(\log \log n)}^2}{n^{5/2}}\sum _{i=1}^{n}E{Z}_{ijl}^2\hfill \\ & \le & C_2\sum _{n=1}^{\infty }\frac{{(\log \log n)}^2}{n^{5/2}}n\hfill \\ & \le & C_3\sum _{n=1}^{\infty }\frac{{(\log \log n)}^2}{n^{3/2}}<\infty .\hfill \end{array}$$

By the Borel–Cantelli lemma, one can obtain

$$\begin{array}{c}\hfill \frac{{(\log \log n)}^2}{n^{5/2}}\sum _{i=1}^n{Z}_{ijl}^2\to 0a.s.asn\to \infty .\end{array}$$

The proof of Lemma 7 is completed. □

Lemma 8.

Under the assumptions of Theorem 1, for any $1\le j\le q$, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim \sup }\frac{{(\log \log n)}^{1/2}}{n}\left|\sum _{i=1}^n{Y}_{ij}{\epsilon }_i\right|=0a.s.\end{array}$$

Proof. 

Let

$$\begin{array}{c}\hfill Y_m=\sum _{i=1}^m{Y}_{ij}{\epsilon }_i,1\le m\le n.\end{array}$$

Let ${\mathcal{F}}_m$ be the $\sigma$-algebra generated by the random variables $\left\{{\epsilon }_i,1\le i\le m\right\}$. By the fact that $Y_{ij}$ and ${\epsilon }_i$ are independent, it is easy to compute that the process $\left\{Y_m,{\mathcal{F}}_m,1\le m\le n\right\}$ is a martingale. By Lemmas 1 and 2, for some $2<t<4$, we know

$$\begin{array}{cc}\hfill E|Y_n{|}^t=& E{\left|\sum _{i=1}^n{Y}_{ij}{\epsilon }_i\right|}^t\hfill \\ \hfill \le & C_{1}E\left[{\left(\sum _{i=1}^{n}E\left(Y_{ij}^2{\epsilon }_i^2|{\mathcal{F}}_{i-1}\right)\right)}^{t/2}\right]+C_{2}E\left(\underset{1\le i\le n}{\max }{\left|Y_{ij}{\epsilon }_i\right|}^t\right)\hfill \\ \hfill \le & C_{3}E\left[{\left(\sum _{i=1}^{n}E{Y}_{ij}^2\right)}^{t/2}\right]·{\left[E{\epsilon }_1^2\right]}^{t/2}+C_4\sum _{i=1}^{n}E{\left|Y_{ij}\right|}^t·E{\left|{\epsilon }_1\right|}^t\hfill \\ \hfill \le & C_5{n}^{t/2}+C_{6}n\le C_7{n}^{t/2}.\hfill \end{array}$$

(11)

By the Markov inequality and (11), for any $ϵ>0$, it is easy to obtain

$$\begin{array}{ccc}& & \sum _{n=1}^{\infty }P\left(\frac{{(\log \log n)}^{1/2}}{n}\left|\sum _{i=1}^n{Y}_{ij}{\epsilon }_i\right|>ϵ\right)\hfill \\ & \le & C_1\sum _{n=1}^{\infty }\frac{{(\log \log n)}^{t/2}}{n^t}E{\left|\sum _{i=1}^n{Y}_{ij}{\epsilon }_i\right|}^t\hfill \\ & \le & C_2\sum _{n=1}^{\infty }\frac{{(\log \log n)}^{t/2}}{n^t}{n}^{t/2}\hfill \\ & \le & C_3\sum _{n=1}^{\infty }\frac{{(\log \log n)}^{t/2}}{n^{t/2}}<\infty .\hfill \end{array}$$

By the Borel–Cantelli lemma, we can obtain

$$\begin{array}{c}\hfill \frac{{(\log \log n)}^{1/2}}{n}\left|\sum _{i=1}^n{Y}_{ij}{\epsilon }_i\right|\to 0a.s.asn\to \infty .\end{array}$$

The proof of Lemma 8 is completed. □

Lemma 9.

Under the assumptions of Theorem 1, for any $1\le j,l\le q$, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim \sup }\frac{\log \log n}{n^{3/2}}\left|\sum _{i=1}^n{Z}_{ijl}{\epsilon }_i\right|=0a.s.\end{array}$$

Proof. 

Let $n_k=\left[k^{\alpha }\right]$, $\alpha >2$. By Lemma 2, the Markov inequality, $C_r$ inequality and Cauchy–Schwarz inequality, for any $ϵ>0$, it is easy to see that

$$\begin{array}{ccc}& & \sum _{k=1}^{\infty }P\left(\frac{\log \log n_{k+1}}{n_k^{3/2}}\left|\sum _{i=1}^{n_k}{Z}_{ijl}{\epsilon }_i\right|>ϵ\right)\hfill \\ & \le & C_1\sum _{k=1}^{\infty }\frac{{(\log \log n_{k+1})}^2}{n_k^3}E{\left(\sum _{i=1}^{n_k}{Z}_{ijl}{\epsilon }_i\right)}^2\hfill \\ & \le & C_2\sum _{k=1}^{\infty }\frac{{(\log \log n_{k+1})}^2}{n_k^3}·n_k\sum _{i=1}^{n_k}E{Z}_{ijl}^2{\epsilon }_i^2\hfill \\ & \le & C_3\sum _{k=1}^{\infty }\frac{{(\log \log n_{k+1})}^2}{n_k^2}\sum _{i=1}^{n_k}{(E{Z}_{ijl}^4)}^{1/2}{(E{\epsilon }_i^4)}^{1/2}\hfill \\ & \le & C_4\sum _{k=1}^{\infty }\frac{{(\log \log n_{k+1})}^2}{n_k}\le C_5\sum _{k=1}^{\infty }\frac{{(\log \log (k+1))}^2}{k^{\alpha }}<\infty .\hfill \end{array}$$

Then by the Borel–Cantelli lemma, we obtain

$$\begin{array}{c}\hfill \frac{\log \log n_{k+1}}{n_k^{3/2}}\left|\sum _{i=1}^{n_k}{Z}_{ijl}{\epsilon }_i\right|\to 0a.s.ask\to \infty .\end{array}$$

(12)

Similarly, By Lemma 2 and the Markov inequality and $C_r$ inequality, for any $ϵ>0$, one can obtain

$$\begin{array}{ccc}& & \sum _{k=1}^{\infty }P\left(\frac{\log \log n_{k+1}}{n_k^{3/2}}\underset{n_k<n\le n_{k+1}}{\max }\left|\sum _{i=n_k+1}^n{Z}_{ijl}{\epsilon }_i\right|>ϵ\right)\hfill \\ & \le & C_1\sum _{k=1}^{\infty }\frac{{(\log \log n_{k+1})}^2}{n_k^3}E{\left(\underset{n_k<n\le n_{k+1}}{\max }\left|\sum _{i=n_k+1}^n{Z}_{ijl}{\epsilon }_i\right|\right)}^2\hfill \\ & \le & C_2\sum _{k=1}^{\infty }\frac{{(\log \log n_{k+1})}^2}{n_k^3}E{\left(\sum _{i=n_k+1}^{n_{k+1}}\left|Z_{ijl}{\epsilon }_i\right|\right)}^2\hfill \\ & \le & C_3\sum _{k=1}^{\infty }\frac{{(\log \log n_{k+1})}^2}{n_k^3}·(n_{k+1}-n_k)\sum _{i=n_k+1}^{n_{k+1}}E\left[Z_{ijl}^2{\epsilon }_i^2\right]\hfill \\ & \le & C_4\sum _{k=1}^{\infty }\frac{{(\log \log n_{k+1})}^2}{n_k^3}·(n_{k+1}-n_k)\sum _{i=n_k+1}^{n_{k+1}}{(E{Z}_{ijl}^4)}^{1/2}{(E{\epsilon }_i^4)}^{1/2}\hfill \\ & \le & C_5\sum _{k=1}^{\infty }\frac{{(\log \log n_{k+1})}^2}{n_k^3}·{(n_{k+1}-n_k)}^2\hfill \\ & \le & C_6\sum _{k=1}^{\infty }\frac{{(\log \log (k+1))}^2}{k^{3\alpha }}·k^{2(\alpha -1)}\le C_7\frac{{(\log \log (k+1))}^2}{k^{\alpha +2}}<\infty .\hfill \end{array}$$

By the Borel–Cantelli lemma, it follows that

$$\begin{array}{c}\hfill \frac{\log \log n_{k+1}}{n_k^{3/2}}\underset{n_k<n\le n_{k+1}}{\max }\left|\sum _{i=n_k+1}^n{Z}_{ijl}{\epsilon }_i\right|\to 0a.s.ask\to \infty .\end{array}$$

(13)

Then combining (12) with (13), for $n_k<n\le n_{k+1}$, one can obtain

$$\begin{array}{ccc}& & \underset{n\to \infty }{\lim \sup }\frac{\log \log n}{n^{3/2}}\left|\sum _{i=1}^n{Z}_{ijl}{\epsilon }_i\right|\hfill \\ & \le & \underset{k\to \infty }{\lim \sup }\frac{\log \log n_{k+1}}{n_k^{3/2}}\left|\sum _{i=1}^{n_k}{Z}_{ijl}{\epsilon }_i\right|+\underset{k\to \infty }{\lim \sup }\frac{\log \log n_{k+1}}{n_k^{3/2}}\underset{n_k<n\le n_{k+1}}{\max }\left|\sum _{i=n_k+1}^n{Z}_{ijl}{\epsilon }_i\right|\hfill \\ & \to & 0a.s.ask\to \infty .\hfill \end{array}$$

The proof of Lemma 9 is completed. □

Lemma 10.

Under the assumptions of Theorem 1, for any $1\le j,l,k\le q$, we have

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim \sup }\frac{{(\log \log n)}^{3/2}}{n^2}\left|\sum _{i=1}^n{Y}_{ij}{Z}_{ilk}\right|=0a.s.\end{array}$$

Proof. 

By Lemma 2, the Markov inequality, $C_r$ inequality and Cauchy–Schwarz inequality, for any $ϵ>0$, it is easy to see that

$$\begin{array}{ccc}& & \sum _{n=1}^{\infty }P\left(\frac{{(\log \log n)}^{3/2}}{n^2}\left|\sum _{i=1}^n{Y}_{ij}{Z}_{ilk}\right|>ϵ\right)\hfill \\ & \le & C_1\sum _{n=1}^{\infty }\frac{{(\log \log n)}^{3t/4}}{n^t}E{\left|\sum _{i=1}^n{Y}_{ij}{Z}_{ilk}\right|}^{t/2}\hfill \\ & \le & C_2\sum _{n=1}^{\infty }\frac{{(\log \log n)}^{3t/4}}{n^t}{n}^{t/2-1}\sum _{i=1}^{n}E\left[{\left|Y_{ij}\right|}^{t/2}{\left|Z_{ilk}\right|}^{t/2}\right]\hfill \\ & \le & C_3\sum _{n=1}^{\infty }\frac{{(\log \log n)}^{3t/4}}{n^{t/2+1}}\sum _{i=1}^n{\left(E|Y_{ij}{|}^t\right)}^{1/2}{\left(E|Z_{ilk}{|}^t\right)}^{1/2}\hfill \\ & \le & C_4\sum _{n=1}^{\infty }\frac{{(\log \log n)}^{3t/4}}{n^{t/2}}<\infty .\hfill \end{array}$$

where $2<t\le 4$ is defined in Assumption 1.

By the Borel–Cantelli lemma, we obtain

$$\begin{array}{c}\hfill \frac{{(\log \log n)}^{3/2}}{n^2}\left|\sum _{i=1}^n{Y}_{ij}{Z}_{ilk}\right|\to 0a.s.asn\to \infty .\end{array}$$

The proof of Lemma 10 is completed. □

4. Proof

Recall (1) and (3), by Taylor’s expansion expansion with the Lagrange remainder, there exists $\lambda \in (0,1)$, and ${\mathit{\theta }}^{*}=\mathit{\theta }+\lambda (\hat{\mathit{\theta }}-\mathit{\theta })$

$$\begin{array}{cc}\hfill {\hat{\epsilon }}_i=& {\epsilon }_i-[r_{\hat{\mathit{\theta }}}(X_{i-1},\dots ,X_{i-p})-r_{\mathit{\theta }}(X_{i-1},\dots ,X_{i-p})]\hfill \\ \hfill =& {\epsilon }_i-\sum _{j=1}^q({\hat{\theta }}_j-{\theta }_j)Y_{ij}-\frac{1}{2}\sum _{j=1}^q\sum _{l=1}^q({\hat{\theta }}_j-{\theta }_j)({\hat{\theta }}_l-{\theta }_l)Z_{ijl}.\hfill \end{array}$$

(14)

Then by (14), we can obtain

$$\begin{array}{cc}\hfill & \sqrt{\frac{n}{Var({\epsilon }_1^2)}}\left({\hat{\sigma }}_n^2-{\sigma }^2\right)\hfill \\ \hfill =& \sqrt{\frac{n}{Var({\epsilon }_1^2)}}\left(\frac{1}{n}\sum _{i=1}^n{\hat{\epsilon }}_i^2-\frac{1}{n}\sum _{i=1}^n{\epsilon }_i^2+\frac{1}{n}\sum _{i=1}^n{\epsilon }_i^2-\frac{1}{n}\sum _{i=1}^{n}E{\epsilon }_i^2\right)\hfill \\ \hfill =& \sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n({\hat{\epsilon }}_i^2-{\epsilon }_i^2)\right]+\sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n({\epsilon }_i^2-E{\epsilon }_i^2)\right]\hfill \\ \hfill =& \sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n{\left(\sum _{j=1}^q({\hat{\theta }}_j-{\theta }_j)Y_{ij}\right)}^2\right]\hfill \\ \hfill & +\sqrt{\frac{1}{16nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n{\left(\sum _{j=1}^q\sum _{l=1}^q({\hat{\theta }}_j-{\theta }_j)({\hat{\theta }}_l-{\theta }_l)Z_{ijl}\right)}^2\right]\hfill \\ \hfill & -\sqrt{\frac{4}{nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n\sum _{j=1}^q({\hat{\theta }}_j-{\theta }_j)Y_{ij}{\epsilon }_i\right]\hfill \\ \hfill & -\sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n\sum _{j=1}^q\sum _{l=1}^q({\hat{\theta }}_j-{\theta }_j)({\hat{\theta }}_l-{\theta }_l)Z_{ijl}{\epsilon }_i\right]\hfill \\ \hfill & +\sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n\sum _{j=1}^q\sum _{l=1}^q\sum _{k=1}^q({\hat{\theta }}_j-{\theta }_j)({\hat{\theta }}_l-{\theta }_l)({\hat{\theta }}_k-{\theta }_k)Y_{ij}{Z}_{ilk}\right]\hfill \\ \hfill & +\sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n({\epsilon }_i^2-E{\epsilon }_i^2)\right]\hfill \\ \hfill =:& I_{n1}+I_{n2}-I_{n3}-I_{n4}+I_{n5}+I_{n6}.\hfill \end{array}$$

(15)

Recall the elementary inequality

$$\begin{array}{c}\hfill {\left(\sum _{i=1}^q{a}_i{b}_i\right)}^2\le \left(\sum _{i=1}^q{a}_i^2\right)\left(\sum _{i=1}^q{b}_i^2\right).\end{array}$$

(16)

For $I_{n1}$, by (5) and (16) and Lemma 6, it is easy to know that

$$\begin{array}{cc}\hfill I_{n1}=& \sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n{\left(\sum _{j=1}^q({\hat{\theta }}_j-{\theta }_j)Y_{ij}\right)}^2\right]\hfill \\ \hfill \le & \sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2·\sum _{j=1}^q\sum _{i=1}^n{Y}_{ij}^2\hfill \\ \hfill =& \frac{1}{\sqrt{Var({\epsilon }_1^2)}}\frac{n}{\log \log n}\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2·\sum _{j=1}^q\frac{\log \log n}{n^{3/2}}\sum _{i=1}^n{Y}_{ij}^2\hfill \\ \hfill \to & 0a.s.asn\to \infty .\hfill \end{array}$$

(17)

For $I_{n2}$, by (5) and (16) and Lemma 7, one can obtain

$$\begin{array}{cc}\hfill I_{n2}=& \sqrt{\frac{1}{16nVar({\epsilon }_1^2)}}\left[\sum _{i=1}^n{\left(\sum _{j=1}^q\sum _{l=1}^q({\hat{\theta }}_j-{\theta }_j)({\hat{\theta }}_l-{\theta }_l)Z_{ijl}\right)}^2\right]\hfill \\ \hfill \le & \sqrt{\frac{1}{16nVar({\epsilon }_1^2)}}\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2·\sum _{l=1}^q{({\hat{\theta }}_l-{\theta }_l)}^2·\sum _{j=1}^q\sum _{l=1}^q\sum _{i=1}^n{Z}_{ijl}^2\hfill \\ \hfill =& \frac{1}{4\sqrt{Var({\epsilon }_1^2)}}{\left(\frac{n}{\log \log n}\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2\right)}^2·\sum _{j=1}^q\sum _{l=1}^q\frac{{(\log \log n)}^2}{n^{5/2}}\sum _{i=1}^n{Z}_{ijl}^2\hfill \\ \hfill \to & 0a.s.asn\to \infty .\hfill \end{array}$$

(18)

For $I_{n3}$, by (5) and (16) and Lemma 8, one can obtain

$$\begin{array}{cc}\hfill I_{n3}=& \sqrt{\frac{4}{nVar({\epsilon }_1^2)}}\left[\sum _{j=1}^q({\hat{\theta }}_j-{\theta }_j)·\sum _{i=1}^n{Y}_{ij}{\epsilon }_i\right]\hfill \\ \hfill \le & \sqrt{\frac{4}{nVar({\epsilon }_1^2)}}{\left[\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2·\sum _{j=1}^q{\left(\sum _{i=1}^n{Y}_{ij}{\epsilon }_i\right)}^2\right]}^{1/2}\hfill \\ \hfill =& \frac{2}{\sqrt{Var({\epsilon }_1^2)}}{\left[\frac{n}{\log \log n}\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2·\sum _{j=1}^q{\left(\frac{{(\log \log n)}^{1/2}}{n}\sum _{i=1}^n{Y}_{ij}{\epsilon }_i\right)}^2\right]}^{1/2}\hfill \\ \hfill \to & 0a.s.asn\to \infty .\hfill \end{array}$$

(19)

For $I_{n4}$, by (5) and (16) and Lemma 9, we have

$$\begin{array}{cc}\hfill I_{n4}=& \sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\left[\sum _{j=1}^q\sum _{l=1}^q({\hat{\theta }}_j-{\theta }_j)({\hat{\theta }}_l-{\theta }_l)\sum _{i=1}^n{Z}_{ijl}{\epsilon }_i\right]\hfill \\ \hfill \le & \sqrt{\frac{1}{nVar({\epsilon }_1^2)}}{\left[\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2·\sum _{l=1}^q{({\hat{\theta }}_l-{\theta }_l)}^2·\sum _{j=1}^q\sum _{l=1}^q{\left(\sum _{i=1}^n{Z}_{ijl}{\epsilon }_i\right)}^2\right]}^{1/2}\hfill \\ \hfill =& \frac{1}{\sqrt{Var({\epsilon }_1^2)}}{\left[{\left(\frac{n}{\log \log n}\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2\right)}^2·\sum _{j=1}^q\sum _{l=1}^q{\left(\frac{\log \log n}{n^{3/2}}\sum _{i=1}^n{Z}_{ijl}{\epsilon }_i\right)}^2\right]}^{1/2}\hfill \\ \hfill \to & 0a.s.asn\to \infty .\hfill \end{array}$$

(20)

For $I_{n5}$, by (5) and (16) and Lemma 10, we know

$$\begin{array}{cc}\hfill I_{n5}=& \sqrt{\frac{1}{nVar({\epsilon }_1^2)}}\left[\sum _{j=1}^q\sum _{l=1}^q\sum _{k=1}^q({\hat{\theta }}_j-{\theta }_j)({\hat{\theta }}_l-{\theta }_l)({\hat{\theta }}_k-{\theta }_k)\sum _{i=1}^n{Y}_{ij}{Z}_{ilk}\right]\hfill \\ \hfill \le & \sqrt{\frac{1}{nVar({\epsilon }_1^2)}}{\left[\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2·\sum _{l=1}^q{({\hat{\theta }}_l-{\theta }_l)}^2·\sum _{k=1}^q{({\hat{\theta }}_k-{\theta }_k)}^2·\sum _{j=1}^q\sum _{l=1}^q\sum _{k=1}^q{\left(\sum _{i=1}^n{Y}_{ij}{Z}_{ilk}\right)}^2\right]}^{1/2}\hfill \\ \hfill =& \frac{1}{\sqrt{Var({\epsilon }_1^2)}}{\left[{\left(\frac{n}{\log \log n}\sum _{j=1}^q{({\hat{\theta }}_j-{\theta }_j)}^2\right)}^3·\sum _{j=1}^q\sum _{l=1}^q\sum _{k=1}^q{\left(\frac{{(\log \log n)}^{3/2}}{n^2}\sum _{i=1}^n{Y}_{ij}{Z}_{ilk}\right)}^2\right]}^{1/2}\hfill \\ \hfill \to & 0a.s.asn\to \infty .\hfill \end{array}$$

(21)

Combing (17)–(21), one can obtain

$$\begin{array}{c}\hfill I_{n1}+I_{n2}-I_{n3}-I_{n4}+I_{n5}\to 0a.s.asn\to \infty .\end{array}$$

(22)

For $I_{n6}$, by Lemma 5, it is obviously that

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\frac{1}{\sqrt{kVar({\epsilon }_1^2)}}\sum _{i=1}^k({\epsilon }_i^2-E{\epsilon }_i^2)\le x\right\}=\Phi (x)a.s.\end{array}$$

(23)

Finally, (6) follows by combining (15), (22) with (23) and Lemma 3, thus the proof of Theorem 1 is completed.

5. Examples

Some examples are given in this section to verify the almost sure central limit theorem for the error variance estimator for some special nonlinear autoregressive models. The first example is a degenerate model, that is, AR(1) progresses.

Example 1.

An AR(1) model is a family of $\left\{X_i\right\}$ of random variables such that for every $i\ge 1$

$$X_i=\theta X_{i-1}+{\epsilon }_i,$$

where $\{{\epsilon }_i,i\ge 1\}$ is a collection of i.i.d. random variables with zero mean and finite variance ${\sigma }^2$. We also assume that $E\exp \left\{\gamma \right|{\epsilon }_i{\epsilon }_j\left|\right\}<\infty$ for some $\gamma >0$ and any $i,j\ge 1$. It is obviously that $\left\{X_i\right\}$ is a stationary model under the condition $\left|\theta \right|<1$.

It is easy to check that the Assumption 1 holds naturally. For Assumption 2, by Theorem 1 of Wang et al. [26] and $E\exp \left\{\gamma \right|{\epsilon }_i{\epsilon }_j\left|\right\}<\infty$, (5) holds for the least squares estimator $\hat{\theta }$. Therefore, we have the following statement for AR(1) progression due to Theorem 1.

Theorem 2.

Suppose $\left\{d_k\right\}$ is a sequence of positive numbers satisfying conditions (C1) and (C2). For the above AR(1) model, if $E\exp \left\{\gamma \right|{\epsilon }_i{\epsilon }_j\left|\right\}<\infty$ for some $\gamma >0$ and any $i,j\ge 1$, for any $x\in \mathbb{R}$, one can obtain

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\frac{\sqrt{k}}{\sqrt{Var({\epsilon }_1^2)}}\left({\hat{\sigma }}_k^2-{\sigma }^2\right)\le x\right\}=\Phi (x)a.s.\end{array}$$

The next example concerns the self-exciting threshold autoregressive (SETAR) progresses.

Example 2.

Let $\{X_i,i\ge p\}$ be a sequence of stationary and geometrically ergodic random variable satisfying the following continuous SETAR($p,l,d$) progresses.

$$X_i=\left\{\begin{array}{c}{a}_0+\sum _{m=1}^p{a}_m{X}_{i-m}+{\epsilon }_i,\mathit{if}{X}_{i-d}\in R_1,\hfill \\ a_0+\sum _{m=1}^p{a}_j{X}_{i-m}+\sum _{k=2}^j{b}_k(X_{i-d}-r_{k-1})+{\epsilon }_i,\mathit{if}{X}_{i-d}\in R_j,j=2,\dots ,l\hfill \end{array}\right.$$

where $\left\{{\epsilon }_i\right\}$ is a collection of i.i.d. random variables with zero mean and finite variance ${\sigma }^2$, $R_1,\dots ,R_l$ are the different regions with $R_s=(r_{s-1},r_s]$ for $1\le s\le l$, and $-\infty =r_0<r_1<r_2<\cdots <r_{l-1}<r_l=+\infty$ are the thresholds. Let ${\mathit{\theta }}_0={(a_0,\dots ,a_p,b_2,\dots ,b_l,r_1,\dots ,r_{l-1})}^{\top }\in \Theta \subset {\mathbb{R}}^q$ be the true parameters of the progresses and $\mathit{\theta }={({\overline{a}}_0,\dots ,{\overline{a}}_p,{\overline{b}}_2,\dots ,{\overline{b}}_l,{\overline{r}}_1,\dots ,{\overline{r}}_{l-1})}^{\top }$, ${\tilde{\mathit{X}}}_i={(X_i,\dots ,X_{i-p+1})}^{\top }$, $q=p+2l-1$.

Condition $\mathfrak{C}$ Suppose that $\left\{{\epsilon }_i\right\}$ has the density h and the density f of $X_i$ is continuous and has a support including the interval $[r_{\min }-\eta ,r_{\max }+\eta ],\eta >0$ where $r_{\min }=\min \{{\overline{r}}_1:\mathit{\theta }\in \Theta \}$, $r_{\max }=\max \{{\overline{r}}_{l-1}:\mathit{\theta }\in \Theta \}$. There is some $\epsilon >0$ such that ${\overline{r}}_{k-1}\le {\overline{r}}_k-\epsilon$ for all $\mathit{\theta }\in \Theta$ and $k=2,\dots ,l$.

By Corollary 3.1 of Liebscher [2], under Condition $\mathfrak{C}$ and $E|{\epsilon }_i{|}^{\gamma }<\infty$, $E\left|\right|{\tilde{\mathit{X}}}_i{\left|\right|}^{\gamma }<\infty$, $\gamma >4$, the Assumption 2 holds. Therefore, we have the following result for SETAR progresses due to Theorem 1.

Theorem 3.

Suppose $\left\{d_k\right\}$ is a sequence of positive numbers satisfying conditions (C1) and (C2). For the above SETAR($p,l,d$) progresses, under Assumption 1 and Condition $\mathfrak{C}$ and $E|{\epsilon }_i{|}^{\gamma }<\infty$, $E\left|\right|{\tilde{\mathit{X}}}_i{\left|\right|}^{\gamma }<\infty$, $\gamma >4$, for any $x\in \mathbb{R}$, one can obtain

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\frac{\sqrt{k}}{\sqrt{Var({\epsilon }_1^2)}}\left({\hat{\sigma }}_k^2-{\sigma }^2\right)\le x\right\}=\Phi (x)a.s.\end{array}$$

Next, we will consider the threshold-exponential AR progresses.

Example 3.

Let $R_j,j=1,\cdots ,K$ be non-overlapping and non-empty intervals of $\mathbb{R}$ such that ${\bigcup }_j{R}_j=\mathbb{R}$. A combined threshold-exponential AR progresses is defined by

$$X_i=\sum _{j=1}^K({\alpha }_j+{\beta }_j{X}_{i-1})I\{X_{i-1}\in R_j\}+c{e}^{-\gamma X_{i-1}^2}{X}_{i-1}+{\epsilon }_i,$$

with $X_0=x_0$ and $\left\{{\epsilon }_i\right\}$ is a collection of i.i.d. random variables with zero mean. Let the true parameters ${\mathit{\theta }}_0={({\alpha }_1,\cdots ,{\alpha }_K,{\beta }_1,\cdots ,{\beta }_K,c,\gamma )}^{\top }\in \Theta \subset {\mathbb{R}}^q$ and the parameters $\mathit{\theta }={({\overline{\alpha }}_1,\cdots ,{\overline{\alpha }}_K,{\overline{\beta }}_1,\cdots ,{\overline{\beta }}_K,\overline{c},\overline{\gamma })}^{\top }$ with $q=2K+2$.

For Assumption 2, if $c\ne 0$, $\gamma >0$, $|{\beta }_j|<1$, $j=1,\cdots ,K$ and $E|{\epsilon }_i{|}^{2+\delta }<\infty$ for some $\delta >0$, by Theorem 4 of Yao [28], (5) holds for the least squares estimator $\hat{\mathit{\theta }}$. Therefore, we have the following statement for threshold-exponential AR progresses due to Theorem 1.

Theorem 4.

Suppose $\left\{d_k\right\}$ is a sequence of positive numbers satisfying conditions (C1) and (C2). For the above threshold-exponential AR progresses, if $c\ne 0$, $\gamma >0$, $|{\beta }_j|<1$, $j=1,\cdots ,K$ and $E|{\epsilon }_i{|}^{2+\delta }<\infty$ for some $\delta >0$ and $E{\epsilon }_1^4<\infty$, then under Assumption 1, for any $x\in \mathbb{R}$, one can obtain

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\frac{\sqrt{k}}{\sqrt{Var({\epsilon }_1^2)}}\left({\hat{\sigma }}_k^2-{\sigma }^2\right)\le x\right\}=\Phi (x)a.s.\end{array}$$

Next, we will consider the multilayer perceptrons progress.

Example 4.

Multilayer perceptrons progressesw have become popular in nonlinear modeling due to its universal approximation ability. Such an example is the model described below which has p input units feeding by variables $X_{i-1},\cdots ,X_{i-p}$ at time i, a hidden layer with K units and one output unit which provides the variable $X_i$

$$X_i=\sum _{j=1}^K{\alpha }_j\psi \left(\sum _{l=1}^p{\beta }_{lj}{X}_{i-l}+{\beta }_{0j}\right)+{\alpha }_0+{\epsilon }_i,$$

where $\left\{{\epsilon }_i\right\}$ is a collection of i.i.d. random variables with zero mean. Let the true parameters ${\mathit{\theta }}_0={({\alpha }_0,\cdots ,{\alpha }_K,{\beta }_{lj},0\le l\le p,1\le j\le K)}^{\top }\in \Theta \subset {\mathbb{R}}^q$ and the parameters $\mathit{\theta }={({\overline{\alpha }}_0,\cdots ,{\overline{\alpha }}_K,{\overline{\beta }}_{lj},0\le l\le p,1\le j\le K)}^{\top }$ with $q=1+K(p+1)$.

For Assumption 2 and sigmoid map $\psi (x)=tanh(x)=\frac{e^x-e^{-x}}{e^x+e^{-x}}$, if for all $\mathit{\theta }$ are different from ${\mathit{\theta }}_0$, there exists $x\in {\mathbb{R}}^p$ such that $r_{\mathit{\theta }}(x)\ne r_{{\mathit{\theta }}_0}(x)$, $E|{\epsilon }_i{|}^{6+\delta }<\infty$ for some $\delta >0$ and the matrix $I_0$ is regular, where

$$I_0=2{\int }_{{\mathbb{R}}^p}{M}_{{\mathit{\theta }}_0}(x){\mu }_{{\mathit{\theta }}_0}(dx),M_{\mathit{\theta }}(x)={(\frac{\partial r_{\mathit{\theta }}(x)}{\partial {\theta }_i}·\frac{\partial r_{\mathit{\theta }}(x)}{\partial {\theta }_j})}_{1\le i,j\le q},$$

then by Theorem 5 of Yao [28], (5) holds for the least squares estimator $\hat{\mathit{\theta }}$. Therefore, we have the following statement for multilayer perceptrons due to Theorem 1.

Theorem 5.

Suppose $\left\{d_k\right\}$ is a sequence of positive numbers satisfying conditions (C1) and (C2). For the univariate multilayer perceptrons progress with $\psi (x)=tanh(x)$, if for all θ different from ${\mathit{\theta }}_0$, there exists $x\in {\mathbb{R}}^p$ such that $r_{\mathit{\theta }}(x)\ne r_{{\mathit{\theta }}_0}(x)$, $E|{\epsilon }_i{|}^{6+\delta }<\infty$ for some $\delta >0$ and the matrix $I_0$ is regular, then under Assumption 1, for any $x\in \mathbb{R}$, one can obtain

$$\begin{array}{c}\hfill \underset{n\to \infty }{\lim }\frac{1}{D_n}\sum _{k=1}^n{d}_{k}I\left\{\frac{\sqrt{k}}{\sqrt{Var({\epsilon }_1^2)}}\left({\hat{\sigma }}_k^2-{\sigma }^2\right)\le x\right\}=\Phi (x)a.s.\end{array}$$

6. Conclusions

In this paper, using Taylor’s expansion, the Borel–Cantelli lemma and the classical almost sure central limit theorem for independent random variables, the authors establish the almost sure central limit theorem for the error variance estimator for nonlinear autoregressive progresses with independent and identical distributed errors. The results extend the almost sure central limit theorem for the error variance estimator to the nonlinear autoregressive progresses. Four examples, first-order autoregressive processes, self-exciting threshold autoregressive processes, threshold-exponential AR progresses and multilayer perceptrons progress, are given to verify the results. In the future, we will try to investigate the almost sure central limit theorem for the error variance estimator for nonlinear autoregressive progresses with dependent errors and the moderate deviation principle for the error variance estimator for nonlinear autoregressive progresses with independent errors.