Almost Sure Central Limit Theorem for Error Variance Estimator in Pth-Order Nonlinear Autoregressive Processes

: In this paper, under some suitable assumptions, using the Taylor expansion, Borel–Cantelli lemma and the almost sure central limit theorem for independent random variables, the almost sure central limit theorem for error variance estimator in the pth-order nonlinear autoregressive processes with independent and identical distributed errors was established. Four examples, first-order autoregressive processes, self-exciting threshold autoregressive processes


Introduction
Over the past twenty years, there has been an increasing interest in the nonlinear time series literature, for example, the monograph by Tong [1] represents a good account of nonlinear time series models.Compared to linear models, studying the properties of estimators in nonlinear time series models is technically more complex and difficult.In this paper, we will investigate the properties of estimators in nonlinear autoregressive processes.
Throughout this paper, we always assume that {ε i , i ∈ Z} is a sequence of independent and identically distributed random variables with mean zero, finite variance σ 2 .{X i , i ∈ Z} is a sequence of strictly stationary real random variables which satisfies nonlinear autoregressive processes of order p X i = r θ X i−1 , . . ., X i−p + ε i , (1) for some θ = θ 1 , . . ., θ q ′ ∈ Θ ⊂ R q , where r θ , θ ∈ Θ, is a family of known measurable functions from R p → R. Obviously, X i−1 , . . ., X i−s are independent of {ε j , j ≥ i}.
In recent years, many authors have studied the properties of estimators for the error sequence.One research interest is the error density estimator, for example, Liebscher [2] proved the law of logarithm and the law of iterated logarithm of the M-estimator in the nonlinear autoregressive processes of order p with independent errors.Cheng and Sun [3] studied the goodness-of-fit test of the errors in the nonlinear autoregressive processes of order p with independent and identical distributed errors.Fu and Yang [4] obtained the asymptotic normality of error kernel density estimators in the pth-order nonlinear autoregressive processes with independent and identical distributed errors.Cheng [5] obtained the asymptotic distribution of the maximum of a suitably normalized deviation of the density estimator from the expectation of the kernel error density.Li [6] established the asymptotic normality of the L p -norms of error density estimators in the pth-order nonlinear autoregressive processes with independent and identical distributed errors.Kim et al. [7] considered the goodness-of-fit test of the errors in the nonlinear autoregressive processes of order p with a stationary α-mixing error.Cheng [8] considered the uniform strong consistency of the classical Glivenko-Cantelli Theorem for the residual-based empirical error in the pth-order nonlinear autoregressive processes with independent and identical distributed errors.Liu and Zhang [9] established the law of the iterated logarithm for error density estimators in the pth-order nonlinear autoregressive processes with independent and identical distributed errors.
The other research interest is the error variance estimator.Cheng [10] obtained the consistency and asymptotic normality of the variance estimator in the pth-order nonlinear autoregressive processes with independent and identical distributed errors.As we know, there are few results about the error variance estimators except for Cheng [10], and there are no results for the almost sure central limit theorem for the error variance estimator, and therefore, we will study the almost sure central limit theorem for the error variance estimator in this paper.
The almost sure central limit theorem (ASCLT, for short) has been first introduced independently by Brosamler [11] and Schatte [12].Since then many interesting results have been discovered in this field.The classical ASCLT for a sequence {X, X n ; n ≥ 1} of i.i.d.random variables with zero means states that when Var(X) = σ 2 , for all x ∈ R with the logarithmic averages However, logarithmic averaging is not the only one providing a.s.convergence for partial sums of i.i.d.random variables.Peligrad and Révész [13] showed that (2) holds with d k = (log k) α /k, α > −1.Berkes and Csáki [14] showed that (2) holds also if To compare these results, H örmann [15], Tong et al. [16], Miao [17], Li [18], Zhang [19,20], Wu and Jiang [21], and Li and Zhang [22][23][24] showed that the a.s.limit (2) holds for any weight sequence {d k } satisfying a mild growth condition similar to Kolmogorov's condition on the law of iterated logarithm.
The paper is organized as follows.In Section 1, the significance and background of research is introduced.Some assumptions and main results are stated in Section 2. Several useful lemmas are listed in Section 3. The proofs are listed in Section 4. Examples are stated in Section 5.In the sequel, we denote with C, C 1 , C 2 , • • • generic constants that may be different in each of its appearances.I{A} denotes the indicator function of the set A. Φ(•) denotes the distribution function of the standard normal random variable N .

Main Results
The goal of this paper is to study the properties of the estimator of the error variance σ 2 by means of the observations {X 1 , X 2 , • • • , X n } in model (1).The main difficulty is that we do not observe the error {ε 1 , ε 2 , • • • , ε n }, the structure of estimation of parameters is complex, and the residuals are unknown.We need to use Taylor's expansion and many other techniques to deal with it.This is the greatest contribution of this paper.We will follow the following steps.Firstly, we compute an estimator θ = ( θ1 , . . ., θq ) ′ of unknown parameter θ.Secondly, based on the estimator θ and model (1), we calculate the following residuals εi = X i − r θ X i−1 , . . ., X i−p , i = 1, 2, . . ., n. (3) Finally, using the above residuals, we estimate the error variance σ 2 by using the following equation Before giving the main results, we need the following basic assumptions for model (1) which will be used throughout the paper.For 1 ≤ i ≤ n and 1 ≤ j ≤ q, let where E[M 4  1 Assumption 2. Let θ = ( θ1 , . . . ,θq ) ′ be a strong consistent estimator for θ satisfying the following law of iterated logarithm where and C is a positive constant.
Remark 1.By Corollary 2.2 of Klimko and Nelson [25], we know that the least square estimator for the stochastic process under some suitable conditions satisfies (5).For the first-order autoregressive progresses, Wang et al. [26] proved that the least square estimator of the unknown parameters meets (5).For smooth threshold autoregressive progresses, Chan and Tong [27] obtained the conditional least square estimators of the unknown parameters that satisfy (5).For general nonlinear autoregressive progresses of order p, Liebscher [2] established M-estimators for the unknown parameters that satisfy (5), Yao [28] obtained (5) for least square estimators of nonlinear autoregressive progresses.
Now, we will state the main result for the almost sure central limit theorem of the error variance estimator σ2 .
Theorem 1. Suppose that {d k } is a sequence of positive numbers satisfying the following conditions: For model (1), under the Assumptions 1 and 2, if Eε 4 1 < ∞, for all x ∈ R, we have Then, under the assumptions of Theorem 1, (6) also holds.
Remark 2. If the conditions (C1) and (C2) of Theorem 1 is satisfied for some sequence {D n }, then it is also satisfied for any other sequence D Remark 3. It is easy to show that d k = l(k)/k, where l(x) is slowly varying at infinity and D n → ∞, satisfies the conditions (C1) and (C2).

Preliminary Lemmas
Some useful lemmas which are needed to prove the main result are given in the following section.
is a martingale and p > 0, then there exists constant C depending only on p such that Proof.The proof of Lemma 2 is obvious by Assumption 1 and the H ölder inequality.
Lemma 3. Assume that {G n , n ≥ 1} is a sequence of random variables satisfying the ASCLT with the weight {d k } defined as in Theorem 1, that is Let {R n , n ≥ 1} be a sequence of random variables converging almost surely to zero.Then, {G n + R n , n ≥ 1} also satisfies the ASCLT.That is Proof.For fixed x ∈ R and η > 0, recall that {G n , n ≥ 1} satisfies the ASCLT, then we have Remark that Then, we can conclude that Noting that {R n , n ≥ 1} is a sequence of random variables converging almost surely to zero and the arbitrariness of η, the desired conclusion follows from above discussion.Lemma 4 (Zhang [30], Lemma 2.10, P.391).Let {ζ n , n ≥ 1} be a sequence of uniformly bounded random variables and {d n }, {D n } be defined as in Theorem 1.If there exist constants C > 0 and δ > 0 and a sequence of positive numbers {a(k Lemma 5. Let {ε i , i ≥ 1} be a sequence of independent and identically distributed random variables with mean zero, finite variance σ 2 and Eε 4 1 < ∞.Let {d n }, {D n } be defined as in Theorem 1. Then for all x ∈ R, Suppose that f is a bounded Lipschitz function.By classical central limit theorem, we have By the conclusions in Section 2 of Peligrad and Shao [31] and Theorem 7.1 of Billingsley [32], we know that ( 7) is equivalent to Hence, to prove (7), it suffices to show that For convenience, let ) ).Notice that {ε i , i ∈ Z} are independent, both f and f ′ are bounded, then we conclude that for 1 jVar(ε 2 1 ) then by Lemma 4 with δ = 1/2 and a(k) ≡ 0, (8) holds, and therefore, the proof of ( 7) is completed.Lemma 6.Under the assumptions of Theorem 1, for any 1 ≤ j ≤ q, we have By Lemma 2 and the Markov inequality, for any ϵ > 0, it is easy to known that Then by the Borel-Cantelli lemma, we obtain Similarly, by Lemma 2 and the Markov inequality, for any ϵ > 0, one can obtain By the Borel-Cantelli lemma, it follows that log log n k+1 Then combining ( 9) with (10), for n k < n ≤ n k+1 , one can obtain lim sup Thus, the proof of Lemma 6 is completed.
Lemma 7.Under the assumptions of Theorem 1, for any 1 ≤ j, l ≤ q, we have Proof.By Lemma 2 and the Markov inequality, for any ϵ > 0, it is easy to see that By the Borel-Cantelli lemma, one can obtain The proof of Lemma 7 is completed.
Lemma 8.Under the assumptions of Theorem 1, for any 1 ≤ j ≤ q, we have Let F m be the σ-algebra generated by the random variables {ε i , 1 ≤ i ≤ m}.By the fact that Y ij and ε i are independent, it is easy to compute that the process {Y m , F m , 1 ≤ m ≤ n} is a martingale.By Lemmas 1 and 2, for some 2 < t < 4, we know By the Markov inequality and (11), for any ϵ > 0, it is easy to obtain By the Borel-Cantelli lemma, we can obtain The proof of Lemma 8 is completed.
Lemma 9.Under the assumptions of Theorem 1, for any 1 ≤ j, l ≤ q, we have By Lemma 2, the Markov inequality, C r inequality and Cauchy-Schwarz inequality, for any ϵ > 0, it is easy to see that Then by the Borel-Cantelli lemma, we obtain Similarly, By Lemma 2 and the Markov inequality and C r inequality, for any ϵ > 0, one can obtain By the Borel-Cantelli lemma, it follows that log log n k+1 Then combining ( 12) with ( 13), for n k < n ≤ n k+1 , one can obtain lim sup The proof of Lemma 9 is completed.
Lemma 10.Under the assumptions of Theorem 1, for any 1 ≤ j, l, k ≤ q, we have Proof.By Lemma 2, the Markov inequality, C r inequality and Cauchy-Schwarz inequality, for any ϵ > 0, it is easy to see that where 2 < t ≤ 4 is defined in Assumption 1.By the Borel-Cantelli lemma, we obtain The proof of Lemma 10 is completed.

Examples
Some examples are given in this section to verify the almost sure central limit theorem for the error variance estimator for some special nonlinear autoregressive models.The first example is a degenerate model, that is, AR(1) progresses.
Example 1.An AR(1) model is a family of {X i } of random variables such that for every i ≥ 1 where {ε i , i ≥ 1} is a collection of i.i.d.random variables with zero mean and finite variance σ 2 .We also assume that E exp{γ|ε i ε j |} < ∞ for some γ > 0 and any i, j ≥ 1.It is obviously that {X i } is a stationary model under the condition |θ| < 1.
It is easy to check that the Assumption 1 holds naturally.For Assumption 2, by Theorem 1 of Wang et al. [26] and E exp{γ|ε i ε j |} < ∞, (5) holds for the least squares estimator θ.Therefore, we have the following statement for AR(1) progression due to Theorem 1. Theorem 2. Suppose {d k } is a sequence of positive numbers satisfying conditions (C1) and (C2).For the above AR(1) model, if E exp{γ|ε i ε j |} < ∞ for some γ > 0 and any i, j ≥ 1, for any x ∈ R, one can obtain The next example concerns the self-exciting threshold autoregressive (SETAR) progresses.

Example 2.
Let {X i , i ≥ p} be a sequence of stationary and geometrically ergodic random variable satisfying the following continuous SETAR(p, l, d) progresses.
By Corollary 3.1 of Liebscher [2], under Condition C and E|ε i | γ < ∞, E|| Xi || γ < ∞, γ > 4, the Assumption 2 holds.Therefore, we have the following result for SETAR progresses due to Theorem 1. Theorem 3. Suppose {d k } is a sequence of positive numbers satisfying conditions (C1) and (C2).For the above SETAR(p, l, d) progresses, under Assumption 1 and Condition C and E|ε Next, we will consider the threshold-exponential AR progresses.
Example 3. Let R j , j = 1, • • • , K be non-overlapping and non-empty intervals of R such that j R j = R.A combined threshold-exponential AR progresses is defined by with X 0 = x 0 and {ε i } is a collection of i.i.d.random variables with zero mean.Let the true parameters δ > 0, by Theorem 4 of Yao [28], (5) holds for the least squares estimator θ.Therefore, we have the following statement for threshold-exponential AR progresses due to Theorem 1. Next, we will consider the multilayer perceptrons progress.
Example 4. Multilayer perceptrons progressesw have become popular in nonlinear modeling due to its universal approximation ability.Such an example is the model described below which has p input units feeding by variables X i−1 , • • • , X i−p at time i, a hidden layer with K units and one output unit which provides the variable X i where {ε i } is a collection of i.i.d.random variables with zero mean.Let the true parameters θ 0 = (α 0 , • • • , α K , β lj , 0 ≤ l ≤ p, 1 ≤ j ≤ K) ⊤ ∈ Θ ⊂ R q and the parameters then by Theorem 5 of Yao [28], (5) holds for the least squares estimator θ.Therefore, we have the following statement for multilayer perceptrons due to Theorem 1.
Theorem 5. Suppose {d k } is a sequence of positive numbers satisfying conditions (C1) and (C2).
For the univariate multilayer perceptrons progress with ψ(x) = tanh(x), if for all θ different from θ 0 , there exists x ∈ R p such that r θ (x) ̸ = r θ 0 (x), E|ε i | 6+δ < ∞ for some δ > 0 and the matrix I 0 is regular, then under Assumption 1, for any x ∈ R, one can obtain

Conclusions
In this paper, using Taylor's expansion, the Borel-Cantelli lemma and the classical almost sure central limit theorem for independent random variables, the authors establish the almost sure central limit theorem for the error variance estimator for nonlinear autoregressive progresses with independent and identical distributed errors.The results extend the almost sure central limit theorem for the error variance estimator to the nonlinear autoregressive progresses.Four examples, first-order autoregressive processes, self-exciting threshold autoregressive processes, threshold-exponential AR progresses and multilayer perceptrons progress, are given to verify the results.In the future, we will try to investigate the almost sure central limit theorem for the error variance estimator for nonlinear autoregressive progresses with dependent errors and the moderate deviation principle for the error variance estimator for nonlinear autoregressive progresses with independent errors.