The Consistency of Estimators in a Heteroscedastic Partially Linear Model with ρ⁻-Mixing Errors

Abstract

This paper studies a heteroscedastic partially linear model based on ${\rho }^{-}$-mixing random errors, stochastically dominated and with zero mean. Under some suitable conditions, the strong consistency and $p$-th $\left(p>0\right)$ mean consistency of least squares (LS) estimators and weighted least squares (WLS) estimators for the unknown parameter are investigated, and the strong consistency and $p$-th $\left(p>0\right)$ mean consistency of the estimators for the non-parametric component are also studied. These results include the corresponding ones of independent, negatively associated (NA), and ${\rho }^{*}$-mixing random errors as special cases. At last, two simulations are presented to support the theoretical results.

1. Introduction

Consider the following heteroscedastic partially linear model:

$$y^{\left(t\right)}\left(x_{in},z_{in}\right)=z_{in}\beta +h\left(x_{in}\right)+{\sigma }_{in}{\epsilon }^{(t)}\left(x_{in}\right),1\le t\le r,1\le i\le n,$$

(1)

where ${\sigma }_{in}^2=f\left(u_{in}\right)$, $z_{in}\in R$, $x_{in}\in R^p$, $u_{in}\in R^p$, and $\left(x_{in},z_{in},u_{in}\right)$ are known and nonrandom design points, $\beta$ represents an unknown parameter, $f(\cdot )$ and $h(\cdot )$ represent unknown functions, which are defined on a compact set $M\subset R^p$, $y^{\left(t\right)}\left(x_{in},z_{in}\right)$ stands for the $t$-th variables that can be observable at points $\left(x_{in},z_{in}\right)$, and $\left\{{\epsilon }^{\left(t\right)}\left(x_{in}\right),1\le t\le r,1\le i\le n\right\}$ stands for random errors.

In order to analyze the effect of temperature on electricity usage, Engle et al. [1] proposed the partially linear model

$$y_i={x^{\prime }}_i\beta +h\left(z_i\right)+{\epsilon }_i,1\le i\le n.$$

(2)

Since then, many statisticians have studied partially linear regression models. The model (2) was further investigated by Heckman [2], Speckman [3], Gao [4], Härdle et al. [5], Hu et al. [6], Zeng and Liu [7], and so forth. Some applications of the model were given. Inspired by the model (2), a more general model was proposed by Gao et al. [8]:

$$y_i=x_i\beta +h\left(z_i\right)+{\sigma }_i{\epsilon }_i,1\le i\le n.$$

(3)

Gao et al. [8] established the asymptotic normality of least squares (LS) and weighted least squares (WLS) estimators for $\beta$ based on the family of non-parametric estimators for $h(\cdot )$ and $f(\cdot )$ in the model (3). Baek and Liang [9] investigated the asymptotic property in the model (3) for negatively associated errors. Zhou et al. [10] derived the moment consistency for $\beta$ and $h(\cdot )$ in model (3) under negatively associated samples. Hu [11] proposed a new partially linear model

$$y^{\left(t\right)}\left(x_{in},z_{in}\right)=z_{in}\beta +h\left(x_{in}\right)+{\epsilon }^{\left(t\right)}\left(x_{in}\right),1\le t\le r,1\le i\le n,$$

(4)

and derived the strong and moment consistencies with independent and $\phi$-mixing errors. Li and Yang [12,13] studied the moment and strong consistencies for $\beta$ and $h(\cdot )$ in the model (4) based on negatively associated samples. Wang et al. [14] and Wu and Wang [15] discussed the moment and strong consistencies for LS and WLS estimators of $\beta$ and $h(\cdot )$ with ${\rho }^{*}$-mixing errors. In the present paper, we will investigate the model (1) based on the model (4). The model (1) can be used in hydrology, biology, and so on (see [16]).

Now, let us recall some concepts of dependent structures. Assume that $\mathbb{N}$ is a set of natural numbers and $S,T\subset \mathbb{N}$ are two non-empty disjoint sets. We define that $\mathrm{dist}\left(S,T\right)=\underset{i\in S,j\in T}{\min }\left|i-j\right|$.

Definition 1 

([17]). A finite collection of random variables $\left\{X_n,n\ge 1\right\}$ is called negatively associated (NA) if for each pair of disjoint subsets $B_1$, $B_2$ of $\mathbb{N}$,

$$Cov\left\{g_1\left(X_k,k\in B_1\right),g_2\left(X_l,l\in B_2\right)\right\}\le 0,$$

where $g_1$ and $g_2$ are any coordinate-wise non-decreasing functions such that the covariance exists.

Definition 2 

([18]). A random sequence $\left\{X_n,n\ge 1\right\}$ is said to be ${\rho }^{*}$-mixing if

$${\rho }^{*}\left(u\right)=\sup \left\{\rho \left(T,U\right):T,U\subset \mathbb{N},\mathrm{dist}\left(T,U\right)\ge u\right\}\to 0\left(u\to \infty \right),$$

where

$$\rho \left(T,U\right)=\sup \left\{\frac{\left|E\left(XY\right)-E\left(X\right)E\left(Y\right)\right|}{\sqrt{Var\left(X\right)Var\left(Y\right)}}:X\in L_2\left(\sigma \left(T\right)\right),Y\in L_2\left(\sigma \left(U\right)\right)\right\},$$

$\sigma \left(T\right)$ and $\sigma \left(U\right)$ are $\sigma$-fields that are generated by $\left\{X_j,j\in T\right\}$ and $\left\{X_k,k\in U\right\}$ respectively, $L_2\left(\sigma \left(T\right)\right)$ is the space of all square integral and $\sigma \left(T\right)$-measurable random variables, and $L_2\left(\sigma \left(U\right)\right)$ is defined in the same way.

Definition 3 

([19,20]). A sequence $\left\{X_n,n\ge 1\right\}$ is said to be ${\rho }^{-}$-mixing if

$${\rho }^{-}\left(u\right)=\sup \left\{{\rho }^{-}\left(T,U\right):T,U\subset \mathbb{N},\mathrm{dist}\left(T,U\right)\ge u\right\}\to 0\left(u\to \infty \right),$$

where

$${\rho }^{-}\left(T,U\right)=0\vee \left\{\frac{Cov\left(g_1\left(X_i,i\in T\right),g_2\left(X_j,j\in U\right)\right)}{\sqrt{Var\left(g_1\left(X_i,i\in T\right)\right)Var\left(g_2\left(X_j,j\in U\right)\right)}}:g_1,g_2\in H\right\},$$

$0\vee x=\max \left\{0,x\right\}$, and $H$ is the set of non-decreasing functions.

We can easily see that ${\rho }^{-}\left(u\right)\le {\rho }^{*}\left(u\right)$ and a ${\rho }^{-}$-mixing sequence is NA (in particular, independent) if and only if ${\rho }^{-}\left(1\right)=0$. Therefore, ${\rho }^{-}$-mixing sequences include ${\rho }^{*}$-mixing sequences and NA sequences as special cases. However, ${\rho }^{*}$-mixing sequences and NA sequences are not always ${\rho }^{-}$-mixing sequences. Zhang and Wang [19] constructed the following example, which is a ${\rho }^{-}$-mixing sequence, but not NA and also not ${\rho }^{*}$-mixing.

Example 1 

([19]). Assume that $\left\{{\xi }_n,n\ge 1\right\}$, $\left\{{\eta }_n,n\ge 1\right\}$, and $\left\{{\zeta }_n,n\ge 1\right\}$ are three independent sequences that are independent and identically distributed (i.i.d.) standard normal random variables. Denote

$$X_n=\{\begin{array}{l}{\xi }_t,n=2t-1\\ -{\xi }_t,n=2t\end{array},Y_n=\{\begin{array}{l}{\eta }_t,n=2^{2t-1}\\ -{\eta }_t,n=2^{2t}\\ {\zeta }_n,\mathrm{else}\end{array},$$

and $Z_n=X_n^2+Y_n$. Then, $\left\{Z_n,n\ge 1\right\}$ is ${\rho }^{-}$-mixing with ${\rho }^{-}\left(2\right)=0$. However, $\left\{Z_n,n\ge 1\right\}$ is neither NA nor ${\rho }^{*}$-mixing.

On one hand, NA sequences have been widely applied to reliability theorem and multivariate statistical analysis (see [21,22]). On the other hand, some Markov Chains and moving average processes are ${\rho }^{*}$-mixing sequences (see [23]). The concept of ${\rho }^{*}$-mixing sequences is important in a lot of areas, for instance, finance, economics, and other sciences (see [24]). Therefore, studying ${\rho }^{-}$-mixing sequences is of considerable significance.

Since Zhang and Wang [19] proposed the concept of ${\rho }^{-}$-mixing sequences, many results on ${\rho }^{-}$-mixing sequences have been established. One can refer to Zhang and Wang [19], Wang and Lu [25], and Yuan and An [26] for some moment inequalities and some limiting behavior; Zhang [20] and Zhang [27] for some central limit theorems; Chen et al. [28] for complete convergence for weighted sums of ${\rho }^{-}$-mixing sequences; Zhang [29] for the complete moment convergence for the partial sum of ${\rho }^{-}$-mixing moving average processes; Wu and Jiang [30] for almost sure convergence of ${\rho }^{-}$-mixing sequences; and Xu and Wu [31] for an almost sure central limit theorem for the self-normalized partial sums.

However, we have not found studies on the model (1) under ${\rho }^{-}$-mixing random errors in the literature. In the present paper, we will study the estimation problem for the model (1) based on the assumption that the errors are ${\rho }^{-}$-mixing sequences that are stochastically dominated and zero mean. The strong consistency and mean consistency of LS estimators and WLS estimators for $\beta$ and $h(\cdot )$ are established respectively based on some suitable conditions. The results obtained in the paper deal with independent errors as well as dependent errors as special cases.

Next, we will recall the definition of stochastic domination.

Definition 4 

([32]). A random sequence $\left\{Y_i,i\ge 1\right\}$ is stochastically dominated by a random variable $Y$ if

$$P\left(\left|Y_i\right|>y\right)\le cP\left(\left|Y\right|>y\right)$$

for some $c>0$, every $y\ge 0$ and each $n\ge 1$.

The remainder of this paper is organized as follows. The LS estimators and WLS estimators of $\beta$ based on the family of non-parametric estimators for $h(\cdot )$ and some conditions are introduced in Section 2. We give the main results in Section 3. Several lemmas are given in Section 4. We provide the proofs of the main results in Section 5. Two simulations are carried out in Section 6. We conclude the paper in Section 7. Throughout the paper, let $C$ denote positive constants whose values may be different in various places. “i.i.d.” stands for independent and identically distributed. $\Vert \cdot \Vert$ stands for the Euclidean norm.

2. Estimation and Conditions

Assume that $\left\{y^{(t)}\left(x_{in},z_{in}\right),z_{in}\in R,x_{in}\in M,u_{in}\in M,1\le t\le r,1\le i\le n\right\}$ satisfies the model (1) and $W_{nj}\left(x\right)=W_{nj}\left(x;x_1,x_2,\cdots ,x_n\right)$ is a weight function that is measurable on the compact set $M$. For simplicity and convenience, the model (1) can be written as

$$y_i^{\left(t\right)}=z_i\beta +h\left(x_i\right)+{\sigma }_i{\epsilon }_i^{\left(t\right)},1\le t\le r,1\le i\le n.$$

(5)

We denote ${\tilde{z}}_j=z_j-{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x_j\right)z_i}$, ${\tilde{y}}_{(k)}^j=y_j^{\left(k\right)}-\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x_j\right)y_i^{\left(t\right)}}}$, ${\gamma }_i=1/{\sigma }_i^2$, $1\le k\le r$, $1\le j\le n$, ${\tilde{T}}_n^2={\displaystyle \sum _{i=1}^n{\tilde{z}}_i^2}$, and ${\tilde{U}}_n^2={\displaystyle \sum _{i=1}^n{\gamma }_i{\tilde{z}}_i^2}$.

For the model (5), one can get from $E\left({\epsilon }_i^{\left(t\right)}\right)=0$ that $h\left(x_i\right)=E\left(y_i^{(t)}-z_i\beta \right)$ for $1\le t\le r$, $1\le i\le n$. Thus, for any given $\beta$, we can define the non-parametric estimator of $h(\cdot )$ in terms of

$$h_{r,n}\left(x,\beta \right)=\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right)}}\left(y_i^{(t)}-z_i\beta \right).$$

(6)

Hence, the LS estimators of $\beta$ can be defined by

$${\widehat{\beta }}_{r,n}^{\left(LS\right)}=\arg \underset{\beta }{\min }{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{\left[y_i^{(t)}-z_i\beta -h_{r,n}\left(x_i,\beta \right)\right]}^2}}.$$

(7)

By (7), we have

$${\widehat{\beta }}_{r,n}^{\left(LS\right)}=\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{\tilde{z}}_i{\tilde{y}}_i^{(t)}/{\tilde{T}}_n^2}}.$$

(8)

When the random errors are heteroscedastic, we modify ${\widehat{\beta }}_{r,n}^{\left(LS\right)}$ to a WLS estimator. We can define the WLS estimators of $\beta$ in terms of

$${\widehat{\beta }}_{r,n}^{\left(WLS\right)}=\arg \underset{\beta }{\min }{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{\left[\left(y_i^{(t)}-z_i\beta -h_{r,n}\left(x_i,\beta \right)\right)/{\sigma }_i\right]}^2}}.$$

(9)

By (9), we derive that

$${\widehat{\beta }}_{r,n}^{\left(WLS\right)}=\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{\gamma }_i{\tilde{z}}_i{\tilde{y}}_i^{(t)}/{\tilde{U}}_n^2}}.$$

(10)

Taking into account ${\widehat{\beta }}_{r,n}^{\left(LS\right)}$ and ${\widehat{\beta }}_{r,n}^{\left(WLS\right)}$, we define the estimator of $h(\cdot )$ respectively:

$${\widehat{h}}_{r,n}\left(x\right)=\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right)\left(y_i^{\left(t\right)}-z_i{\widehat{\beta }}_{r,n}^{\left(LS\right)}\right)}}$$

(11)

and

$${\tilde{h}}_{r,n}\left(x\right)=\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right)\left(y_i^{\left(t\right)}-z_i{\widehat{\beta }}_{r,n}^{\left(WLS\right)}\right)}}.$$

(12)

In order to obtain the relevant theorems, several important conditions are given below.

$\left(C_1\right)$

(i) $\underset{n\to \infty }{\lim }{\tilde{T}}_n^2/n=C$;

(ii) $0<s_0\le \underset{u\in M}{\inf }f\left(u\right)\le \underset{u\in M}{\sup }f\left(u\right)\le S_0<\infty$;

(iii) $f(\cdot )$ and $h(\cdot )$ are continuous functions on compact set $M$.

$\left(C_2\right)$

(i) $\underset{x\in M}{\sup }{\displaystyle {\sum }_{i=1}^n\left|W_{ni}\left(x\right)\right|}=O\left(1\right)$;

(ii)$\underset{i\ge 1,x\in M}{\sup }\left|W_{ni}\left(x\right)\right|=O\left(n^{-\alpha }\right)$ for some $\alpha >0$.

$\left(C_3\right)$

(i) $\underset{x\in M}{\sup }\left|{\displaystyle {\sum }_{i=1}^n{W}_{ni}\left(x\right)-1}\right|=o\left(1\right)$;

(ii) $\underset{x\in M}{\sup }{\displaystyle {\sum }_{i=1}^n\left|W_{ni}\left(x\right)\right|I\left(\Vert x_i-x\Vert >\delta \right)}=o\left(1\right)$ for any $\delta >0$.

$\left(C_4\right)$

$\underset{x\in M}{\sup }\left|{\displaystyle {\sum }_{i=1}^n{W}_{ni}\left(x\right)z_i}\right|=O\left(1\right)$.

Remark 1.

Conditions$\left(C_1\right)$(i) (ii) are some regular conditions that are often imposed in studies of LS and WLS estimators in heteroscedastic partially linear models. One can refer to [5,8,9] and so on.$\left(C_1\right)$(iii) is mild and holds for most commonly used functions, such as polynomial and trigonometric functions (see [9]). Conditions$\left(C_2\right)$–$\left(C_4\right)$are often applied to investigate strong consistency (see [9,33]) and mean consistency (see [10,16]).$\left(C_2\right)$(ii) is weaker than the corresponding conditions of [16] and [33]. Thus, the above conditions are very mild. Moreover, by$\left(C_1\right)$(i) (ii), one can get that

$${\tilde{T}}_n^{-2}{\displaystyle \sum _{i=1}^n\left|{\tilde{z}}_i\right|}\le C$$

(13)

and

$${\tilde{U}}_n^{-2}{\displaystyle \sum _{i=1}^n\left|{\gamma }_i{\tilde{z}}_i\right|}\le C.$$

(14)

3. Main Results

In this paper, let $\left\{{\epsilon }_i^{\left(t\right)},1\le t\le r,1\le i\le n\right\}$ be a ${\rho }^{-}$-mixing sequence with zero mean, which is stochastically dominated by a random variable $\epsilon$.

Theorem 1.

Assume that$\left(C_1\right)$-$\left(C_3\right)$hold. If$E{\left|\epsilon \right|}^p<\infty$for some$p>2$, then

$${\widehat{\beta }}_{r,n}^{\left(LS\right)}\stackrel{a.s.}{\to }\beta$$

(15)

as$\min \left(r,n\right)\to \infty$and

$${\widehat{\beta }}_{r,n}^{\left(WLS\right)}\stackrel{a.s.}{\to }\beta .$$

(16)

as$\min \left(r,n\right)\to \infty$.

Theorem 2.

Under the conditions of Theorem 1, in addition, if$\left(C_4\right)$holds, then

$$\underset{x\in M}{\sup }\left|{\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)\right|\stackrel{a.s.}{\to }0$$

(17)

as$\min \left(r,n\right)\to \infty$and

$$\underset{x\in M}{\sup }\left|{\tilde{h}}_{r,n}\left(x\right)-h\left(x\right)\right|\stackrel{a.s.}{\to }0.$$

(18)

as$\min \left(r,n\right)\to \infty$.

Theorem 3.

Assume that$\left(C_1\right)$-$\left(C_3\right)$holds. If$E{\left|\epsilon \right|}^p<\infty$for some$p\ge 2$, then

$$\underset{\min \left(r,n\right)\to \infty }{\lim }E{\left|{\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta \right|}^p=0$$

(19)

and

$$\underset{\min \left(r,n\right)\to \infty }{\lim }E{\left|{\widehat{\beta }}_{r,n}^{\left(WLS\right)}-\beta \right|}^p=0.$$

(20)

Theorem 4.

Under the conditions of Theorem 3, in addition, if$\left(C_4\right)$holds, then

$$\underset{\min \left(r,n\right)\to \infty }{\lim }\underset{x\in M}{\sup }E{\left|{\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)\right|}^p=0$$

(21)

and

$$\underset{\min \left(r,n\right)\to \infty }{\lim }\underset{x\in M}{\sup }E{\left|{\tilde{h}}_{r,n}\left(x\right)-h\left(x\right)\right|}^p=0.$$

(22)

Remark 2.

Since${\rho }^{-}$-mixing sequences include NA (in particular, independent) and${\rho }^{*}$-mixing sequences, Theorems 1–4 also apply for NA and${\rho }^{*}$-mixing sequences.

4. Some Lemmas

From the definition of ${\rho }^{-}$-mixing sequences, we can get the first lemma.

Lemma 1.

If$\left\{X_i,i\ge 1\right\}$is a${\rho }^{-}$-mixing sequence with mixing coefficients${\rho }^{-}\left(u\right)$, then$\left\{f_i\left(X_i\right),i\ge 1\right\}$is still a${\rho }^{-}$-mixing sequence with mixing coefficients not greater than${\rho }^{-}\left(u\right)$. Here,$f_1,f_2,\cdots$are non-decreasing functions (non-increasing functions).

Lemma 2

(Rosenthal-type inequality, [25,29]). If $\left\{X_i,i\ge 1\right\}$ is a ${\rho }^{-}$-mixing sequence of zero mean with $E{\left|X_i\right|}^p<\infty$ for some $p\ge 2$, then there exists a constant $C>0$ depending only on $p$ and ${\rho }^{-}\left(s\right)$ such that

$$E\left(\underset{1\le i\le n}{\max }{\left|S_i\right|}^p\right)\le C\left\{{\displaystyle \sum _{i=1}^{n}E{\left|X_i\right|}^p+{\left[{\left({\displaystyle \sum _{i=1}^{n}E{X}_i^2}\right)}^{p/2}\right]}^p}\right\},$$

for any $n\ge 1$, here $S_i={\displaystyle \sum _{j=1}^i{X}_j}$.

Lemma 3

([32]). If $\left\{X_i,i\ge 1\right\}$ is a random sequence that is stochastically dominated by a random variable $X$, for every $a>0$ and $\beta >0$, we have

$$E{\left|X_i\right|}^{\beta }I\left(\left|X_i\right|\le a\right)\le c_1\left[E{\left|X\right|}^{\beta }I\left(\left|X\right|\le a\right)+a^{\beta }P\left(\left|X\right|>a\right)\right],$$

$$E{\left|X_i\right|}^{\beta }I\left(\left|X_i\right|>a\right)\le c_{2}E{\left|X\right|}^{\beta }I\left(\left|X\right|>a\right).$$

Therefore,

$$E{\left|X_i\right|}^{\beta }\le cE{\left|X\right|}^{\beta },$$

where $c$ is a positive constant.

Lemma 4.

Let$\left\{{\epsilon }_i^{\left(t\right)},1\le t\le r,1\le i\le n\right\}$be a${\rho }^{-}$-mixing sequence of zero mean. Suppose that$\left\{c_{ni}\left(v\right),1\le i\le n,n\ge 1\right\}$is an array of functions defined on a compact set$M$such that$\underset{v\in M}{\sup }{\displaystyle \sum _{i=1}^n\left|c_{ni}\left(v\right)\right|}=O\left(1\right)$and$\underset{i\ge 1,v\in M}{\sup }\left|c_{ni}\left(v\right)\right|=O\left(n^{-\gamma }\right)$for some$\gamma >0$. If$E{\left|\epsilon \right|}^p<\infty$for some$p>2$, then

$$\underset{v\in M}{\sup }\left|\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){\epsilon }_i^{\left(t\right)}}}\right|\stackrel{a.s.}{\to }0$$

(23)

as$\min \left(r,n\right)\to \infty$.

Proof. 

Denote

$${\epsilon }_{1i}^{\left(t\right)}=-r^{\frac{1}{p}}I\left({\epsilon }_i^{\left(t\right)}<-r^{\frac{1}{p}}\right)+{\epsilon }_i^{\left(t\right)}I\left(\left|{\epsilon }_i^{\left(t\right)}\right|\le r^{\frac{1}{p}}\right)+r^{\frac{1}{p}}I\left({\epsilon }_i^{\left(t\right)}>r^{\frac{1}{p}}\right),$$

$${\epsilon }_{2i}^{\left(t\right)}={\epsilon }_i^{\left(t\right)}-{\epsilon }_{1i}^{\left(t\right)}=\left({\epsilon }_i^{\left(t\right)}+r^{\frac{1}{p}}\right)I\left({\epsilon }_i^{\left(t\right)}<-r^{\frac{1}{p}}\right)+\left({\epsilon }_i^{\left(t\right)}-r^{\frac{1}{p}}\right)I\left({\epsilon }_i^{\left(t\right)}>r^{\frac{1}{p}}\right),$$

$${{\epsilon }^{\prime }}_i^{\left(t\right)}={\epsilon }_{1i}^{\left(t\right)}-E{\epsilon }_{1i}^{\left(t\right)}$$

and

$${{\epsilon }^{″}}_i^{\left(t\right)}={\epsilon }_{2i}^{\left(t\right)}-E{\epsilon }_{2i}^{\left(t\right)}.$$

Without loss of generality, one can suppose that $c_{ni}\left(v\right)>0$. Hence, we know by Lemma 1 that $\left\{c_{ni}\left(v\right){\epsilon }_i^{\left(t\right)},1\le t\le r,1\le i\le n\right\}$, $\left\{c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)},1\le t\le r,1\le i\le n\right\}$, and $\left\{c_{ni}\left(v\right){{\epsilon }^{″}}_i^{\left(t\right)},1\le t\le r,1\le i\le n\right\}$ are also ${\rho }^{-}$-mixing sequences with zero mean. Note that ${\epsilon }_i^{\left(t\right)}={{\epsilon }^{\prime }}_i^{\left(t\right)}+{{\epsilon }^{″}}_i^{\left(t\right)}$. Hence, for any $v\in M$, we have

$$\begin{array}{cc}\hfill A_{r,n}& =:P\left(\left|\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){\epsilon }_i^{\left(t\right)}}}\right|>\epsilon \right)\hfill \\ & \le P\left(\left|{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}}}\right|>r\epsilon /2\right)+P\left(\left|{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){{\epsilon }^{″}}_i^{\left(t\right)}}}\right|>r\epsilon /2\right)\hfill \\ & =:\left(A_{r,n}^{\left(1\right)}+A_{r,n}^{\left(2\right)}\right).\hfill \end{array}$$

(24)

The proof of (24) is similar to that of the Lemma 3.3 in Zhou and Hu [34]. By Lemma 3, we have $E{\left|{\epsilon }_i^{\left(t\right)}\right|}^p\le CE{\left|\epsilon \right|}^p<\infty$. Hence, for every $s>p>2$, from the Markov inequality, Lemma 2 and $E{\left|\epsilon \right|}^p<\infty$, we get that

$$\begin{array}{cc}\hfill A_{r,n}^{\left(1\right)}& \le C{r}^{-s}E{\left|{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}}}\right|}^s\hfill \\ & \le C{r}^{-s}\left\{{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left|c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}\right|}^s}}+\left[{\left({\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left(c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}\right)}^2}}\right)}^{s/2}\right]\right\}\hfill \\ & \le C{r}^{-s}\underset{z\in M}{\sup }\left\{{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left|c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}\right|}^s}}+\left[{\left({\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left(c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}\right)}^2}}\right)}^{s/2}\right]\right\}\hfill \\ & \le C{r}^{-s}\left\{\underset{z\in M}{\sup }{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left|c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}\right|}^s}}+\left[\underset{z\in M}{\sup }{\left({\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left(c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}\right)}^2}}\right)}^{s/2}\right]\right\}\hfill \\ & \le C{r}^{-s}\left(r^{s/p}{n}^{-\gamma \left(s-1\right)}+r^{s/2}{n}^{-\gamma s/2}\right)\le C{r}^{-s/2}\hfill \end{array}$$

(25)

and

$$\begin{array}{cc}\hfill A_{r,n}^{\left(2\right)}& \le C{r}^{-p}E{\left|{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){{\epsilon }^{″}}_i^{\left(t\right)}}}\right|}^p\hfill \\ & \le C{r}^{-p}\left\{\underset{z\in M}{\sup }{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left|c_{ni}\left(v\right){{\epsilon }^{″}}_i^{\left(t\right)}\right|}^p}}+\left[\underset{z\in M}{\sup }{\left({\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left(c_{ni}\left(v\right){{\epsilon }^{″}}_i^{\left(t\right)}\right)}^2}}\right)}^{p/2}\right]\right\}\hfill \\ & \le C{r}^{-p}\left(r{n}^{-\gamma \left(p-1\right)}+r^{p/2}{n}^{-\gamma p/2}\right)\hfill \\ & \le C\left(r^{-p+1}+r^{-p/2}\right).\hfill \end{array}$$

(26)

Hence, it follows from (24) through (26) and $s>p>2$ that

$${\displaystyle \sum _{r=1}^{\infty }{A}_{r,n}}\le C{\displaystyle \sum _{r=1}^{\infty }\left(r^{-s/2}+r^{-p+1}+r^{-p/2}\right)}<\infty .$$

By the Borel–Cantenlli lemma, we obtain for any $v\in M$ that

$$\left|\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){\epsilon }_i^{\left(t\right)}}}\right|\stackrel{a.s.}{\to }0$$

as $\min \left(r,n\right)\to \infty$. Therefore, (23) follows. □

Lemma 5.

Let$\left\{{\epsilon }_i^{\left(t\right)},1\le t\le r,1\le i\le n\right\}$be a${\rho }^{-}$-mixing random sequence of zero mean. Suppose that$\left\{c_{ni}\left(v\right),1\le i\le n,n\ge 1\right\}$is an array of functions defined on a compact set$M$such that$\underset{v\in M}{\sup }{\displaystyle \sum _{i=1}^n\left|c_{ni}\left(v\right)\right|}=O\left(1\right)$and$\underset{i\ge 1,v\in M}{\sup }\left|c_{ni}\left(v\right)\right|=O\left(n^{-\alpha }\right)$. If$E{\left|\epsilon \right|}^p<\infty$for some$p\ge 2$, then

$$\underset{\min \left(r,n\right)\to \infty }{\lim }\underset{v\in M}{\sup }E{\left|\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){\epsilon }_i^{\left(t\right)}}}\right|}^p=0.$$

(27)

Proof. 

Using the notations in the proof of Lemma 4 and by $C_p$ inequality (let $\left\{X_i,i\ge 1\right\}$ be a random sequence, then $E{\left|{\displaystyle \sum _{i=1}^n{X}_i}\right|}^p\le n^{p-1}{\displaystyle \sum _{i=1}^{n}E{\left|X_i\right|}^p}$ for $p>1$), one gets that

$$\begin{array}{cc}\hfill E{\left|\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){\epsilon }_i^{\left(t\right)}}}\right|}^p& \le 2^{p-1}\left(E{\left|\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}}}\right|}^p+E{\left|\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){{\epsilon }^{″}}_i^{\left(t\right)}}}\right|}^p\right)\hfill \\ & =:2^{p-1}\left(G_{r,n}^{\left(1\right)}+G_{r,n}^{\left(2\right)}\right).\hfill \end{array}$$

(28)

For every $s>p\ge 2$, by Lemma 2, Lemma 3, and $E{\left|\epsilon \right|}^p<\infty$, we derived that

$$\begin{array}{cc}\hfill \underset{v\in M}{\sup }{G}_{r,n}^{\left(1\right)}& \le {\left(\underset{v\in M}{\sup }E{\left|\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}}}\right|}^s\right)}^{p/s}\hfill \\ & \le C{\left[{\left(\frac{1}{r}\right)}^s\left(\underset{v\in M}{\sup }E{\left|{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{c}_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}}}\right|}^s\right)\right]}^{p/s}\hfill \\ & \le C{\left(\frac{1}{r}\right)}^p(\underset{v\in M}{\sup }{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left|c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}\right|}^s}}{+{\left[\underset{v\in M}{\sup }\left({\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left(c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}\right)}^2}}\right)\right]}^{s/2})}^{p/s}\hfill \\ & \le C{\left(\frac{1}{r}\right)}^p{\left(r^{s/p}{n}^{-\gamma \left(s-1\right)}+r^{s/2}{n}^{-\gamma s/2}\right)}^{p/s}\hfill \\ & \le C{r}^{-p/2}\hfill \end{array}$$

(29)

and

$$\begin{array}{ccc}\hfill \underset{v\in M}{\sup }{G}_{r,n}^{\left(2\right)}& \le \hfill & C{\left(\frac{1}{r}\right)}^p(\underset{v\in M}{\sup }{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left|c_{ni}\left(v\right){{\epsilon }^{″}}_i^{\left(t\right)}\right|}^p}}\hfill \\ & & +{\left[\underset{v\in M}{\sup }\left({\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^{n}E{\left(c_{ni}\left(v\right){{\epsilon }^{\prime }}_i^{\left(t\right)}\right)}^2}}\right)\right]}^{p/2})\hfill \\ & \le \hfill & C{\left(\frac{1}{r}\right)}^p\left(r{n}^{-\gamma \left(p-1\right)}+r^{p/2}{n}^{-\gamma p/2}\right)\hfill \\ & \le \hfill & C{r}^{-p/2}.\hfill \end{array}$$

(30)

Therefore, (27) follows from (28) through (30). □

5. Proofs of the Main Results

By (5), (8), and (10), we derive that

$${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta ={\tilde{T}}_n^{-2}\left(\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{\tilde{z}}_i{\tilde{e}}_i^{\left(t\right)}}}+{\displaystyle \sum _{i=1}^n{\tilde{z}}_i\stackrel{{\displaystyle ⌣}}{h}\left(x_i\right)}\right)$$

(31)

and

$${\widehat{\beta }}_{r,n}^{\left(WLS\right)}-\beta ={\tilde{U}}_n^{-2}\left(\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{\gamma }_i{\tilde{z}}_i{\tilde{e}}_i^{\left(t\right)}}}+{\displaystyle \sum _{i=1}^n{\gamma }_i{\tilde{z}}_i\stackrel{{\displaystyle ⌣}}{h}\left(x_i\right)}\right),$$

(32)

where ${\tilde{e}}_j^{\left(k\right)}=e_j^{\left(k\right)}-\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x_j\right)e_i^{\left(t\right)}}}$, $\stackrel{{\displaystyle ⌣}}{h}\left(x\right)=h\left(x\right)-{\displaystyle \sum _{j=1}^n{W}_{nj}\left(x\right)h\left(x_j\right)}$ and $e_i^{\left(t\right)}={\sigma }_i{\epsilon }_i^{\left(t\right)}$, $1\le k\le r$, $1\le j\le n$.

Proof of Theorem 1.

We only need to prove (16) since the proof of (15) is analogous. By (32), we can get that

$$\begin{array}{cc}\hfill {\widehat{\beta }}_{r,n}^{\left(WLS\right)}-\beta & ={\tilde{U}}_n^{-2}\left[\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{\gamma }_i{\tilde{z}}_i{\sigma }_i{\epsilon }_i^{\left(t\right)}}}-{\displaystyle \sum _{j=1}^n{\gamma }_j{\tilde{z}}_j\left(\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x_j\right){\sigma }_i{\epsilon }_i^{(t)}}}\right)}+{\displaystyle \sum _{i=1}^n{\gamma }_i{\tilde{z}}_i\stackrel{{\displaystyle ⌣}}{h}(x_i)}\right]\hfill \\ & =:I_{r,n}^{\left(1\right)}-I_{r,n}^{\left(2\right)}+I_{r,n}^{\left(3\right)}\hfill \\ & \le \left|I_{r,n}^{\left(1\right)}\right|+\left|I_{r,n}^{\left(2\right)}\right|+\left|I_{r,n}^{\left(3\right)}\right|.\hfill \end{array}$$

(33)

Observe that $I_{r,n}^{\left(1\right)}=\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n\left({\tilde{U}}_n^{-2}{\gamma }_i{\sigma }_i{\tilde{z}}_i\right){\epsilon }_i^{\left(t\right)}}}\triangleq \frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{a}_{ni}{\epsilon }_i^{\left(t\right)}}}$. Hence, it follows from $\left(C_1\right)$(i) and (ii) and (14) that

$$\underset{1\le i\le n}{\max }\left|a_{ni}\right|\le C\underset{1\le i\le n}{\max }\left|{\gamma }_i{\tilde{z}}_i\right|{\tilde{U}}_n^{-1}\cdot {\tilde{U}}_n^{-1}=O\left(n^{-1/2}\right)$$

(34)

and

$${\displaystyle \sum _{i=1}^n\left|a_{ni}\right|\le C{\displaystyle \sum _{i=1}^n\left|{\gamma }_i{\tilde{z}}_i\right|{\tilde{U}}_n^{-2}=O\left(1\right)}}.$$

(35)

Thus, by Lemma 4, we have

$$\left|I_{r,n}^{\left(1\right)}\right|\stackrel{a.s.}{\to }0$$

(36)

as $\min \left(r,n\right)\to \infty$. Note that

$$I_{r,n}^{\left(2\right)}=\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n\left({\displaystyle \sum _{j=1}^n{\tilde{U}}_n^{-2}{\gamma }_j{\sigma }_i{\tilde{z}}_j{W}_{ni}\left(x_j\right)}\right){\epsilon }_i^{\left(t\right)}}}=:\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{a^{\prime }}_{ni}{\epsilon }_i^{\left(t\right)}}}.$$

Hence, it follows from $\left(C_1\right)$(ii), $\left(C_2\right)$, and (14) that

$$\underset{1\le i\le n}{\max }\left|{a^{\prime }}_{ni}\right|\le C\underset{i\ge 1,x\in M}{\sup }\left|W_{ni}\left(x\right)\right|\cdot {\displaystyle \sum _{i=1}^n\left|{\gamma }_j{\tilde{z}}_j\right|{\tilde{U}}_n^{-2}}=O\left(n^{-\alpha }\right)$$

(37)

and

$${\displaystyle \sum _{i=1}^n\left|{a^{\prime }}_{ni}\right|\le C\underset{x\in M}{\sup }{\displaystyle \sum _{i=1}^n\left|W_{ni}\left(x\right)\right|}\cdot {\displaystyle \sum _{i=1}^n\left|{\gamma }_j{\tilde{z}}_j\right|{\tilde{U}}^{-2}}=O\left(1\right)},$$

(38)

where $\alpha$ is the same as that in $\left(C_2\right)$(ii).

Thus, by Lemma 4, one can get that

$$\left|I_{r,n}^{\left(2\right)}\right|\stackrel{a.s.}{\to }0$$

(39)

as $\min \left(r,n\right)\to \infty$. By (14), we derive that

$$\left|I_{r,n}^{\left(3\right)}\right|\le \underset{x\in M}{\sup }\left|\stackrel{{\displaystyle ⌣}}{h}\left(x\right)\right|{\displaystyle \sum _{i=1}^n\left|{\gamma }_i{z}_i\right|/{\tilde{U}}_n^2}\le C\underset{x\in M}{\sup }\left|\stackrel{{\displaystyle ⌣}}{h}\left(x\right)\right|.$$

(40)

By $\left(C_1\right)$(iii), $\left(C_2\right)$(i), and $\left(C_3\right)$, we obtain that

$$\begin{array}{ccc}\hfill \underset{x\in M}{\sup }\left|\stackrel{{\displaystyle ⌣}}{h}\left(x\right)\right|& \le \hfill & \underset{x\in M}{\sup }\left|{\displaystyle \sum _{j=1}^n{W}_{nj}\left(x\right)-1}\right|\left|h\left(x\right)\right|+\underset{x\in M}{\sup }{\displaystyle \sum _{j=1}^n\left|W_{nj}\left(x\right)\right|}\left|h\left(x\right)-h\left(x_j\right)\right|\hfill \\ & \le \hfill & \underset{x\in M}{\sup }\left|{\displaystyle \sum _{j=1}^n{W}_{nj}\left(x\right)-1}\right|\left|h\left(x\right)\right|\hfill \\ & & +\underset{x\in M}{\sup }{\displaystyle \sum _{j=1}^n\left|W_{nj}\left(x\right)\right|}\left|h\left(x\right)-h\left(x_j\right)\right|I\left(\Vert x-x_j\Vert >\delta \right)\hfill \\ & & +\underset{x\in M}{\sup }{\displaystyle \sum _{j=1}^n\left|W_{nj}\left(x\right)\right|}\left|h\left(x\right)-h\left(x_j\right)\right|I\left(\Vert x-x_j\Vert \le \delta \right)\hfill \\ & =\hfill & o\left(1\right).\hfill \end{array}$$

(41)

Thus, by (40) and (41), we have

$$\left|I_{r,n}^{\left(3\right)}\right|\to 0$$

(42)

as $\min \left(r,n\right)\to \infty$. Therefore, (16) follows from (33), (36), (39), and (42). □

Proof of Theorem 2.

We only need to prove (18) since the proof of (17) is analogous. In light of (12), we have

$$\begin{array}{cc}\hfill {\tilde{h}}_{r,n}\left(x\right)-h\left(x\right)& =\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x_i\right)}\left(z_i\beta +h\left(x_i\right)+{\sigma }_i{\epsilon }_i^{\left(t\right)}-z_i{\widehat{\beta }}_{r,n}^{\left(WLS\right)}\right)}-h\left(x\right)\hfill \\ & =\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right)z_i}\left(\beta -{\widehat{\beta }}_{r,n}^{\left(WLS\right)}\right)}-\stackrel{{\displaystyle ⌣}}{h}\left(x\right)+\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right){\sigma }_i}{\epsilon }_i^{\left(t\right)}}.\hfill \end{array}$$

(43)

Hence,

$$\begin{array}{ccc}\hfill \underset{x\in M}{\sup }\left|{\tilde{h}}_{r,n}\left(x\right)-h\left(x\right)\right|& \le \hfill & \underset{x\in M}{\sup }\left|{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right)z_i}\right|\left|\beta -{\widehat{\beta }}_{r,n}^{\left(WLS\right)}\right|+\underset{x\in M}{\sup }\left|\stackrel{{\displaystyle ⌣}}{h}\left(x\right)\right|\hfill \\ & & +\underset{x\in M}{\sup }\left|\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right){\sigma }_i}{\epsilon }_i^{\left(t\right)}}\right|\hfill \\ & =\hfill & :J_{r,n}^{\left(1\right)}+J_{r,n}^{\left(2\right)}+J_{r,n}^{\left(3\right)}.\hfill \end{array}$$

(44)

By (16) and $\left(C_4\right)$, we have

$$J_{r,n}^{\left(1\right)}\stackrel{a.s.}{\to }0$$

(45)

as $\min \left(r,n\right)\to \infty$. From (41), it follows that

$$J_{r,n}^{\left(2\right)}\to 0$$

(46)

as $\min \left(r,n\right)\to \infty$. By $\left(C_1\right)$(ii) and $\left(C_2\right)$, we can get that

$$\underset{i\ge 1,x\in M}{\sup }\left|W_{ni}\left(x\right){\sigma }_i\right|\le C\underset{i\ge 1,x\in M}{\sup }\left|W_{ni}\left(x\right)\right|=O\left(n^{-\alpha }\right),$$

and

$$\underset{x\in M}{\sup }{\displaystyle \sum _{i=1}^n\left|W_{ni}\left(x\right){\sigma }_i\right|}\le C\underset{x\in M}{\sup }{\displaystyle \sum _{i=1}^n\left|W_{ni}\left(x\right)\right|}=O\left(1\right).$$

Hence, from Lemma 4, it follows that

$$J_{r,n}^{\left(3\right)}\stackrel{a.s.}{\to }0.$$

(47)

as $\min \left(r,n\right)\to \infty$. Therefore, (18) follows from (44) through (47). □

Proof of Theorem 3.

We only need to prove (20) since the proof of (19) is analogous. By (33), we have

$$\begin{array}{cc}\hfill {\widehat{\beta }}_{r,n}^{\left(WLS\right)}-\beta & ={\tilde{U}}_n^{-2}\left[\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{\gamma }_i{\tilde{z}}_i{\sigma }_i{\epsilon }_i^{\left(t\right)}}}-{\displaystyle \sum _{j=1}^n{\gamma }_j{\tilde{z}}_j\left(\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x_j\right){\sigma }_i{\epsilon }_i^{(t)}}}\right)}+{\displaystyle \sum _{i=1}^n{\gamma }_i{\tilde{z}}_i\stackrel{{\displaystyle ⌣}}{h}(x_i)}\right]\hfill \\ & =:I_{r,n}^{\left(1\right)}-I_{r,n}^{\left(2\right)}+I_{r,n}^{\left(3\right)}.\hfill \end{array}$$

Hence, it follows by $C_p$ inequality that

$$E{\left|{\widehat{\beta }}_{r,n}^{\left(WLS\right)}-\beta \right|}^p\le 3^{p-1}\left(E{\left|I_{r,n}^{\left(1\right)}\right|}^p+E{\left|I_{r,n}^{\left(2\right)}\right|}^p+E{\left|I_{r,n}^{\left(3\right)}\right|}^p\right).$$

(48)

The rest of the proof is similar to the proof of (16), so we omitted the details here. □

Proof of Theorem 4.

We only need to prove (22) since the proof of (21) is analogous. By (43), we derive that

$$\begin{array}{cc}\hfill {\tilde{h}}_{r,n}\left(x\right)-h\left(x\right)& =\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right)}\left(z_i\beta +h\left(x\right)+{\sigma }_i{\epsilon }_i^{\left(t\right)}-z_i{\widehat{\beta }}_{r,n}^{\left(WLS\right)}\right)}-h\left(x\right)\hfill \\ & =\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right)z_i}\left(\beta -{\widehat{\beta }}_{r,n}^{\left(WLS\right)}\right)}-\stackrel{{\displaystyle ⌣}}{h}\left(x\right)+\frac{1}{r}{\displaystyle \sum _{t=1}^r{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right){\sigma }_i}{\epsilon }_i^{\left(t\right)}}\hfill \\ & =:J_{r,n}^{\left(1\right)}-J_{r,n}^{\left(2\right)}+J_{r,n}^{\left(3\right)}.\hfill \end{array}$$

Hence, by $C_p$ inequality, we derive that

$$E{\left|{\tilde{h}}_{r,n}\left(x\right)-h\left(x\right)\right|}^p\le 3^{p-1}\left(E{\left|J_{r,n}^{\left(1\right)}\right|}^p+E{\left|J_{r,n}^{\left(2\right)}\right|}^p+E{\left|J_{r,n}^{\left(3\right)}\right|}^p\right).$$

(49)

Since ${\left|J_{r,n}^{\left(1\right)}\right|}^p\le {\left(\underset{x\in M}{\sup }\left|{\displaystyle \sum _{i=1}^n{W}_{ni}\left(x\right)z_i}\right|\right)}^p{\left|\beta -{\widehat{\beta }}_{r,n}^{\left(WLS\right)}\right|}^p$, together with (20) and $\left(C_4\right)$, we can get that

$$\underset{\min \left(r,n\right)\to \infty }{\lim }\underset{x\in M}{\sup }E{\left|J_{r,n}^{\left(1\right)}\right|}^p=0.$$

(50)

From (41), it follows that

$$\underset{\min \left(r,n\right)\to \infty }{\lim }\underset{x\in M}{\sup }E{\left|J_{r,n}^{\left(2\right)}\right|}^p=0.$$

(51)

By $\left(C_1\right)$(ii) and $\left(C_2\right)$, we can get that

$$\underset{i\ge 1,x\in M}{\sup }\left|W_{ni}\left(x\right){\sigma }_i\right|\le C\underset{i\ge 1,x\in M}{\sup }\left|W_{ni}\left(x\right)\right|=O\left(n^{-\alpha }\right),$$

and

$$\underset{x\in M}{\sup }{\displaystyle \sum _{i=1}^n\left|W_{ni}\left(x\right){\sigma }_i\right|}\le C\underset{x\in M}{\sup }{\displaystyle \sum _{i=1}^n\left|W_{ni}\left(x\right)\right|}=O\left(1\right).$$

Hence, by Lemma 5, one can get that

$$\underset{\min \left(r,n\right)\to \infty }{\lim }\underset{x\in M}{\sup }E{\left|J_{r,n}^{\left(3\right)}\right|}^p=0.$$

(52)

Therefore, (22) follows from (49) through (52). □

6. Numerical Simulations

In this section, we will verify the validity of the theoretical results by two simulations.

6.1. Simulation 1

We will simulate a partially linear model

$$y_i^{\left(t\right)}=z_i\beta +h\left(x_i\right)+{\sigma }_i{\epsilon }_i^{\left(t\right)},r=n/2,1\le i\le n,$$

(53)

where $\beta =2.5$, $h\left(x\right)=\sin \left(2\pi x\right)$, $z_i={\left(-1\right)}^i\cdot \frac{i}{n}$, ${\sigma }_i=1$, $1\le i\le n$, and random errors $\left\{{\epsilon }_i^{\left(t\right)}\right\}$ have the common distribution as that of $\left\{Z_n\right\}$ in Example 1 of Section 1. Then, $\left\{{\epsilon }_i^{\left(t\right)}\right\}$ is a ${\rho }^{-}$-mixing sequence, and it is neither NA nor ${\rho }^{*}$-mixing.

In particular, we take the weight function $W_{ni}(\cdot )$ as the following nearest neighbor weight function (see [11,35]). Without loss of generality, denote $M=\left[0,1\right]$ and $x_i=\frac{i}{n}$ $\left(1\le i\le n\right)$. For each $x\in M$, we rewrite

$$\left|x_1-x\right|,\left|x_2-x\right|,\cdots ,\left|x_n-x\right|$$

as follows:

$$\left|x_{R_1(x)}-x\right|\le \left|x_{R_2\left(x\right)}-x\right|\le ,\cdots ,\le \left|x_{R_n\left(x\right)}-x\right|.$$

Take $k_n=\left[n^{0.8}\right]$ and define the nearest neighbor weight function as

$$W_{ni}\left(x\right)=\{\begin{array}{l}\frac{1}{k_n},\mathrm{if}\left|x_i-x\right|\le \left|x_{R_{k_n}}-x\right|,\\ 0,\mathrm{else}.\end{array}$$

where the sample sizes are taken as $n=100$, $600$, $1200$, $1900$, $2700$, and $3600$ and the points $x$ are taken as $x=0.2$, $0.4$, $0.6$, and $0.8$, respectively. We compute ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ and ${\widehat{h}}_{r,n}(x)-h\left(x\right)$ for $1000$ times, respectively. The boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ are provided in Figure 1, Figure 2, Figure 3 and Figure 4, the violin plots of ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ are provided in Figure 5, Figure 6, Figure 7 and Figure 8, the curves of $h\left(x\right)$ and ${\widehat{h}}_{r,n}\left(x\right)$ are provided in Figure 9, and the mean squared errors (MSE) of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}$ and ${\widehat{h}}_{r,n}\left(x\right)$ are presented in Table 1 and Table 2, respectively.

Figure 1. Boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ with $\beta =2.5$ and $x=0.2$.

Figure 2. Boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ with $\beta =2.5$ and $x=0.4$.

Figure 3. Boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ with $\beta =2.5$ and $x=0.6$.

Figure 4. Boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ with $\beta =2.5$ and $x=0.8$.

Figure 5. Violin plots of ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ with $\beta =2.5$ and $x=0.2$.

Figure 6. Violin plots of ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ with $\beta =2.5$ and $x=0.4$.

Figure 7. Violin plots of ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ with $\beta =2.5$ and $x=0.6$.

Figure 8. Violin plots of ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ with $\beta =2.5$ and $x=0.8$.

Figure 9. Curves of $h\left(x\right)=\sin \left(2\pi x\right)$ and ${\widehat{h}}_{r,n}\left(x\right)$ with $\beta =2.5$ and $n=1200$.

Table 1. The MSEs of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}$ with $\beta =2.5$ and $h\left(x\right)=\sin \left(2\pi x\right)$.

$\mathit{x}$	$\mathit{n}=100$	$\mathit{n}=600$	$\mathit{n}=1200$	$\mathit{n}=1900$	$\mathit{n}=2700$	$\mathit{n}=3600$
0.2	0.029564	0.0040429	0.0027393	0.0017879	0.0011778	0.00096205
0.4	0.027563	0.0048107	0.002557	0.0013618	0.0012225	0.00083697
0.6	0.032418	0.0045437	0.0030247	0.0017816	0.001016	0.0007695
0.8	0.026715	0.0049648	0.0026033	0.0014911	0.00099622	0.00083525

Table 2. The MSEs of ${\widehat{h}}_{r,n}\left(x\right)$ with $\beta =2.5$ and $h\left(x\right)=\sin \left(2\pi x\right)$.

$\mathit{x}$	$\mathit{n}=100$	$\mathit{n}=600$	$\mathit{n}=1200$	$\mathit{n}=1900$	$\mathit{n}=2700$	$\mathit{n}=3600$
0.2	0.072899	0.0132846	0.0064122	0.00450755	0.00408198	0.00309342
0.4	0.064407	0.012605	0.0061076	0.0040173	0.0037297	0.0027017
0.6	0.0651945	0.01143844	0.0061647	0.0048246	0.0031893	0.002914
0.8	0.067254	0.0134688	0.0701644	0.00515616	0.00397691	0.0030646

6.2. Simulation 2

We will simulate a partially linear model

$$y_i^{\left(t\right)}=z_i\beta +h\left(x_i\right)+{\sigma }_i{\epsilon }_i^{\left(t\right)},r=n/2,1\le i\le n,$$

(54)

where $\beta =3.5$, $h\left(x\right)=\cos \left(\pi x\right)$, $z_i={\left(-1\right)}^i\cdot \frac{i}{n}$, ${\sigma }_i=1$, $1\le i\le n$, and random errors $\left\{{\epsilon }_i^{\left(t\right)}\right\}$ have the same distribution as $\left\{Z_n\right\}$ in Example 1 of Section 1. Then, $\left\{{\epsilon }_i^{\left(t\right)}\right\}$ is a ${\rho }^{-}$-mixing sequence, and it is neither NA nor ${\rho }^{*}$-mixing.

Using the same estimating methods as model (53), we compute ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ and ${\widehat{h}}_{r,n}(x)-h\left(x\right)$ for $1000$ times in model (54) under different values of $n$, respectively. The boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ are provided in Figure 10, Figure 11, Figure 12 and Figure 13, the violin plots of ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ are provided in Figure 14, Figure 15, Figure 16 and Figure 17, the curves of $h\left(x\right)$ and ${\widehat{h}}_{r,n}\left(x\right)$ are provided in Figure 18, and the MSEs of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}$ and ${\widehat{h}}_{r,n}\left(x\right)$ are presented in Table 3 and Table 4, respectively.

Figure 10. Boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ with $\beta =3.5$ and $x=0.2$.

Figure 11. Boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ with $\beta =3.5$ and $x=0.4$.

Figure 12. Boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ with $\beta =3.5$ and $x=0.6$.

Figure 13. Boxplots of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ with $\beta =3.5$ and $x=0.8$.

Figure 14. Violin plots of ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ with $\beta =3.5$ and $x=0.2$.

Figure 15. Violin plots of ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ with $\beta =3.5$ and $x=0.4$.

Figure 16. Violin plots of ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ with $\beta =3.5$ and $x=0.6$.

Figure 17. Violin plots ${\widehat{h}}_{r,n}\left(x\right)-h\left(x\right)$ with $\beta =3.5$ and $x=0.8$.

Figure 18. Curves of $h\left(x\right)=\cos \left(\pi x\right)$ and ${\widehat{h}}_{r,n}\left(x\right)$ with $\beta =3.5$ and $n=1200$.

Table 3. The MSEs of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}$ with $\beta =3.5$ and $h\left(x\right)=\cos \left(\pi x\right)$.

$\mathit{x}$	$\mathit{n}=100$	$\mathit{n}=600$	$\mathit{n}=1200$	$\mathit{n}=1900$	$\mathit{n}=2700$	$\mathit{n}=3600$
0.2	0.028735	0.0046935	0.002547	0.0017209	0.0010944	0.00085433
0.4	0.032703	0.0053621	0.0022595	0.0015934	0.001206	0.0008074
0.6	0.027853	0.0048229	0.0025756	0.0014096	0.0011848	0.00084765
0.8	0.03042	0.004848	0.002352	0.0014649	0.0011824	0.00084588

Table 4. The MSEs of ${\widehat{h}}_{r,n}\left(x\right)$ with $\beta =3.5$ and $h\left(x\right)=\cos \left(\pi x\right)$.

$\mathit{x}$	$\mathit{n}=100$	$\mathit{n}=600$	$\mathit{n}=1200$	$\mathit{n}=1900$	$\mathit{n}=2700$	$\mathit{n}=3600$
0.2	0.025536	0.0068733	0.0035259	0.0025282	0.0019082	0.0015027
0.4	0.029859	0.0061945	0.0032347	0.0022321	0.002002	0.0016778
0.6	0.02415	0.005475	0.0040859	0.0026873	0.0017436	0.0014589
0.8	0.027164	0.0055904	0.0036775	0.0024326	0.0017396	0.0014935

It can be seen from Figure 1, Figure 2, Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8 and Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15, Figure 16 and Figure 17 that regardless of the values of $x$, ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ and ${\widehat{h}}_{r,n}(x)-h\left(x\right)$ fluctuate to zero and the ranges of ${\widehat{\beta }}_{r,n}^{\left(LS\right)}-\beta$ and ${\widehat{h}}_{r,n}(x)-h\left(x\right)$ decrease as $n$ increases. From Table 1, Table 2, Table 3 and Table 4, one can see that regardless of the values of $x$, the MSEs decrease gradually as $n$ increases. Hence the estimators get closer and closer to their real values as $n$ increases. Figure 9 and Figure 18 further show that the estimators of function $h\left(x\right)$ have good effects. The simulation results directly reflect our theoretical results.

7. Conclusions

In this paper, we mainly investigated the asymptotic properties of the estimators for the unknown parameter and non-parametric component in the heteroscedastic partially linear model (1). A lot of authors have derived the asymptotic properties of the estimators in partially linear models with independent random errors (see [4,5,6,8,33]). However, in many applications, the random errors are not independent. Here, we assumed that the random errors are ${\rho }^{-}$-mixing, which includes independent, NA, and ${\rho }^{*}$-mixing random variables as special cases. Under some suitable conditions, the strong consistency and $p$-th $\left(p>0\right)$ mean consistency of the LS estimator and WLS estimator for the unknown parameter $\beta$ were investigated, and the strong consistency and $p$-th $\left(p>0\right)$ mean consistency of the estimators for the non-parametric component $h(\cdot )$ were also studied. The results obtained in this paper include the corresponding ones of independent random errors, NA random errors (see [16]), and ${\rho }^{*}$-mixing random errors as special cases. Furthermore, for the model (1), we carried out simulations to study the numerical performance of the asymptotic properties for the estimators of the unknown parameter and non-parametric component for the first time. ${\rho }^{-}$-mixing sequences are widely used dependent sequences. Therefore, investigating the limit properties of the estimators in regression models under ${\rho }^{-}$-mixing errors in future studies is an interesting subject.