Asymptotic Distributions of M-Estimates for Parameters of Multivariate Time Series with Strong Mixing Property ^†

Abstract

The publication is devoted to studying asymptotic properties of statistical estimates of the distribution parameters $u\in R^q$ of a multidimensional random stationary time series $z_t\in R^m,t\in \mathbb{Z}$ satisfying the strong mixing conditions. We consider estimates ${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)$, ${\overline{z}}_n={\left(z_1^{\mathrm{T}},\dots ,z_n^{\mathrm{T}}\right)}^{\mathrm{T}}\in R^{mn}$ that provide in asymptotic $n\to \infty$ the maximum values for some objective functions $Q_n\left({\overline{z}}_n;u\right)$, which have properties similar to the well-known property of local asymptotic normality. These estimates are constructed by solving the equations ${\delta }_n\left({\overline{z}}_n;u\right)=0$, where ${\delta }_n\left({\overline{z}}_n;u\right)$ are arbitrary functions for which ${\delta }_n\left({\overline{z}}_n;u\right)-\underset{h}{\mathrm{grad}}{Q}_n\left({\overline{z}}_n;u+n^{-1/2}h\right)\to 0$ $\left(n\to \infty \right)$ in $P_{n,u}\left({\overline{z}}_n\right)$-probability uniformly on $u\in U$, were $U$ is compact in $R^q$. In many cases, the estimates ${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)$ have the same asymptotic properties as well-known M-estimates defined by equations ${\widehat{u}}_n^Q\left({\overline{z}}_n\right)$ $=\underset{u\in U}{arg\hspace{0.17em}max}{Q}_n\left({\overline{z}}_n;u\right)$ but often can be much simpler computationally. We consider an algorithmic method for constructing estimates ${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)$, which is similar to the accumulation method first proposed by R. Fischer and rigorously developed by L. Le Cam. The main theoretical result of the article is the proof of the theorem, in which conditions of the asymptotic normality of estimates ${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)$ are formulated, and the expression is proposed for their matrix of asymptotic mean-square deviations $\underset{n\to \infty }{\lim }n{\mathrm{E}}_{n,u}\left\{\left({\widehat{u}}^{\delta }\left({\overline{z}}_n\right)-u\right){\left({\widehat{u}}^{\delta }\left({\overline{z}}_n\right)-u\right)}^{\mathrm{T}}\right\}$.

1. Introduction. Methods of Construction Asymptotically Efficient Estimates for Parameters of Stationary Time Series

In applications of mathematical statistics to modern problems of data analysis in natural science and technology, it is often impossible to use the classical observation models in the form of a sequence of independent identically distributed random variables (i.i.d. model). As a rule, the i.i.d. model does not provide sufficient accuracy of statistical inferences about the unknown parameters of the investigated physical processes, distorted by noise, if both of them are stationary random processes.

Thus, it is important to generalize the classical results of the statistical theory of parameter estimation, developed for the i.i.d. model, in order to apply them to actual practical problems in the analysis of real physical processes.

In modern systems for analyzing physical wave fields, a large number of parameters are simultaneously measured, and many sensors are used to improve the accuracy of the analysis. That is, multidimensional time series $z_t\in R^m,t\in \mathbb{Z}$ are subjected to statistical processing, and vector parameters are estimated as a result of this processing.

For many statistical models of multivariate time series, it is impossible to synthesize statistically efficient estimates ${\widehat{u}}_n^{\mathrm{ef}}\left({\overline{z}}_n\right)$ of vector parameters $u$ for which the standard deviation matrices are minimal for any finite size n of observations and are equal to the inverse Fisher information matrix:

$$K_n^{\mathrm{ef}}\left(u\right)={\mathrm{E}}_{n,u}\left\{\left({\widehat{u}}_n^{\mathrm{ef}}\left({\overline{z}}_n\right)-u\right){\left({\widehat{u}}_n^{\mathrm{ef}}\left({\overline{z}}_n\right)-u\right)}^{\mathrm{T}}\right\}=J_n^{-1}\left(u\right),$$

(1)

where $J_n\left(u\right)={\displaystyle \underset{R^{mn}}{\int }\left({\nabla }_u{p}_{z,n}\left({\overline{x}}_n;u\right)\right){\left({\nabla }_u{p}_{z,n}\left({\overline{x}}_n;u\right)\right)}^{\mathrm{T}}{p}_{z,n}^{-1}\left({\overline{x}}_n;u\right)d{\overline{x}}_n}$;

$$\begin{gathered} {\overline{x}}_n={\left(x_1^{\mathrm{T}},\dots ,x_n^{\mathrm{T}}\right)}^{\mathrm{T}}\in R^{mn};{\nabla }_u{p}_z\left({\overline{x}}_n;u\right) \\ ={\left(\frac{\partial }{\partial u_k}{p}_z\left({\overline{x}}_n;u\right),k\in \overline{1,q}\right)}^{\mathrm{T}}; \end{gathered}$$

$p_z\left({\overline{x}}_n;u\right)$ is the probability density of the observations ${\overline{z}}_n$.

At the same time, asymptotically efficient (AE) estimates ${\widehat{u}}_n^{\mathrm{ae}}\left({\overline{z}}_n\right)$ can be constructed for a wide class of multivariate time series with interdependent elements $z_t$ possessing a strong mixing property [1]. For AE-estimates, equality (1) is attained asymptotically for $n\to \infty$:

$$K^{\mathrm{ae}}\left(u\right)=\underset{n\to \infty }{\lim }n{\mathrm{E}}_{n,u}\left\{\left({\widehat{u}}_n^{\mathrm{ae}}\left({\overline{z}}_n\right)-u\right){\left({\widehat{u}}_n^{\mathrm{ae}}\left({\overline{z}}_n\right)-u\right)}^T\right\}=\underset{n\to \infty }{\lim }n{J}_n^{-1}\left(u\right).$$

They can be found in the class $ℛ$ of regular estimates $\widehat{u}\left({\overline{z}}_n\right)$ for which the random quantities $\sqrt{n}\left(\widehat{u}\left({\overline{z}}_n\right)-u\right)$, $u\in U$ have limit distributions with finite second moments. This statement is one of the results of the extensive asymptotic theory of statistical inference for random time series, which is most fully presented in [2]. Fundamental results in this theory were obtained in the known publications [3,4,5,6]. In these books, sufficient conditions were established under which AE-estimates exist for many probabilistic models of random time series and continuous processes.

The main condition under which the AE-estimates can be constructed is the local asymptotic normality (LAN) of the likelihood ratio ${\mathrm{L}}_n\left({\overline{z}}_n\right)$ of observations ${\overline{z}}_n$ [3]. It means that the likelihood ratio of the observations ${\overline{z}}_n$ admits the following asymptotic expansion:

$$L_n\left({\overline{z}}_n\right)=ln\frac{p_{z,n}\left({\overline{z}}_n;u+n^{-1/2}h\right)}{p_{z,n}\left({\overline{z}}_n;u\right)}=h^{\mathrm{T}}{\Delta }_n\left({\overline{z}}_n;u\right)-\frac{1}{2}{h}^{\mathrm{T}}{\Gamma }_n\left(u\right)h+{\alpha }_n\left({\overline{z}}_n;u,h\right),$$

(2)

where $\underset{n\to \infty }{\lim }{\Gamma }_n\left(u\right)=\Gamma \left(u\right)=\underset{n\to \infty }{\lim }{n}^{-1}{J}_n^{}\left(u\right)$; ${\Delta }_n\left({\overline{z}}_n;u\right)\in R^q$ is a family of statistics for which probability distributions tend as $n\to \infty$ to the q-dimensional Gaussian distributions with the parameters $\left(0,\Gamma \left(u\right)\right)$ uniformly in $u\in U$; ${\alpha }_n\left({\overline{z}}_n;u,t\right)\to 0$ ($n\to \infty$) in $d{P}_n\left({\overline{z}}_n\right)$-probability uniformly in $u\in U$; $\left|h\right|<c$ where $c$ is any number.

Many publications, for example, [7,8,9,10,11,12,13,14], have been devoted to proving the LAN property for various probabilistic models of time series other than the i.i.d model. The results of research in this direction, obtained up to the end of the twentieth century, are summarized in the monograph [2]. It was shown that the LAN property is inherent in a wide class of multidimensional time series and continuous random processes.

The formulation of the LAN condition (2) largely determined the further development and practical applications of the asymptotic estimation theory. In the well-known monograph [6], it is shown that under the LAN condition, the maximum likelihood estimate belongs to the class $ℛ$ of regular statistical estimates and is an AE-estimate.

At the same time, using the decomposition (2) of the likelihood function of observations, new AE-estimates were constructed, which differ from the traditional maximum likelihood estimates and are computationally simpler. An elegant and, in many cases, the most computationally simple method for constructing AE-estimates, was proposed in [3,4]. It is based on R. Fisher’s [15] idea of “improving” the quality of some “simple” estimate to the quality of an AE-estimate. In mentioned publications, L. Le Cam showed that the AE-estimate can be obtained using the equation:

$${\widehat{u}}_n^{\mathrm{ae}}\left({\overline{z}}_n\right)=u_n^{\ast }\left({\overline{z}}_n\right)-n^{-1/2}{\Gamma }_n^{-1}\left(u_n^{\ast }\left({\overline{z}}_n\right)\right){\Delta }_n\left({\overline{z}}_n;u_n^{\ast }\left({\overline{z}}_n\right)\right),$$

(3)

where $u_n^{\ast }\left({\overline{z}}_n\right)$ is an arbitrary $\sqrt{n}$-consistent estimate of the parameter $u$ for which the quantities $\sqrt{n}\left(u^{\ast }\left({\overline{z}}_n\right)-u\right)$, $u\in U$, $n\in \mathbb{Z}$ have the property: for any $\epsilon >0$ there is $C_{\epsilon }>0$, such that $\underset{u\in U,n\in {\mathbb{Z}}^{+}}{\sup }\left[P_{n,u}\left\{\left|\sqrt{n}\left(u^{\ast }\left({\overline{z}}_n\right)-u\right)\right|>C_{\epsilon }\right\}\right]<\epsilon$.

Note that Equation (3) defines a whole class of AE-estimates, the quality of which is asymptotically equivalent to the quality of the ML-estimate, since ${\Delta }_n\left({\overline{z}}_n;u\right)$, ${\Gamma }_n\left(u\right)$ in the LAN expansion (2) and the $\sqrt{n}$-consistent estimate $u_n^{\ast }\left({\overline{z}}_n\right)$ are not unique functions. For this reason, in many practically important cases, formula (3) allows one to obtain AE-estimates, which are computationally much simpler than ML-estimates.

2. Construction of M-Estimates for Parameters of Stationary Time Series with Suitable Asymptotical Properties

The AE-estimates have some disadvantages from the point of view of practical applications. First, they can be synthesized only if the probability density $p_{n,z}\left({\overline{x}}_n;u\right)$ of the observations ${\overline{z}}_n$ is fully known. In practice, some important details of this density are often not fully defined. Only a certain class $\mathcal{K}$ is known to which this density belongs. Second, the quality of AE-estimates is often unstable to deviations of the actual density $p_{n,z}\left({\overline{x}}_n;u\right)$ from the assumed one for which they were synthesized. Even a small deviation from the expected density can lead to a significant loss in the accuracy of the AE-estimate.

In the publications [16,17], methods were developed for constructing estimates that are robust to changes in the distribution of observations, and in many applications, such robust estimates are preferable to AE-estimates. A robust estimate $\widehat{u}\left({\overline{z}}_n\right)$ is constructed by finding the global maximum of a certain objective function $Q_n\left({\overline{z}}_n;u\right)$(a criterion of estimation quality), which differs from likelihood function:

$$\widehat{u}\left({\overline{z}}_n\right)=\underset{u\in U}{arg max}{Q}_n\left({\overline{z}}_n;u\right).$$

(4)

In addition to robust estimates, estimates synthesized using Equation (4) arise in other problems of mathematical statistics. The examples include Bayesian estimation problems, estimation problems with interfering (nuisance) parameters, problems arising in the analysis of natural and economic dynamical systems.

The estimates obtained as the maxima of some objective functions $Q_n\left({\overline{z}}_n;u\right)$ were called “M-estimates”. Apart from books [16,17], they were considered in many other publications, for example, in [18,19]. In most of these publications, the M-estimates were constructed and analyzed for the i.i.d. model of random observations.

The authors are not aware of publications in which the asymptotic properties of M-estimates were studied with a sufficient level of mathematical rigor for multidimensional stationary random time series that have a strong mixing property. The authors are also unaware of publications devoted to the construction of computationally simple estimates that are asymptotically equivalent in quality to M-estimates.

In this paper, we consider an approach to solving these problems from the standpoint of view of the asymptotic theory of statistical inference [2], which is based on Le Cam’s concept of local asymptotically normality.

We suppose that random objective function $Q_n\left({\overline{z}}_n;u\right)$ is twice differentiable in $P_{n,u}$-probability with respect to components of the vector $u\in U$; that is, there exist the following family of vector statistics $d_n\left({\overline{z}}_n;u\right)$ and matrix function $F_n\left({\overline{z}}_n;u\right)$:

$$\begin{gathered} d_n\left({\overline{z}}_n;u\right)={\left(d_{n,k}\left({\overline{z}}_n;u\right)=\frac{\partial }{\partial u_k}{Q}_n\left({\overline{z}}_n;u\right),k\in \overline{1,q}\right)}^{\mathrm{T}}={\nabla }_u{Q}_n\left({\overline{z}}_n;u\right) \\ \in R^q, \\ {F}_n\left({\overline{z}}_n;u\right)=\left[\frac{\partial }{\partial u_l}{d}_{n,k}\left({\overline{z}}_n;u\right),k,l\in \overline{1,q}\right]={\Delta }_u{Q}_n\left({\overline{z}}_n;u\right)\in R^{q\times q}. \end{gathered}$$

(5)

In this case, the M-estimate (4) is one of the roots ${\tilde{u}}_n\left({\overline{z}}_n\right)$ of the following equation system with respect to the parameter $u$:

$$d_n\left({\overline{z}}_n;u\right)=0.$$

(6)

In this paper, we show how to find the estimate ${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)$, which is a root of the equation system (6), and, at the same time, it is an $\sqrt{n}$-consistent estimate of the parameter u. It is proved in Theorem 1 that under certain restrictions, such an estimate ${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)$ can be found using the algorithm

$${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)=u_n^{\ast }\left({\overline{z}}_n\right)-n^{-1/2}{\Phi }_n^{}\left(u_n^{\ast }\left({\overline{z}}_n\right)\right){\delta }_n\left({\overline{z}}_n;u_n^{\ast }\left({\overline{z}}_n\right)\right),$$

(7)

where ${\delta }_n\left({\overline{z}}_n;u\right)=n^{-1/2}{d}_n\left({\overline{z}}_n;u\right)$; ${\Phi }_n^{}\left(u\right)=$ $n^{-1}{\mathrm{E}}_u\left\{F_n\left({\overline{z}}_n;u\right)\right\}$; $u_n^{\ast }\left({\overline{z}}_n\right)$ is any $\sqrt{n}$-consis- tent estimate of the parameter u.

Conditions are formulated in Theorem 1 on the family of statistics ${\delta }_n\left({\overline{z}}_n;u\right)$ and the sequence of the matrix functions ${\Phi }_n^{}\left(u\right)$ that are sufficient for the asymptotic normality of the estimate (7): $\mathfrak{L}\left\{\sqrt{n}\left({\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)-u\right)\right\}\to \mathbb{N}\left(0,D\left(u\right)\right)$ $\left(n\to \infty \right)$, where the asymptotic covariance matrix $D\left(u\right)=\underset{n\to \infty }{\lim }n{\mathrm{E}}_u\left\{\left({\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)-u\right){\left({\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)-u\right)}^{\mathrm{T}}\right\}$ is equal to

$$D\left(u\right)={\Phi }^{-1}\left(u\right)\Psi \left(u\right){\Phi }^{-1}\left(u\right)\Psi \left(u\right)=\underset{n\to \infty }{\lim }{\mathrm{E}}_u\left\{{\delta }_n\left({\overline{z}}_n;u\right){\delta }_n^{\mathrm{T}}\left({\overline{z}}_n;u\right)\right\}\Phi \left(u\right)=\underset{n\to \infty }{\lim }{\Phi }_n\left(u\right).$$

The corollary of Theorem 1 describes a method for constructing another estimate ${\tilde{u}}_n^{\delta }\left({\overline{z}}_n\right)$ that has the same asymptotical distribution as the estimate (7) but does not require an auxiliary $\sqrt{n}$-consistent estimate $u_n^{\ast }\left({\overline{z}}_n\right)$.

Note that the statements of Theorem 1 and the corollary were formulated earlier in [20]. In our paper, the above statements are proved under more general assumptions, and simpler proofs are given.

Theorem 1.

A. There exists a$\sqrt{n}$-consistent estimate $u_n^{\ast }\left({\overline{z}}_n\right)$ of the parameter u.

B. Let the family of statistics${\delta }_n\left({\overline{z}}_n,u\right)\in R^m$,$u\in U$, and the sequence of positive definite symmetric$q\times q$-matrix functions${\Phi }_n\left(u\right)$satisfy the following constraints:

B1. For each value of the parameter$u\in U$, the sequence of statistics${\delta }_n\left({\overline{z}}_n,u\right)$is asymptotically normal with zero mean and the covariance matrix$\Psi \left(u\right)$:

$$\mathfrak{L}\left\{{\delta }_n\left({\overline{z}}_n,u\right)\right\}\to \mathcal{N}(0,\Psi \left(u\right))\left(n\to \infty \right)$$

where $\Psi \left(u\right)=\underset{n\to \infty }{\lim }{\mathrm{E}}_u\left\{{\delta }_n\left({\overline{z}}_n,u\right){\delta }_n^{\mathrm{T}}\left({\overline{z}}_n,u\right)\right\}.$

B2. For each value of the parameter$u\in U$, the following asymptotic expansion of the statistic${\delta }_n\left({\overline{z}}_n,u\right)$holds:

$${\delta }_n\left({\overline{z}}_n;u+n^{-1/2}h\right)={\delta }_n\left({\overline{z}}_n;u\right)+{\Phi }_n\left(u\right)h+{\beta }_n\left({\overline{z}}_n;u,h\right),\left|h\right|<cfor\forall c;$$

where$\underset{u\in U,\left|h\right|<c}{\sup }{P}_{n,u}\left\{\left|{\beta }_n\left({\overline{z}}_n;u,h\right)\right|>\epsilon \right\}\to 0\left(n\to \infty \right)$for any$\epsilon >0$;

$$\underset{n\in {\mathbb{Z}}^{+},u\in U}{\inf }\det {\Phi }_n\left(u\right)>d;\underset{n\to \infty }{\lim }\underset{u\in U}{\sup }\Vert {\Phi }_n^{-1}\left(u\right)-{\Phi }^{-1}\left(u\right)\Vert =0;\underset{u\in U}{\sup }\Vert {\Phi }^{-1}\left(u\right)\Vert <C;$$

${\Phi }_{}^{-1}\left(u\right)$is a continuous function of$u\in U$.

Then the following statement is true:

For any$\sqrt{n}$-consistent estimate$u_n^{\ast }\left({\overline{z}}_n\right)$of the parameter$u\in U$, the statistic

$${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)=u_n^{\ast }\left({\overline{z}}_n\right)-n^{-1/2}{\Phi }_n^{-1}\left(u_n^{\ast }\left({\overline{z}}_n\right)\right){\delta }_n\left({\overline{z}}_n;u_n^{\ast }\left({\overline{z}}_n\right)\right)$$

(8)

is the$\sqrt{n}$-consistent and asymptotically normal estimate of the parameter$u\in U$with the moments$\left(0,D\left(u\right)\right)$:

$$\mathfrak{L}\left\{\sqrt{n}\left({\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)-u\right)\right\}\to \mathbb{N}\left(0,D\left(u\right)\right)\left(n\to \infty \right)$$

where$D\left(u\right)$=${\Phi }_{}^{-1}\left(u\right)\Psi \left(u\right){\Phi }_{}^{-1}\left(u\right)$.

Corollary 1.

(a) Let, for any $n\in {\mathbb{Z}}^{+}$, a statistic ${\tilde{u}}_n^{\delta }\left({\overline{z}}_n\right)$ be the root of the equation ${\delta }_n\left({\overline{z}}_n;u\right)=0$ with respect to the parameter $u\in U$ with probability equal to 1.

(b) Let the statistic ${\tilde{u}}_n^{\delta }\left({\overline{z}}_n\right)$ also is a $\sqrt{n}$-consistent estimate of the parameter $u\in U$. Then the statistic ${\tilde{u}}_n^{\delta }\left({\overline{z}}_n\right)$ is asymptotically normal with the moments $\left(0,D\left(u\right)\right)$.

Remark 1.

(a) The statement similar to Statement (T1) of Theorem 1 was proved in [3,4] in the case when the objective function $Q_n\left({\overline{z}}_n;u\right)$ is the likelihood function of ${\overline{z}}_n$ having the LAN property (2). In this case ${\delta }_n\left({\overline{z}}_n;u\right)$ $\equiv$ ${\Delta }_n\left({\overline{z}}_n;u\right)$, the matrix function ${\Phi }_n^{}\left(u\right)$ $\equiv$ ${\Gamma }_n\left(u\right)$ and

$$\mathfrak{L}\left\{{\Delta }_n\left({\overline{z}}_n;u\right)\right\}\to \mathbb{N}\left(0,\Gamma \left(u\right)\right)\left(n\to \infty \right);\Gamma \left(u\right)=\underset{n\to \infty }{\lim }{n}^{-1}{J}_n^{}\left(u\right),$$

where $J_n\left(u\right)$ is the Fisher matrix. It follows from Theorem 1, that in this case

$$D\left(u\right)={\Gamma }_{}^{-1}\left(u\right)\Gamma \left(u\right){\Gamma }_{}^{-1}\left(u\right)={\Gamma }_{}^{-1}\left(u\right).$$

Consequently, the statistic

$${\widehat{u}}_n^{\Delta }\left({\overline{z}}_n\right)=u_n^{\ast }\left({\overline{z}}_n\right)-n^{-1/2}{\Gamma }_n^{-1}\left(u_n^{\ast }\left({\overline{z}}_n\right)\right){\Delta }_n\left({\overline{z}}_n;u_n^{\ast }\left({\overline{z}}_n\right)\right)$$

is asymptotically normal with the parameters$\left(0,\Gamma \left(u\right)\right)$, and hence, it is the asymptotically efficient estimate of the parameter u.

(b) It follows from the corollary of Theorem 1 that a statistic ${\tilde{u}}_n^{\Delta }\left({\overline{z}}_n\right)$, which has the property: ${\Delta }_n\left({\overline{z}}_n;{\tilde{u}}_n^{\Delta }\left({\overline{z}}_n\right)\right)=0$ with probability equal to one, and at the same time is a $\sqrt{n}$-consistent estimate of the parameter $u\in U$, is asymptotically normal with the moments $\left(0,\Gamma \left(u\right)\right)$. Consequently, the statistic ${\tilde{u}}_n^{\Delta }\left({\overline{z}}_n\right)$ is the asymptotically efficient estimate of the parameter $u\in U$.

Thus, Theorem 1 is, in some sense, an extension of Le Cam’s results to the case of an arbitrary objective function $Q_n\left({\overline{z}}_n;u\right)$ whose gradient satisfies conditions B1, B2 of Theorem 1.

3. Proof of Theorem 1

In the course of proving Theorem 1, we will omit, if it is obvious, the dependence of functional quantities on the observations ${\overline{z}}_n$ and sometimes denote their dependence on the parameter u by a subscript.

In these notations, the definition of the estimate ${\widehat{u}}_n\left({\overline{z}}_n\right)$ can be written as

$${\widehat{u}}_n=u_n^{\ast }-n^{-1/2}{\Phi }_n^{-1}\left(u_n^{\ast }\right){\delta }_n\left(u_n^{\ast }\right).$$

Then we can write the following chain of equalities:

$$\begin{gathered} \sqrt{n}\left({\widehat{u}}_n-u\right)=\sqrt{n}\left(u_n^{\ast }-u\right)-{\Phi }_n^{-1}\left(u_n^{\ast }\right){\delta }_n\left(u_n^{\ast }\right)= \\ =-{\Phi }_n^{-1}\left(u\right){\delta }_n\left(u\right)+\left[\sqrt{n}\left(u_n^{\ast }-u\right)-{\Phi }_n^{-1}\left(u_n^{\ast }\right){\delta }_n\left(u_n^{\ast }\right)+{\Phi }_n^{-1}\left(u\right){\delta }_n\left(u\right)\right]= \\ =-{\Phi }_n^{-1}\left(u\right){\delta }_n\left(u\right)+{\xi }_{n,u}\left(u_n^{\ast }\right), \end{gathered}$$

(9)

where ${\xi }_{n,u}\left(u_n^{\ast }\right)$ = $\sqrt{n}\left(u_n^{\ast }-u\right)+{\Phi }_n^{-1}\left(u_n^{\ast }\right)\left[-{\delta }_n\left(u_n^{\ast }\right)+{\delta }_n\left(u\right)\right]$. It follows from (9):

$${\Phi }_n^{}\left(u_n^{\ast }\right){\xi }_{n,u}\left(u_n^{\ast }\right)=-{\delta }_n\left(u_n^{\ast }\right)+{\delta }_n\left(u\right)+{\Phi }_n^{}\left(u_n^{\ast }\right)\sqrt{n}\left(u_n^{\ast }-u\right)={\rho }_{n,u}\left(u_n^{\ast }\right).$$

(10)

By denoting ${\tau }_{n,u}^{\ast }$ = $\sqrt{n}\left(u_n^{\ast }-u\right)$, we obtain from (10):

$${\delta }_n\left(u+{\tau }_{n,u}^{\ast }/\sqrt{n}\right)-{\delta }_n\left(u\right)={\Phi }_n^{}\left(u+{\tau }_{n,u}^{\ast }/\sqrt{n}\right){\tau }_{n,u}^{\ast }-{\rho }_{n,u}\left({\tau }_{n,u}^{\ast }\right),$$

(11)

where the random quantities ${\tau }_{n,u}^{\ast }$, $n\in {\mathbb{Z}}^{+}$, $u\in U$ have the property: for any $\epsilon >0$ there is ${С}_{\epsilon }>0$ such that $\underset{u\in U,n\in {\mathbb{Z}}^{+}}{\sup }\left[P_{n,u}\left\{\left|{\tau }_{n,u}^{\ast }\right|>{С}_{\epsilon }\right\}\right]<\epsilon$.

At the same time, from condition B2 of Theorem 1, we obtain:

$${\delta }_n\left({\overline{z}}_n;u+n^{-1/2}h\right)-{\delta }_n\left({\overline{z}}_n;u\right)={\Phi }_n\left(u\right)h+{\beta }_{n,u}\left({\overline{z}}_n;h\right),$$

(12)

where $\underset{u\in U,\left|h\right|<c}{\sup }{P}_{n,u}\left\{\left|{\beta }_{n,u}\left({\overline{z}}_n;h\right)\right|>\epsilon \right\}\to 0\left(n\to \infty \right)$.

The comparison Equations (11) and (12) allow us to prove the following Lemma.

Lemma 1.

Under the conditions of Theorem 1, the following convergences take place for any $\epsilon >0$:

(a) $\underset{n\to \infty }{\lim }\underset{u\in U}{\sup }{P}_{n,u}\left\{\left|{\rho }_{n,u}\left(u_n^{\ast }\right)\right|>\epsilon \right\}=0$, (b) $\underset{n\to \infty }{\lim }\underset{u\in U}{\sup }{P}_{n,u}\left\{\left|{\xi }_{n,u}\left(u_n^{\ast }\right)\right|>\epsilon \right\}=0$.

The proof of Lemma 1 is given in Section 5.

The following statement will be needed below.

Lemma 2.

Let some random variables${\phi }_n$and${\eta }_n$have the properties:

(a) $\underset{n\to \infty }{\lim }{\mathfrak{L}}_n\left\{{\phi }_n\right\}=\underset{n\to \infty }{\lim }{P}_n\left\{{\phi }_n<x\right\}=F\left(x\right)$; (b) for any $\epsilon >0$ $\underset{n\to \infty }{\lim }{P}_n\left\{\left|{\eta }_n\right|>\epsilon \right\}=0$.

Then$\underset{n\to \infty }{\lim }{\mathfrak{L}}_n\left\{{\phi }_n+{\eta }_n\right\}=\underset{n\to \infty }{\lim }{P}_n\left\{{\phi }_n+{\eta }_n<x\right\}=F\left(x\right)$.

The proof of Lemma 2 is quite simple, and we omit it.

Taking into account Equations (9)–(12) and statements of Lemmas 1 and 2, we can write the following equalities:

$$\mathfrak{L}\left\{\sqrt{n}\left({\widehat{u}}_n-u\right)\right\}=\underset{n\to \infty }{lim}\mathfrak{L}\left\{{\Phi }_n^{-1}\left(u\right){\delta }_n\left(u\right)+{\xi }_{u,n}\right\}=\underset{n\to \infty }{lim}\mathfrak{L}\left\{{\Phi }_n^{-1}\left(u\right){\delta }_n\left(u\right)\right\},$$

where the existence of the limits follows from conditions B1, B2 of Theorem 1. According to conditions B1 of Theorem 1, we have:

$\underset{n\to \infty }{lim}\mathfrak{L}\left\{{\delta }_n\left(u\right)\right\}=\mathbb{N}\left(0;\Psi \left(u\right)\right)$ where $\Psi \left(u\right)=\underset{n\to \infty }{\lim }{\mathrm{E}}_n\left\{{\delta }_n\left(u\right){\delta }_n^{\mathrm{T}}\left(u\right)\right\}$

Therefore:

$\underset{n\to \infty }{lim}\mathfrak{L}\left\{{\Phi }_n^{-1}\left(u\right){\delta }_n\left(u\right)\right\}$ = $\mathbb{N}\left(0;D\left(u\right)\right)$, where $D\left(u\right)={\Phi }_{}^{-1}\left(u\right)\Psi \left(u\right){\Phi }_{}^{-1}\left(u\right)$. □

4. Proof of Corollary

Under the conditions B1, B2 of Theorem 1, the statistic ${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)$ in Equation (8) is asymptotically normal with the moments $\left(0,D\left(u\right)\right)$ for any $\sqrt{n}$-consistent estimate $u_n^{\ast }\left({\overline{z}}_n\right)$. Consequently, due to condition (b) of the corollary, the statistic

$${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)={\tilde{u}}_n^{\delta }\left({\overline{z}}_n\right)+n^{-1/2}{\Phi }_n^{-1}\left({\tilde{u}}_n^{\delta }\left({\overline{z}}_n\right)\right){\delta }_n\left({\overline{z}}_n;{\tilde{u}}_n^{\delta }\left({\overline{z}}_n\right)\right)$$

is asymptotically normal with the moments $\left(0,D\left(u\right)\right)$.

But by virtue of condition (a) of the corollary, we have that ${\widehat{u}}_n^{\delta }\left({\overline{z}}_n\right)={\tilde{u}}_n^{\delta }\left({\overline{z}}_n\right)$ with probability equal to one. Hence, the statistic ${\tilde{u}}_n^{\delta }\left({\overline{z}}_n\right)$ is asymptotically normal with the moments $\left(0,D\left(u\right)\right)$. □

5. Proof of Lemma 1

(a) For any $\epsilon >0$, q > 0 and $u\in U$, we can write the following equation:

$$\begin{gathered} P_{n,u}\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \right\}= \\ =P_{n,u}\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \cap \left|{\tau }_{n,u}^{\ast }\right|\le q\right\}+P_{n,u}\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \cap \left|{\tau }_{n,u}^{\ast }\right|>q\right\}. \end{gathered}$$

(13)

Let denote $P_{n,u}\left(\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \right\}|\left\{\left|{\tau }_{n,u}^{\ast }\right|<q\right\}\right)$ the conditional probability of the event $\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \right\}$ under the condition of the event $\left\{\left|{\tau }_{n,u}^{\ast }\right|<q\right\}$. Then (13) can be rewritten as:

$$\begin{gathered} P_{n,u}\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \right\}=P_{n,u}\left(\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \right\}|\left\{\left|{\tau }_{n,u}^{\ast }\right|\le q\right\}\right)P_{n,u}\left\{\left|{\tau }_{n,u}^{\ast }\right|\le q\right\}+ \\ +P_{n,u}\left(\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \right\}|\left\{\left|{\tau }_{n,u}^{\ast }\right|>q\right\}\right)P_{n,u}\left\{\left|{\tau }_{n,u}^{\ast }\right|>q\right\}. \end{gathered}$$

(14)

According to (11), there is $C_{\epsilon }>0$ such that $\underset{u\in U,n\in {\mathbb{Z}}^{+}}{\sup }\left[P_{n,u}\left\{\left|{\tau }_{n,u}^{\ast }\right|>C_{\epsilon }\right\}\right]<\epsilon$ for any $\epsilon >0$. It follows then from (14) that for any $\epsilon >0$ and $u\in U$

$$P_{n,u}\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \right\}\text{<}{P}_{n,u}\left(\left\{\left|{\rho }_{n,u}\left({\tau }_n^{\ast }\right)\right|>\epsilon \right\}|\left\{\left|{\tau }_{n,u}^{\ast }\right|<C_{\epsilon }\right\}\right),$$

(15)

where ${\rho }_{n,u}\left({\tau }_{n,u}^{\ast }\right)={\delta }_n\left(u+{\tau }_{n,u}^{\ast }/\sqrt{n}\right)-{\delta }_n\left(u\right)-{\Phi }_n^{}\left(u+{\tau }_{n,u}^{\ast }/\sqrt{n}\right){\tau }_{n,u}^{\ast }$.

According to (12), for any $\epsilon >0$, $u\in U$ and $\left|h\right|<C_{\epsilon }$

$$\underset{u\in U}{\sup }{P}_{n,u}\left\{\left|{\beta }_{n,u}\left({\overline{z}}_n;h\right)\right|>\epsilon \right\}\to 0\left(n\to \infty \right),$$

(16)

where ${\beta }_{n,u}\left({\overline{z}}_n;h\right)={\delta }_n\left({\overline{z}}_n;u+n^{-1/2}h\right)-{\delta }_n\left({\overline{z}}_n;u\right)-{\Phi }_n\left(u\right)h$,

It follows from (15), (16) that for any $\epsilon >0$ $\underset{n\to \infty }{\lim }\underset{u\in U}{\sup }{P}_{n,u}\left\{\left|{\rho }_{n,u}\left(u_n^{\ast }\right)\right|>\epsilon \right\}=0$.

(b) Since $\left|{\xi }_{n,u}\left(u_n^{\ast }\right)\right|\le \Vert {\Phi }_{n,u}^{-1}\left(u_n^{\ast }\right)\Vert \left|{\rho }_{n,u}\left(u_n^{\ast }\right)\right|$, to prove statement (b) of Lemma 1, it suffices to check that $\Vert {\Phi }_n^{-1}\left(u_n^{\ast }\right)\Vert$ is bounded in probability. Since ${\Phi }_n^{-1}\left(u\right)$ satisfies conditions B2 of Theorem 1, for any $\epsilon >0$ there exists $C_{\epsilon }>0$ that for all n the following inequality holds: $P_{n,u}\left\{\Vert {\Phi }_{n,u}^{-1}\left(u_n^{\ast }\right)\Vert \ge C_{\epsilon }\right\}<\epsilon$. So, we can write:

$$\begin{gathered} P_{n,u}\left\{\left|{\xi }_{n,u}\left(u_n^{\ast }\right)\right|>\epsilon \right\}= \\ {P}_{n,u}\left(\left\{\left|{\rho }_{n,u}\left(u_n^{\ast }\right)\right|>\epsilon \right\}\cap \left(\Vert {\Phi }_{n,u}^{-1}\left(u_n^{\ast }\right)\Vert <C_{\epsilon }\right)\right)+ \\ +P_{n,u}\left(\left\{\left|{\rho }_{n,u}\left(u_n^{\ast }\right)\right|>\epsilon \right\}\cap \left\{\Vert {\Phi }_{n,u}^{-1}\left(u_n^{\ast }\right)\Vert \ge C_{\epsilon }\right\}\right)\le \\ \le P_{n,u}\left(\left\{\left|{\rho }_{n,u}\left(u_n^{\ast }\right)\right|>\epsilon \right\}\cap \left(\Vert {\Phi }_{n,u}^{-1}\left(u_n^{\ast }\right)\Vert <C_{\epsilon }\right)\right)+\epsilon . \end{gathered}$$

Since $\left|{\rho }_{u,n}\left(u_n^{\ast }\right)\right|$ satisfies statement (a) of Lemma 1, one can find a number $N_{\epsilon }$ such that $\underset{u\in U,n>N_{\epsilon }}{\sup }{P}_{u,n}\left\{\left|{\xi }_{u,n}\left(u_n^{\ast }\right)\right|>\epsilon \right\}$< 2$\epsilon$. □

6. Conclusions

The paper investigates the asymptotic properties of statistical estimates for the vector parameter $u\in R^q$ of a stationary multidimensional random time series $z_t\in R^m,t\in \mathbb{Z}$ satisfying the strong mixing conditions. We have considered estimates $\tilde{u}\left({\overline{z}}_n\right)$ that are solutions of the equations ${\nabla }_u{Q}_n\left({\overline{z}}_n;u\right)=0$, ${\overline{z}}_n={\left(z_1^{\mathrm{T}},\dots ,z_n^{\mathrm{T}}\right)}^{\mathrm{T}}$, where $Q_n\left({\overline{z}}_n;u\right)$ is some objective function for which ${\nabla }_u{Q}_n\left({\overline{z}}_n;u\right)$ satisfies the constraints of Theorem 1. We have proved that under these constraints, the estimates $\tilde{u}\left({\overline{z}}_n\right)$ are $\sqrt{n}$-consistent and asymptotically normal with a limit covariance matrix uniquely determined by the objective function $Q_n\left({\overline{z}}_n;u\right)$.

The results of this paper are a generalization of the methods for constructing and analyzing the asymptotic properties of M-estimates, which were previously studied for the case of independent identically distributed observations.