Estimator’s Properties of Specific Time-Dependent Multivariate Time Series

Abstract

There is now a vast body of literature on ARMA and VARMA models with time-dependent or time-varying coefficients. A large part of it is based on local stationary processes using time rescaling and assumptions of regularity with respect to time. A recent paper has presented an alternative asymptotic theory for the parameter estimators based on several distinct assumptions that seem difficult to verify at first look, especially for time-dependent VARMA or tdVARMA models. The purpose of the present paper is to detail several examples that illustrate the verification of the assumptions in that theory. These assumptions bear on the moments of the errors, the existence of the information matrix, but also how the coefficients of the pure moving average representation of the derivatives of the residuals (with respect to the parameters and evaluated at their true value) behave. We will do that analytically for two bivariate first-order models, an autoregressive model, and a moving average model, before sketching a generalization to higher-order models. We also show simulation results for these two models illustrating the analytical results. As a consequence, not only the assumptions can be checked but the simulations show how well the small sample behavior of the estimators agrees with the theory.

1. Introduction

Consider a multivariate time series $(x_t;t=1,\dots ,n)$ of length n and dimension r. It is supposed to be generated by an array process $(x_t^{\left(n\right)};t=1,\dots ,n;n>0)$. There is now a vast body of literature on ARMA and VARMA models with time-dependent or time-varying coefficients. For a recent review of VARMA models, including their time-dependent variants, see [1].

The history of time-dependent models for time series started with the works of [2,3,4,5,6]. Several of them, [2,4,5] and also [7,8,9,10,11,12], focussed on the temporal aspects, while others [3,6] were more interested in the spectral point of view.

We will not consider the numerous studies related to the time-dependent spectral approaches, except for the following one that had considerable attraction. We mean the theory based on local stationarity processes (LSP) due to Dahlhaus (see [13,14,15,16,17]). We will not repeat here the many contributions about it since they are nicely summarized in [18], and a few more recent references are mentioned in [19]. See also [20,21,22,23,24,25,26,27,28,29,30].

Other somewhat related approaches to ours include [31] (Chapter 17), which treats tdVAR models by Gaussian maximum likelihood (but does not discuss asymptotic properties), generalized autoregressive score (GAS) models of [32], testing parameter constancy against deterministically time-varying parameters, e.g., ref. [33] (Section 6.3) and references therein, generalized to VAR models in [34], smooth online parameter estimation approach [35], deep-learning approaches [36], using state-space methods [37], a non-parametric approach [38], or by using an explicit representation for ARMA recursions with either deterministically or stochastically varying coefficients [39].

If a few of the above-mentioned papers treat multivariate time-dependent models, like [25,31,36], they are often tdVAR, and, apparently, never tdVMA and tdVARMA models, with the notable exception of [38], which is semi-parametric and uses a kernel-density estimator. Ref. [40] explains that, although in theory a tdMA process can be written as an infinite tdAR process, it is not efficient to fit a high-order model. Since the first edition of [41] in 1970 and the practical studies that followed, it is well known that most time series in many fields like economics, sociology, tourism, agriculture, energy, and so on, are better fitted by ARIMA models, possibly on transformed data, rather than by AR models. Based on 13,238 monthly series taken from the Industrial Short-Term Indicator section of the EUROSTAT database on 15 Member States of the European Union and a few series from the United States and Japan, ref. [42] observed that the airline model, i.e., the seasonal ARIMA(0,1,1)(0,1,1${)}_{12}$ model on the log-transformed data, was best-fitting for 61% of the series, whereas only 2% were best fitted by an autoregression. There is no reason why the primacy of moving averages would not extend to multivariate time series and time-dependent models, although there is no empirical study at this stage. A few of the previously mentioned papers for univariate ARMA models with time-dependent coefficients include MA coefficients like [7,10,11,12]. In the multivariate case, ref. [17] treats time-varying or time-dependent VARMA (tdVARMA) models in the context of local stationarity but only for Gaussian processes and under the assumption that all the eigenvalues of the true spectral density matrix are uniformly bounded from below, a condition that is difficult to verify in practice. More recently, ref. [38] has provided results for a semi-parametric estimator for tdVARMA models.

In a recent paper [43], estimation results were produced for a wide class of vector ARMA array processes with time-dependent coefficients, denoted tdVARM${\mathrm{A}}^{\left(n\right)}(p,q)$, which includes as special cases both local stationary processes [18] and cyclically time-dependent processes [11]. The assumptions are rather general but complex at first sight, so it would be worthwhile to demonstrate their applicability. Previously, ref. [44] has already treated examples for cyclically tdVARMA stochastic processes. Ref. [43] could also be seen as a generalization to multivariate processes of [12] devoted to univariate ARMA models with time-dependent coefficients, thereby generalizing the autoregressive moving average (ARMA) models popularized by Box and Jenkins, ref. [41]. In [12], the two cases where the coefficients depend only on time t and both t and n were considered with an accent on the former case. The case where the coefficients depend on t but not series length n was generalized to VARMA models by [44]. Ref. [12] contained very simple univariate examples where the theoretical assumptions for the asymptotic properties were checked. We will see that developing simple examples is much more complex in a multivariate setting.

Previously, ref. [45] had shown that, when specialized to VARMA models with constant coefficients, these assumptions coincide with the assumptions for the standard asymptotic properties of the parametric estimation for these models. The problem is that the assumptions in [43] are complex. For instance, that paper contains remarks to address requests from reviewers who could not believe that these assumptions would work, despite the provided proofs. Consequently, we consider here the case of relatively simple array processes and will check analytically a representative sample of the assumptions for bivariate processes. By representative sample, we mean that the other assumptions can be verified using the same arguments. To simplify, we will restrain ourselves to first-order processes, tdVA${\mathrm{R}}^{\left(n\right)}$(1), when $p=1$ and $q=0$, and tdVM${\mathrm{A}}^{\left(n\right)}$(1), when $p=0$ and $q=1$. Even for these two special cases, we drastically limit the form of time-dependency and the number m of parameters. We believe, however, that verifying the assumptions of [43] in these two cases is exemplary for more complex models. These models were used, for specific values of the parameters, in a simulation study presented by [43], so that the present paper completes the information. Not only does estimation give results that are predicted by the theory for sufficiently large series, but we will also see that the values selected for the parameters in these simulation experiments fulfill the requirements.

We do not consider high-dimensional models in this paper for two reasons: first, there is little hope to be able to offer simple examples for $r>2$; and second, since the size of the matrices in the coefficients will be $r\times r$, the number of parameters m increases with $r^2$. With $n=100$, $p=q=2$, and $r=4$, the number of parameters is already $4\times 4^2=64$, plus $4\times (4+1)/2=10$ in the error covariance matrix, leaving only $100-74=26$ degrees of freedom. Moreover, the dimension of the information matrix (defined below) grows with $r^4$, so in that case, it is already $64\times 64$, possibly implying serious computational problems with matrix inversion. A solution would be to extend the sparse identification and estimation approach proposed for VARMA models by [46] and implemented by [47] in the R package bigtime 0.2.3 for R 3.6.0 and above. This is done by using sparsity-inducing convex regularizers. It works even for large-scale VARMA models, under sufficient regularity conditions expressed by the condition $r^{3}log\left(r\sqrt{n}\right)\to 0$.

This article is organized as follows. In Section 2, we introduce the general marginally heteroscedastic tdVARM${\mathrm{A}}^{\left(n\right)}$ array model with the main notations, describe the crucial assumptions under which [43] have proved the asymptotic properties of the Gaussian maximum likelihood estimator, and present approximations of the true information matrix. Section 3 contains our results: in Section 3.1, we consider a tdVA${\mathrm{R}}^{\left(n\right)}$(1) process and a tdVM${\mathrm{A}}^{\left(n\right)}$(1) process in Section 3.2. In both cases, after reducing the number of parameters progressively, we prove analytically that the assumptions can be verified, provide the constraints on the true values, and simulation results with non-Gaussian errors to assess the quality of the estimates and their standard errors. Section 3.3 and Section 3.4 are short attempts for generalization but, of course, the model complexity does not permit a complete analytical treatment. The paper ends with a discussion of the results in Section 4. There are three appendices for details about Section 3.1 and Section 3.2.

2. Materials and Methods

2.1. The General tdVARMA Array Model

Let $\theta ={({\theta }_1,\dots ,{\theta }_m)}^T$, $m\ge 1$, where ^T denotes transposition, be the parameter belonging to $\mathsf{\Theta }$, an open set of ${\mathbf{R}}^m$, and ${\theta }^0\in \mathsf{\Theta }$, the true value of $\theta$. Let $P_{\theta }$ be the probability measure on $\mathsf{\Theta }$. Let the $r\times r$ matrices $A_{t1}^{\left(n\right)},\dots ,A_{tp}^{\left(n\right)}$, and $B_{t1}^{\left(n\right)},\dots ,B_{tq}^{\left(n\right)}$, as well as $g_t^{\left(n\right)}$, be deterministic functions of time t and, possibly, of n. Let $\{{ϵ}_t;t\in \mathbf{N}\}$ be a sequence of zero-mean independent random variables with a covariance matrix $\mathsf{\Sigma }$ which is invertible, and with finite moments of order $4+2\delta$ with $\delta >0$. Denoting ⊗ the Kronecker product, and $vec(.)$ which transforms a matrix into a column vector, we let ${\kappa }_t=E\left(vec\left({ϵ}_t{ϵ}_t^T\right)vec{\left({ϵ}_t{ϵ}_t^T\right)}^T\right)=E\left(\left({ϵ}_t{ϵ}_t^T\right)\otimes \left({ϵ}_t{ϵ}_t^T\right)\right)$, which can depend on t, but it will not in all our examples. For $t<1$, we suppose that ${ϵ}_t$ and $x_t^{\left(n\right)}$ are both equal to zero. According to [43], the tdVARM${\mathrm{A}}^{\left(n\right)}(p,q)$ array process can be defined by the equation

$$\begin{array}{c}\hfill x_t^{\left(n\right)}=\sum _{i=1}^p{A}_{ti}^{\left(n\right)}{x}_{t-i}^{\left(n\right)}+g_t^{\left(n\right)}{ϵ}_t+\sum _{j=1}^q{B}_{tj}^{\left(n\right)}{g}_{t-j}^{\left(n\right)}{ϵ}_{t-j}.\end{array}$$

(1)

Note that $g_t^{\left(n\right)}$ implies marginal heteroscedasticity, meaning that the errors do not have a constant covariance matrix but well a time-dependent covariance matrix. Unfortunately, the theory in [43] does not accommodate conditional heteroscedasticity, as seen in VARMA-GARCH models, such as VARMA-BEKK models (see [48,49,50]).

The model consists of replacing the coefficients $A_{ti}^{\left(n\right)}$, $i=1,\dots ,p$, $B_{tj}^{\left(n\right)}$, $j=1,\dots ,q$, and $g_t^{\left(n\right)}$ by adding an argument $\left(\theta \right)$ so that the coefficients in (1) correspond to the value at $\theta ={\theta }^0$. We suppose that the parameters involved in the $A_{ti}^{\left(n\right)}\left(\theta \right)$ and the $B_{tj}^{\left(n\right)}\left(\theta \right)$, on the one hand, and those in $g_t^{\left(n\right)}\left(\theta \right)$, on the other hand, are distinct and ordered that way. Let $m_{AB}\le m$ be the number of parameters of the first type so that the derivative of $g_t^{\left(n\right)}\left(\theta \right)$ with respect to ${\theta }_i$ is identically zero for $i=1,\dots ,m_{AB}$. Of course, we also need to replace $g_{t-j}^{\left(n\right)}{ϵ}_{t-j}$ with the residuals $e_{t-j}\left(\theta \right)$, $j=0,\dots ,q$, noting that $e_{t-j}\left({\theta }^0\right)=g_{t-j}^{\left(n\right)}{ϵ}_{t-j}$. Let ${\alpha }_t^{\left(n\right)}\left(\theta \right)=log\det \left({\mathsf{\Sigma }}_t^{\left(n\right)}\left(\theta \right)\right))+e_t^{\left(n\right)T}\left(\theta \right){\mathsf{\Sigma }}_t^{\left(n\right)-1}\left(\theta \right)e_t^{\left(n\right)}\left(\theta \right)$, where ${\mathsf{\Sigma }}_t^{\left(n\right)}\left(\theta \right)=E_{\theta }\left(e_t^{\left(n\right)}\left(\theta \right)e_t^{\left(n\right)T}\left(\theta \right)\right)=g_t^{\left(n\right)}\left(\theta \right)\mathsf{\Sigma }{g}_t^{\left(n\right)T}\left(\theta \right)$. Using a series of length n, we estimate $\theta$ by ${\widehat{\theta }}^{\left(n\right)}={\arg \min }_{\theta \in \mathsf{\Theta }}{\displaystyle \frac{1}{2}}{\sum }_{t=1}^n{\alpha }_t^{\left(n\right)}\left(\theta \right)$, which means using the Gaussian quasi-likelihood method. Under the assumptions given in detail in [43] and summarized in Section 2.2, ${\widehat{\theta }}^{\left(n\right)}\to {\theta }^0$ in probability, and $\sqrt{n}({\widehat{\theta }}^{\left(n\right)}-{\theta }^0)\to N(0,V^{-1}W{V}^{-1})$ in law, both when $n\to \infty$, where V and W are defined in (4) below. Note that using this so-called sandwich or robust asymptotic covariance matrix $V^{-1}W{V}^{-1}$ should improve the estimated standard errors and reduce their negative bias when the errors are not normally distributed, especially for the parameters related to heteroscedasticity.

The assumptions are based on the behavior of the coefficients of a pure moving average decomposition of the process and the derivatives (up to order 3 and evaluated at $\theta ={\theta }^0$) of the residuals, taking into account the assumptions for $t<1$. For the process, the decomposition is $x_t^{\left(n\right)}={\sum }_{k=0}^{t-1}{\psi }_{tk}^{\left(n\right)}{g}_{t-k}^{\left(n\right)}{ϵ}_{t-k}$ and the coefficients are denoted ${\psi }_{tk}^{\left(n\right)}$. We will also use the coefficients of the pure autoregressive decomposition of the process $x_t^{\left(n\right)}={\sum }_{k=1}^{t-1}{\pi }_{tk}^{\left(n\right)}\left(\theta \right)x_{t-k}^{\left(n\right)}+e_t^{\left(n\right)}\left(\theta \right)$ and let ${\pi }_{tk}^{\left(n\right)}={\pi }_{tk}^{\left(n\right)}\left({\theta }^0\right)$. The first-order derivative of $e_t^{\left(n\right)}\left(\theta \right)$ with respect to ${\theta }_i$, $i=1,\dots ,m$, is expressed either as

$$-\sum _{k=1}^{t-1}{\displaystyle \frac{\partial {\pi }_{tk}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}{x}_{t-k}^{\left(n\right)}\mathrm{or}\sum _{k=1}^{t-1}{\psi }_{tik}^{\left(n\right)}\left(\theta \right)g_{t-k}^{\left(n\right)}{ϵ}_{t-k},$$

(2)

and we let ${\psi }_{tik}^{\left(n\right)}={\psi }_{tik}^{\left(n\right)}\left({\theta }^0\right)$. These coefficients can be obtained by recurrence. Indeed, generalizing [44] (Equation (3.8)), it can be shown that

$$\begin{array}{ccc}\hfill {\psi }_{tik}^{\left(n\right)}& =& -\sum _{u=1}^k{\left.{\displaystyle \frac{\partial {\pi }_{tu}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}\right|}_{\theta ={\theta }^0}{\psi }_{t-u,k-u}^{\left(n\right)}.\hfill \end{array}$$

(3)

It is not obvious how to deal with these coefficients that are, however, essential in [43], as will be shown in Section 3.

We need finally to introduce the Hessian matrix V and the outer product of gradient W. They are obtained as limits of averages of time-dependent matrices

$$V=\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}\sum _{t=1}^n{V}_t^{\left(n\right)},W=\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}\sum _{t=1}^n{W}_t^{\left(n\right)},$$

(4)

where the elements $V_{ij}^{\left(n\right)}$ and $W_{ij}^{\left(n\right)}$ for $i,j=1,\dots ,m$ are obtained, respectively, by

$$E_{{\theta }^0}\left({\displaystyle \frac{\partial e_t^{\left(n\right)T}\left(\theta \right)}{\partial {\theta }_i}}{\mathsf{\Sigma }}_t^{\left(n\right)-1}\left(\theta \right){\displaystyle \frac{\partial e_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_j}}\right)+{\displaystyle \frac{1}{2}}tr{\left[{\mathsf{\Sigma }}_t^{\left(n\right)-1}\left(\theta \right){\displaystyle \frac{\partial {\mathsf{\Sigma }}_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}{\mathsf{\Sigma }}_t^{\left(n\right)-1}\left(\theta \right){\displaystyle \frac{\partial {\mathsf{\Sigma }}_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_j}}\right]}_{\theta ={\theta }^0},$$

(5)

where $E_{{\theta }^0}$ is the expectation under $P_{{\theta }^0}$, and

$${\displaystyle \frac{1}{4}}{E}_{{\theta }^0}\left({\displaystyle \frac{\partial {\alpha }_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}{\displaystyle \frac{\partial {\alpha }_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_j}}\right).$$

(6)

The theoretical aspects of the computation of V and W are discussed in [51], with the impact of the distribution on ${\kappa }_t$ for the latter. In particular, as a consequence of our assumptions and notations, for $i,j=1,\dots ,m_{AB}$, there is a simple common expression for the terms in (5) of both V and W:

$$V_{t,ij}^{\left(n\right)}=W_{t,ij}^{\left(n\right)}=tr\left\{{\psi }_{tik}^{\left(n\right)T}{\mathsf{\Sigma }}_t^{\left(n\right)-1}{\psi }_{tjk}^{\left(n\right)}{\mathsf{\Sigma }}_{t-k}^{\left(n\right)}\right\}.$$

(7)

For $i,j>m_{AB}$, the expressions differ: for V, it is based on the second term in (5), while for W, it is based on $tr\left({\mathsf{\Sigma }}_t^{\left(n\right)}(\partial {\mathsf{\Sigma }}_t^{\left(n\right)-1}/\partial {\theta }_i)\right)$, $vec\left(g_t^{\left(n\right)T}(\partial {\mathsf{\Sigma }}_t^{\left(n\right)-1}/\partial {\theta }_i)g_t^{\left(n\right)}\right)$ and makes use of ${\kappa }_t$ (see [51] for details). It is also not obvious at all how to use (4)–(7) for a given tdVARM${\mathrm{A}}^{\left(n\right)}$ model, although these matrices V and W are crucial for obtaining the standard errors of the estimators. Also, obtaining a finite-sample approximation will be useful (see Section 2.3).

2.2. Typical Assumptions

It would be very lengthy to check all the assumptions of [43]. We will examine a representative sample of these assumptions, knowing that the others can be checked in the same way. More precisely,

(i) 

the matrices $A_{ti}^{\left(n\right)}\left(\theta \right)$, $i=1,\dots ,p$, $B_{tj}^{\left(n\right)}\left(\theta \right)$, $j=1,\dots ,q$, and $g_t^{\left(n\right)}\left(\theta \right)$ are three times continuously differentiable with respect to $\theta$;

(ii) 

existence of bounds on the Frobenius norm, denoted by ${\parallel .\parallel }_F$, of $g_t^{\left(n\right)}$, ${\kappa }_t$, ${\mathsf{\Sigma }}_t^{\left(n\right)}\left(\theta \right)$, ${\mathsf{\Sigma }}_t^{\left(n\right)-1}\left(\theta \right)$, and their derivatives with respect to $\theta$ at $\theta ={\theta }_0$;

(iii) 

upper bounds like ${\sum }_{k=\nu }^{t-1}{\parallel {\psi }_{tik}^{\left(n\right)}\parallel }_F^2<N_1{\mathsf{\Phi }}^{\nu -1}$ and ${\sum }_{k=\nu }^{t-1}{\parallel {\psi }_{tik}^{\left(n\right)}\parallel }_F^4<N_2{\mathsf{\Phi }}^{\nu -1}$, $i=1,\dots ,m$, $\nu =1,\dots ,t-1$, where $N_1$ and $N_2$ are positive constants and $0<\mathsf{\Phi }<1$, and similar conditions on derivatives up to the third order;

(iv) 

existence of a strictly positive definite matrix V defined by (4) and (5);

(v) 

existence of a positive definite matrix W defined by (4) and (6);

(vi) 

and, for $i=1,\dots ,m$, an assumption on triple sums of the kind

$$\begin{array}{c}\hfill {\displaystyle \frac{1}{n^2}}\sum _{d=1}^{n-1}\sum _{t=1}^{n-d}\sum _{k=1}^{t-1}{∥g_{t-k}^{\left(n\right)}∥}_F^2{∥{\psi }_{tik}^{\left(n\right)}∥}_F{∥{\psi }_{t+d,i,k+d}^{\left(n\right)}∥}_F=O\left({\displaystyle \frac{1}{n}}\right).\end{array}$$

Note that, contrary to the theory of local stationarity processes, we do not assume regularity conditions on the dependence with respect to time.

2.3. Obtaining Approximations of True V and W

To validate the simulations and the theory, we have implemented an estimation of V and W as a by-product of estimation (see the computational details in [52]). This was done, in particular, for the bivariate first-order models tdVA${\mathrm{R}}^{\left(n\right)}$(1) and tdVM${\mathrm{A}}^{\left(n\right)}$(1) of [43] (Section 4).

Moreover, for given values of the parameters $\theta$, a program written in Matlab (but outside of the estimation program first described in [53]) computes for $t=1,\dots ,n$, successively $g_t^{\left(n\right)}\left(\theta \right)$, ${\mathsf{\Sigma }}_t^{\left(n\right)}\left(\theta \right)=g_t^{\left(n\right)}\left(\theta \right)\mathsf{\Sigma }{g}_t^{\left(n\right)T}\left(\theta \right)$, $A_{t1}^{\left(n\right)}\left(\theta \right)$ or $B_{t1}^{\left(n\right)}\left(\theta \right)$, the ${\psi }_{tik}^{\left(n\right)}\left(\theta \right)$’s for $k=1,\dots ,t$ in (3). For the computation of the standard errors of the estimates, we need the information matrix, thus the Hessian V using (5) and the outer product of gradient W using (6). Approximations of V and W are then obtained using the averages in (4), respectively, without taking the limits. Divided by n, they are used to obtain the “theoretical standard errors” in the simulation results. Note that the number of operations is $O\left(n^2\right)$ but this is done only once for the true value of the parameters, specifically $\theta ={\theta }^0$.

3. Results

To begin with, we consider here two first-order models, i.e., the cases of the tdVA${\mathrm{R}}^{\left(n\right)}$(1) model where $p=1$ and $q=0$, on the one hand, and the tdVM${\mathrm{A}}^{\left(n\right)}$(1) model where $p=0$ and $q=1$, on the other hand. To make it practical, and allow for a fully analytical treatment, we will quickly move to bivariate processes, i.e., the case where $r=2$ in (1). Then, we will consider a tdVARM${\mathrm{A}}^{\left(n\right)}$$(1,1)$ model for which we will provide an expression for the coefficients ${\psi }_{tik}$ in the moving average representation of the first-order derivatives of the residuals. Finally, for the general case of a tdVARM${\mathrm{A}}^{\left(n\right)}$$(p,q)$ model, we will only provide indications on sufficient conditions to demonstrate how it is possible to proceed.

3.1. Treatment of a tdVA$R^{\left(n\right)}$(1) Model

In this section, we consider a tdVA${\mathrm{R}}^{\left(n\right)}$(1) model, first in the general case before taking $r=2$. The model is defined by

$$\begin{array}{c}\hfill x_t^{\left(n\right)}=A_t^{\left(n\right)}\left(\theta \right)x_{t-1}^{\left(n\right)}+e_t^{\left(n\right)}\left(\theta \right),\end{array}$$

(8)

with $e_t^{\left(n\right)}\left({\theta }^0\right)=g_t^{\left(n\right)}{ϵ}_t$, $g_t^{\left(n\right)}=g_t^{\left(n\right)}\left({\theta }^0\right)$, and ${ϵ}_t$ having all moments of order $4+2\delta$, $\delta >0$, and such that $\parallel {\kappa }_t{\parallel }_F$ is bounded. Let also $A_t^{\left(n\right)}=A_t^{\left(n\right)}\left({\theta }^0\right)$. Let us define

$$A_t^{\left(n\right)[k-1]}=\prod _{l=1}^{k-1}{A}_{t-l}^{\left(n\right)},k>1, \mathrm{and} A_t^{\left(n\right)\left[0\right]}=I_r.$$

(9)

It can be checked that ${\psi }_{tk}^{\left(n\right)}=A_{t+1}^{\left(n\right)\left[k\right]}$, $k=0,1,\dots$, and ${\psi }_{tik}^{\left(n\right)}=\partial A_t^{\left(n\right)}\left(\theta \right)/\partial {\theta }_i{|}_{\theta ={\theta }^0}{A}_t^{\left(n\right)[k-1]}$, $i=1,\dots ,m$. Note that the coefficients of the pure autoregressive decomposition of the process are ${\pi }_{t1}^{\left(n\right)}\left(\theta \right)=A_t^{\left(n\right)}\left(\theta \right)$, and ${\pi }_{tk}^{\left(n\right)}\left(\theta \right)=0$, $k>1$.

To be more specific, assume a bivariate process such that the elements of the matrices $A_t^{\left(n\right)}\left(\theta \right)$ are linear functions of time, and the diagonal elements of $g_t^{\left(n\right)}\left(\theta \right)$ are exponential functions of time. More precisely, we suppose that

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{A}_t^{\left(n\right)}\left(\theta \right)=\left(\begin{array}{cc}{A}_{11}^{\prime }& A_{12}^{\prime }\\ A_{21}^{\prime }& A_{22}^{\prime }\end{array}\right)+{\displaystyle \frac{1}{n-1}}(t-{\displaystyle \frac{n+1}{2}})\left(\begin{array}{cc}{A}_{11}^{″}& A_{12}^{″}\\ A_{21}^{″}& A_{22}^{″}\end{array}\right),\hfill \\ g_t^{\left(n\right)}\left(\theta \right)=\left(\begin{array}{cc}exp\left\{{\displaystyle \frac{{\eta }_{11}}{n-1}}(t-{\displaystyle \frac{n+1}{2}})\right\}& 0\\ 0& exp\left\{{\displaystyle \frac{{\eta }_{22}}{n-1}}(t-{\displaystyle \frac{n+1}{2}})\right\}\end{array}\right)\mathrm{and}\mathsf{\Sigma }=\left(\begin{array}{cc}{\sigma }_{11}& 0\\ 0& {\sigma }_{22}\end{array}\right).\hfill \end{array}\right.\end{array}$$

(10)

Remark 1. 

We have taken linear functions of time for illustrative purposes but it should be clear that the theory works in whole generality. The case of a linear function of time with the divisor $n-1$ appearing in (10) is compatible with Dahlhaus LSP theory. This is why we consider array processes instead of stochastic processes. We will come back to this in the discussion.

Now we start examining the typical assumptions of the theory stated in Section 2.2.

3.1.1. Assumptions (i) and (ii)

Assumption (i) is clearly satisfied. Denote $L(t,n)={\displaystyle \frac{1}{n-1}}(t-{\displaystyle \frac{n+1}{2}})$ for $t=1,\dots ,n$ and $-{\displaystyle \frac{1}{2}}$ for $t\le 0$. It is obvious that $\left|L(t,n)\right|\le {\displaystyle \frac{1}{2}}$ for all $t\le n$. This is because $L(1,n)=L(n,n)={\displaystyle \frac{1}{2}}$ so that we have preferred the denominator $n-1$ instead of n used in the theory of locally stationary processes. Using the definition,

$${\mathsf{\Sigma }}_t^{\left(n\right)}\left(\theta \right)=\left(\begin{array}{cc}{\sigma }_{11}{e}^{2{\eta }_{11}L(t,n)}& 0\\ 0& {\sigma }_{22}{e}^{2{\eta }_{22}L(t,n)}\end{array}\right),{\mathsf{\Sigma }}_t^{\left(n\right)-1}\left(\theta \right)=\left(\begin{array}{cc}{\displaystyle \frac{1}{{\sigma }_{11}}}{e}^{-2{\eta }_{11}L(t,n)}& 0\\ 0& {\displaystyle \frac{1}{{\sigma }_{22}}}{e}^{-2{\eta }_{22}L(t,n)}\end{array}\right)$$

(11)

are matrices whose Frobenius norms are bounded uniformly in t and n, from below by a strictly positive number and also from above, hence Assumption (ii) is satisfied.

3.1.2. Assumption (iii)

Only to simplify the analytical expressions, assume that ${\eta }_{11}=0$ in $P_{\theta }$, that the element $(2,1)$ of $A_t^{\left(n\right)}\left(\theta \right)$ is identically zero, and the element $(1,2)$ of $A_t^{\left(n\right)}\left(\theta \right)$ is a constant $A_{12}^{\prime }\ne 0$. This is to have an upper-triangular form (facilitating the analytical treatment) that does not degenerate into a diagonal matrix (as the latter would imply uncorrelated components since $\mathsf{\Sigma }$ is diagonal). To simplify further the details (although it is not necessary for the principles), in addition to ${\eta }_{11}=0$, instead of the full vector of parameters $\theta ={(A_{11}^{\prime },A_{12}^{\prime },A_{22}^{\prime },A_{11}^{″},A_{22}^{″},{\eta }_{22})}^T,$ we put $A_{22}^{\prime }=A_{11}^{″}=0$ and $A_{12}^{\prime }$ is fixed to $A_{12}^{\prime 0}$, so that the vector of parameters to estimate reduces to $\theta ={(A_{11}^{\prime },A_{22}^{″},{\eta }_{22})}^T$, hence $m=3$ and there is one parameter of each kind. The presence of $A_{12}^{\prime 0}\ne 0$ makes sure that the two components of $x_t^{\left(n\right)}$ are not independent. There are assumptions about the true value ${\theta }^0={(A_{11}^{\prime 0},A_{22}^{″0},{\eta }_{22}^0)}^T$ that will be stated later. Then, for $k\ge 2$, we can check by induction that $A_{t,21}^{\left(n\right)[k-1]}=0$ and

$$\begin{array}{cc}\hfill A_{t,11}^{\left(n\right)[k-1]}& ={\left(A_{11}^{\prime 0}\right)}^{k-1},A_{t,22}^{\left(n\right)[k-1]}={\left(A_{22}^{″0}\right)}^{k-1}\prod _{l=1}^{k-1}L(t-l,n),\\ \hfill A_{t,12}^{\left(n\right)[k-1]}& =A_{12}^{\prime 0}\sum _{l=1}^{k-1}{\left(A_{11}^{\prime 0}\right)}^{k-l-1}{\left(A_{22}^{″0}\right)}^{l-1}\prod _{f=k-l}^{k-2}L{(t-f,n)}^{{\delta }_{lf}},\end{array}$$

(12)

where ${\delta }_{lf}=0$, for $l+f\le k-1$, and ${\delta }_{lf}=1$, for $l+f>k-1$, generalizing relations in [44] (Appendix S1). To simplify the notations, we will henceforth omit ⁽ⁿ⁾ in the entries of $A_t^{\left(n\right)[k-1]}$.

From the definition of ${\psi }_{tik}^{\left(n\right)}$ in (3), we deduce ${\psi }_{t3k}^{\left(n\right)}=0$ and

$$\begin{array}{ccc}\hfill {\psi }_{t1k}^{\left(n\right)}& =& \left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right)\left(\begin{array}{cc}{A}_{t,11}^{[k-1]}& A_{t,12}^{[k-1]}\\ 0& A_{t,22}^{[k-1]}\end{array}\right)=\left(\begin{array}{cc}{A}_{t,11}^{[k-1]}& A_{t,12}^{[k-1]}\\ 0& 0\end{array}\right),\hfill \\ \hfill {\psi }_{t2k}^{\left(n\right)}& =& \left(\begin{array}{cc}0& 0\\ 0& L(t,n)\end{array}\right)\left(\begin{array}{cc}{A}_{t,11}^{[k-1]}& A_{t,12}^{[k-1]}\\ 0& A_{t,22}^{[k-1]}\end{array}\right)=\left(\begin{array}{cc}0& 0\\ 0& L(t,n)A_{t,22}^{[k-1]}\end{array}\right).\hfill \end{array}$$

(13)

We define the constant ${\mathsf{\Phi }}^{1/2}=max\{|A_{11}^{\prime 0}|,{\displaystyle \frac{1}{2}}|A_{22}^{″0}|\}$ such that $0<\mathsf{\Phi }<1$. Therefore, we assume $|A_{11}^{\prime 0}| <1$ and $|A_{22}^{″0}| <2$. From (13), since $L(t,n)A_{t,22}^{[k-1]}<{\mathsf{\Phi }}^{(k-1)/2}$, we deduce ${\sum }_{k=\nu }^{t-1}{∥{\psi }_{t2k}^{\left(n\right)}∥}_F^2={\sum }_{k=\nu }^{t-1}{\left[L(t,n)A_{t,22}^{[k-1]}\right]}^2\le {\sum }_{k=\nu }^{t-1}{\mathsf{\Phi }}^{k-1}<N_1{\mathsf{\Phi }}^{\nu -1}$, where $N_1=1/(1-\mathsf{\Phi })$. It is more delicate for ${\psi }_{t1k}^{\left(n\right)}$ for which

$$\begin{array}{c}\hfill \sum _{k=\nu }^{t-1}{∥{\psi }_{t1k}^{\left(n\right)}∥}_F^2=\sum _{k=\nu }^{t-1}\left[{\left(A_{t,11}^{[k-1]}\right)}^2+{\left(A_{t,12}^{[k-1]}\right)}^2\right].\end{array}$$

(14)

The sum of the first term is also bounded by $N_1{\mathsf{\Phi }}^{\nu -1}$ because $A_{t,11}^{[k-1]}<{\mathsf{\Phi }}^{(k-1)/2}$. Thanks to (12), an upper bound of $|A_{t,12}^{[k-1]}|$ equals

$$|A_{12}^{\prime 0}|\sum _{\ell =1}^{k-1}|A_{11}^{\prime 0}{|}^{k-l-1}|A_{22}^{″0}{|}^{l-1}\prod _{f=k-l}^{k-2}{|L(t-f,n)|}^{{\delta }_{lf}}\le |A_{12}^{\prime 0}|(k-1){\mathsf{\Phi }}^{k/2-1},$$

(15)

hence ${\sum }_{k=\nu }^{t-1}{∥{\psi }_{t1k}^{\left(n\right)}∥}_F^2<N_1^{\prime }{\mathsf{\Phi }}^{\nu -1}$, where $N_1^{\prime }={\left(A_{12}^{\prime 0}\right)}^2\mathsf{\Phi }/{(1-\mathsf{\Phi })}^3$ (see the details in Appendix A.1). We could have used [54] to simplify the investigation.

3.1.3. Assumption (iv)

First, to evaluate the elements $(i,j)$, $i=1,2$, of V in (4) with (5), we have to take the limit for $n\to \infty$ of the sum for $t=1,\dots ,n$ of $V_{t,ij}^{\left(n\right)}/n$, which can be computed using (7). But in the tdVA${\mathrm{R}}^{\left(n\right)}$(1) case, using (2) with ${\pi }_{t1}^{\left(n\right)}\left(\theta \right)=A_t^{\left(n\right)}\left(\theta \right)$, it is more simply expressed as

$$V_{t,ij}^{\left(n\right)}=tr\left\{{\left.{\displaystyle \frac{\partial A_t^{T\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}\right|}_{\theta ={\theta }^0}{\mathsf{\Sigma }}_t^{\left(n\right)-1}{\left.{\displaystyle \frac{\partial A_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_j}}\right|}_{\theta ={\theta }^0}E\left(x_{t-1}^{\left(n\right)}{x}_{t-1}^{\left(n\right)T}\right)\right\},i,j=1,\dots ,m_{AB}.$$

The most difficult case is for $i=j=2$: the product of derivatives and ${\mathsf{\Sigma }}_t^{\left(n\right)-1}$ has the element $(2,2)$ equal to $L^2(t,n)exp(-2{\eta }_{22}^{0}L(t,n))/{\sigma }_{22}$. Assume ${\eta }_{22}^0>0$. Then it can be seen that $(1/n){\sum }_{t=1}^n{L}^2(t,n)exp(-2{\eta }_{22}^{0}L(t,n))$ converges to a limit when $n\to \infty$. We will now prove that $E\left(x_{t-1}^{\left(n\right)}{x}_{t-1}^{\left(n\right)T}\right)$ tends to a limit when $n\to \infty$. Denoting $E_{t-k}=exp\left\{2{\eta }_{22}^{0}L(t-k,n)\right\}$, it is a sum for $k=1$ to $t-1$ of matrices

$$\left(\begin{array}{cc}{\sigma }_{11}{\left(A_{11}^{\prime 0}\right)}^{2(k-1)}+{\sigma }_{22}{E}_{t-k}{A}_{t,12}^{[k-1]2}& {\sigma }_{22}{E}_{t-k}{A}_{t,12}^{[k-1]}{\left(A_{22}^{″0}\right)}^{k-1}{K}_{tn}^{[k-1]}\\ {\sigma }_{22}{E}_{t-k}{A}_{t,12}^{[k-1]}{\left(A_{22}^{″0}\right)}^{k-1}{K}_{tn}^{[k-1]}& {\sigma }_{22}{E}_{t-k}{\left(A_{22}^{″0}\right)}^{2(k-1)}{K}_{tn}^{[k-1]}\end{array}\right),$$

(16)

where $K_{tn}^{[k-1]}={\prod }_{l=1}^{k-1}L(t-l,n)$. Using the bounds already presented when discussing ${\psi }_{tik}$, $i=1,2$, and given that $E_{t-k}=exp\left\{2{\eta }_{22}^{0}L(t-k,n)\right\}=exp\left\{2{\eta }_{22}^{0}L(t,n)\right\}$${(exp\{-2{\eta }_{22}^0/(n-1)\})}^k$, it is easy to see that the elements of (16) behave like terms of a geometric series (or terms of two sums of geometric series) for large k, so that, for large t, their sum is finite, strictly positive, and convergent when $n\to \infty$. The existence of the limit $V_{22}$ then follows from Theorem 2.5 of [55]. The two other cases ($i=j=1$, and $i=1,j=2$) are straightforward.

The computation of $V_{33}$ involves only the second term of the definition in (5). The derivative of ${\mathsf{\Sigma }}_t^{\left(n\right)}$ with respect to ${\eta }_{22}$ yields a zero matrix, except the element $(2,2)$, which is $2{\sigma }_{22}L(t,n)$$exp\left\{2{\eta }_{22}^{0}L(t,n)\right\}$. The product of ${\mathsf{\Sigma }}_t^{\left(n\right)-1}$ by that derivative gives a matrix of 0 except for the element $(2,2)$ which is equal to $2L(t,n)$. We have, therefore, to take one-half of the limit for $n\to \infty$ of the sum for $t=1,\dots ,n$ of $4L^2(t,n)/n$. However, the sum of ${\left(t-{\displaystyle \frac{n+1}{2}}\right)}^2/n$ is the variance in a discrete uniform distribution over $\{1,2,\dots ,n\}$ which equals $(n^2-1)/12$. Hence, we obtain $(n+1)/\left(6\right(n-1\left)\right)$ whose limit for $n\to \infty$ equals $1/6$. We can check that the elements not discussed vanish. Note that the factor ${\displaystyle \frac{1}{6}}$ appeared already in a similar univariate example shown by [12] (Example 3).

3.1.4. Assumption (v)

For a Gaussian process, $W=V$ in (6). For a Laplace or a Student distribution (the latter with at least 5 degrees of freedom), in particular, the entries of ${\kappa }_t$ are not smaller than those for a normal distribution [51] (Section 4), hence $W_{33}\ge V_{33}$.

3.1.5. Assumption (vi)

In order to verify Assumption (vi), for example for $i=2$, the simplest case, we have to show that $(1/n^2)$ multiplied by ${\sum }_{d=1}^{n-1}{\sum }_{t=1}^{n-d}{\sum }_{k=1}^{t-1}\parallel g_{t-k}^{\left(n\right)}{\parallel }_F^2{\parallel {\psi }_{t2k}^{\left(n\right)}\parallel }_F$$\parallel {\psi }_{t+d,2,k+d}^{\left(n\right)}{\parallel }_F$ is $O(1/n)$. First $\parallel g_{t-k}^{\left(n\right)}{\parallel }_F^2$ equals $1+exp\left\{2{\eta }_{22}^{0}L(t-k,n)\right\}$, which is bounded by $1+exp\left\{{\eta }_{22}^0\right\}$. Then, we take an upper bound of $\parallel {\psi }_{t2k}^{\left(n\right)}{\parallel }_F$ by ${\mathsf{\Phi }}^{k-1}$ and thus of $\parallel {\psi }_{t+d,2,k+d}^{\left(n\right)}{\parallel }_F$ by ${\mathsf{\Phi }}^{k+d-1}$. The sum for $k=1,\dots ,t-1$ of the product ${\mathsf{\Phi }}^{k-1}{\mathsf{\Phi }}^{k+d-1}={\mathsf{\Phi }}^{d+2k-2}$ is bounded by ${\mathsf{\Phi }}^{d-2}$ times a constant $1/(1-{\mathsf{\Phi }}^2)$. By exchanging the two outside summations, we have to find an upper bound of ${\sum }_{t=1}^{n-1}{\sum }_{d=1}^{n-t}{\mathsf{\Phi }}^{d-1}$ by ${\mathsf{\Phi }}^{-1}$ times the sum for $t=1,\dots ,n-1$ of a constant $1/(1-\mathsf{\Phi })$. Dividing by $n^2$, we have $O(1/n)$. The case where $i=j=1$ is more delicate and will not be detailed but the principle is identical.

Example 1. 

In [43] (Section 4.1.2), simulations results were shown for artificial Gaussian time series generated by a specific case of model Equations (8)–(10), under all the restrictions mentioned above and with the following values for ${\theta }^0={(A_{11}^{\prime 0},A_{22}^{″0},{\eta }_{22})}^T={(0.8,0.75,0.7)}^T$. Moreover, $A_{12}^{\prime }=0.5$ is a constant as above. First, note that these values satisfy the requirements. Indeed ${\mathsf{\Phi }}^{1/2}=max(|A_{11}^{\prime 0}|,{\displaystyle \frac{1}{2}}|A_{22}^{″0}|)<1$ and ${\eta }_{22}>0$. Let us consider 1000 time series of length $n=25,50,100,200,\mathrm{and}400$ obtained using multivariate Laplace, on the one hand, and Student with 5 degrees of freedom, on the other hand. The empirical estimation results are shown, respectively, in

Table 1. Estimation results for the tdVA${\mathrm{R}}^{\left(n\right)}$(1) model (8) under (10) with Laplace errors. (a) emp.est. is the empirical estimate (average over the simulation estimates), (b) emp.s.e. is the empirical standard error (standard deviation over the simulation estimates), (c) est.s.e. is the estimated standard error (average of the standard errors as a by-product of estimation), (d) theor.s.e. is the (approximate) theoretical standard error (finite approximation of standard error using (4)), and (e) % rej. is the percentage of rejection of the test of the hypothesis $H_0:\theta ={\theta }^0$ across the simulations.

Parameter:	Sample	(a)	(b)	(c)	(d)	(e)
True Value:	Size $\mathit{n}$	emp.est.	emp.s.e.	est.s.e.	theor.s.e.	% rej.
$A_{11}^{\prime }$: $0.8$	25	0.7507	0.1295	0.0997	0.1136	13.0
	50	0.7843	0.0811	0.0692	0.0767	8.5
	100	0.7889	0.0558	0.0509	0.0530	6.7
	200	0.7951	0.0374	0.0373	0.0370	6.3
	400	0.7971	0.0269	0.0258	0.0260	5.7
$A_{22}^{″}$: $0.75$	25	0.6014	0.6437	0.5440	0.7030	15.7
	50	0.7004	0.4759	0.4305	0.4827	9.8
	100	0.7041	0.3461	0.3099	0.3363	7.9
	200	0.7347	0.2364	0.2224	0.2360	6.8
	400	0.7461	0.1690	0.1604	0.1662	6.2
${\eta }_{22}$: $0.7$	25	0.6969	0.7711	0.5630	0.7442	18.3
	50	0.7035	0.5422	0.4310	0.5369	11.8
	100	0.7084	0.3865	0.3249	0.3834	11.9
	200	0.7078	0.2684	0.2449	0.2725	8.3
	400	0.6886	0.1948	0.1801	0.1932	7.5

and

Table 2. Estimation results for the tdVA${\mathrm{R}}^{\left(n\right)}$(1) model (8) under (10) with Student errors with 5 degrees of freedom. (a) emp.est. is the empirical estimate (average over the simulation estimates), (b) emp.s.e. is the empirical standard error (standard deviation over the simulation estimates), (c) est.s.e. is the estimated standard error (average of the standard errors as a by-product of estimation), (d) theor.s.e. is the (approximate) theoretical standard error (finite approximation of standard error using (4)), and (e) % rej. is the percentage of rejection of the test of the hypothesis $H_0:\theta ={\theta }^0$ across the simulations.

Parameter:	Sample	(a)	(b)	(c)	(d)	(e)
True Value:	Size $\mathit{n}$	emp.est.	emp.s.e.	est.s.e.	theor.s.e.	% rej.
$A_{11}^{\prime }$: $0.8$	25	0.7608	0.1193	0.0960	0.1136	12.1
	50	0.7773	0.0855	0.0706	0.0767	8.6
	100	0.7918	0.0539	0.0504	0.0530	6.1
	200	0.7938	0.0387	0.0363	0.0370	5.8
	400	0.7977	0.0260	0.0257	0.0260	4.7
$A_{22}^{″}$: $0.75$	25	0.6101	0.6910	0.5501	0.7030	18.5
	50	0.7153	0.4564	0.4290	0.4827	9.3
	100	0.7303	0.3191	0.3097	0.3363	6.6
	200	0.7324	0.2343	0.2237	0.2360	6.3
	400	0.7464	0.1698	0.1607	0.1662	6.6
${\eta }_{22}$: $0.7$	25	0.7051	0.6554	0.5366	1.2453	12.6
	50	0.7102	0.4976	0.3946	0.8984	12.4
	100	0.6936	0.3844	0.3092	0.6416	11.2
	200	0.7072	0.2796	0.2391	0.4560	9.0
	400	0.7052	0.2068	0.1803	0.3232	7.1

.

In both cases (multivariate Laplace and multivariate Student), the estimates in column (a)  (emp.est.) are close to the true value, and nearly always closer when n increases; the sample standard errors in column (b)  (emp.s.e.) decrease with n and are very close to the averages across simulations of the estimated standard errors (obtained using the sandwich formula and estimates $\widehat{V}$ and $\widehat{W}$ of V and W, respectively, see [52] for details) shown in column (c)  (est.s.e.) and the approximate theoretical standard errors in column (d)   (theor.s.e.), also based on a sandwich formula but now on the two finite averages of (4) evaluated at ${\theta }^0$, as discussed in Section 2.3; the percentages of simulations where the hypothesis $H_0({\theta }_i={\theta }_i^0)$ is rejected at the 5% level in column  (e)  (% rej.) are, of course, close to 5%. The results look better for the Laplace than the Student distribution, especially for column (d)   and row ${\eta }_{22}$. As a consequence of these simulations, we see that, for n large enough, the estimates become close to the true value, the standard errors that are the by-product of estimation correspond broadly to the empirical results (although less well for the parameter ${\eta }_{22}$ related to heteroscedasticity), coincide relatively well with the approximated values derived from the theory, and, finally, the level of the test that the parameter equals the true value is relatively close to 5%. Thanks to the sandwich correction, the standard errors are not underestimated. For instance, for ${\eta }_{22}$, $n=400$, and the Laplace distribution, the average of the standard errors using ${\widehat{V}}^{-1}$ instead of ${\widehat{V}}^{-1}\widehat{W}{\widehat{V}}^{-1}$ would lead to $0.1239$ instead of $0.1801$ as shown in column (c)  and $0.1222$ instead of $0.1932$ in column (d). These numbers are far from the empirical standard deviation of $0.1948$. Finally, the results in [43] for a normal distribution in an otherwise similar setup are closer to the expected values than those shown here. For instance, the last row shows the results $0.6913$, $0.1229$, $0.1235$, $0.1222$, and $4.3$ in columns (a) to (e), respectively, with a much better agreement for the empirical, estimated, and theoretical standard errors.

3.2. Treatment of a tdVM$A^{\left(n\right)}$(1) Model

Moving average models are more difficult to study than autoregressive ones. We consider a tdVM${\mathrm{A}}^{\left(n\right)}$(1) model defined by

$$\begin{array}{c}\hfill x_t^{\left(n\right)}=e_t^{\left(n\right)}\left(\theta \right)+B_t^{\left(n\right)}\left(\theta \right)e_{t-1}^{\left(n\right)}\left(\theta \right),\end{array}$$

(17)

with the same notations as in Section 2, and $B_t^{\left(n\right)}\left(\theta \right)=B_{t1}^{\left(n\right)}\left(\theta \right)$. For any $\theta$, the pure autoregressive representation of $x_t^{\left(n\right)}$ is

$$x_t^{\left(n\right)}=e_t^{\left(n\right)}\left(\theta \right)+B_t^{\left(n\right)}\left(\theta \right)x_{t-1}^{\left(n\right)}-B_t^{\left(n\right)}\left(\theta \right)B_{t-1}^{\left(n\right)}\left(\theta \right)x_{t-2}^{\left(n\right)}+B_t^{\left(n\right)}\left(\theta \right)B_{t-1}^{\left(n\right)}\left(\theta \right)B_{t-2}^{\left(n\right)}\left(\theta \right)x_{t-3}^{\left(n\right)}-\dots ,$$

hence

$${\displaystyle \frac{\partial e_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}=-{\displaystyle \frac{\partial B_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}{x}_{t-1}^{\left(n\right)}+{\displaystyle \frac{\partial B_t^{\left(n\right)}\left(\theta \right)B_{t-1}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}{x}_{t-2}^{\left(n\right)}-{\displaystyle \frac{\partial B_t^{\left(n\right)}\left(\theta \right)B_{t-1}^{\left(n\right)}\left(\theta \right)B_{t-2}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}{x}_{t-3}^{\left(n\right)}+\dots$$

Replacing $x_{t-j}^{\left(n\right)}$, $j=1,2,\dots$ by (17) for $\theta ={\theta }^0$, given that $e_t^{\left(n\right)}\left({\theta }^0\right)=g_t^{\left(n\right)}\left({\theta }^0\right){ϵ}_t$, we obtain

$$\begin{array}{ccc}\hfill {\displaystyle \frac{\partial e_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}& =& -{\displaystyle \frac{\partial B_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}(g_{t-1}^{\left(n\right)}{ϵ}_{t-1}+B_{t-1}^{\left(n\right)}{g}_{t-2}^{\left(n\right)}{ϵ}_{t-2})\hfill \\ & +& {\displaystyle \frac{\partial B_t^{\left(n\right)}\left(\theta \right)B_{t-1}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}(g_{t-2}^{\left(n\right)}{ϵ}_{t-2}+B_{t-2}^{\left(n\right)}{g}_{t-3}^{\left(n\right)}{ϵ}_{t-3})\hfill \\ & -& {\displaystyle \frac{\partial B_t^{\left(n\right)}\left(\theta \right)B_{t-1}^{\left(n\right)}\left(\theta \right)B_{t-2}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}(g_{t-3}^{\left(n\right)}{ϵ}_{t-3}+B_{t-3}^{\left(n\right)}{g}_{t-4}^{\left(n\right)}{ϵ}_{t-4})+\dots ,\hfill \end{array}$$

with $g_t^{\left(n\right)}=g_t^{\left(n\right)}\left({\theta }^0\right)$ and $B_t^{\left(n\right)}=B_t^{\left(n\right)}\left({\theta }^0\right)$, like before. Let us denote

$$B_t^{\left(n\right)[k-1]}\left(\theta \right)=\prod _{j=0}^{k-1}{B}_{t-j}^{\left(n\right)}\left(\theta \right),$$

(18)

with elements $B_{t,ij}^{\left(n\right)[k-1]}\left(\theta \right)$, $i,j=1,2$. Hence, from (3),

$${\psi }_{tik}^{\left(n\right)}\left(\theta \right)={(-1)}^k\left[{\displaystyle \frac{\partial B_t^{\left(n\right)[k-1]}\left(\theta \right)}{\partial {\theta }_i}}-{\displaystyle \frac{\partial B_t^{\left(n\right)[k-2]}\left(\theta \right)}{\partial {\theta }_i}}{B}_{t-k+1}^{\left(n\right)}\right].$$

(19)

Assume now $B_{t,21}^{\left(n\right)}\left(\theta \right)=0$ in $P_{\theta }$. To proceed more in detail, in order to simplify the analytic computations (and this is not necessary for numerical computations), we assume also that $B_{t,12}^{\left(n\right)}\left(\theta \right)=B_{12}^{\prime }$, a constant.

We can now examine the typical assumptions of the theory stated in Section 2.2.

3.2.1. Assumptions (i) and (ii)

To obtain nice analytic expressions, we suppose that the diagonal elements of the matrices $B_t^{\left(n\right)}\left(\theta \right)$ and $g_t^{\left(n\right)}\left(\theta \right)$ are exponential functions of time. The elements $(1,2)$ and $(2,1)$ of $g_t^{\left(n\right)}\left(\theta \right)$ are supposed to be different from zero so that the correlation between the two components of ${ϵ}_t$ varies with time. More precisely, using again $L(t,n)={\displaystyle \frac{1}{n-1}}(t-{\displaystyle \frac{n+1}{2}})$, we suppose that

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{B}_t^{\left(n\right)}\left(\theta \right)=\left(\begin{array}{cc}{B}_{11}^{\prime }exp\left\{B_{11}^{″}L(t,n)\right\}& B_{12}^{\prime }\\ 0& B_{22}^{\prime }exp\left\{B_{22}^{″}L(t,n)\right\}\end{array}\right),\hfill \\ g_t^{\left(n\right)}\left(\theta \right)=\left(\begin{array}{cc}exp\left\{{\eta }_{11}L(t,n)\right\}& \alpha \\ \beta & exp\left\{{\eta }_{22}L(t,n)\right\}\end{array}\right)\mathrm{and}\mathsf{\Sigma }=\left(\begin{array}{cc}{\sigma }_{11}& {\sigma }_{12}\\ {\sigma }_{21}& {\sigma }_{22}\end{array}\right),\hfill \end{array}\right.\end{array}$$

(20)

where $\alpha$ and $\beta$ are here constants, with ${\eta }_{11}\ge 0$ and ${\eta }_{22}\ge 0$. Denoting $g_{t,ii}^{\left(n\right)}$ an element of $g_t^{\left(n\right)}\left(\theta \right)$ and omitting the argument $\theta$, ${\mathsf{\Sigma }}_t^{\left(n\right)}\left(\theta \right)$ is equal to

$$\left(\begin{array}{cc}{\sigma }_{11}{g}_{t,11}^{\left(n\right)2}+2{\sigma }_{12}\alpha g_{t,11}^{\left(n\right)}+{\sigma }_{22}{\alpha }^2& {\sigma }_{11}\beta g_{t,11}^{\left(n\right)}+{\sigma }_{12}(\alpha \beta +g_{t,11}^{\left(n\right)}{g}_{t,22}^{\left(n\right)})\\ & +{\sigma }_{22}\alpha g_{t,22}^{\left(n\right)}\\ {\sigma }_{11}\beta g_{t,11}^{\left(n\right)}+{\sigma }_{12}(\alpha \beta +g_{t,11}^{\left(n\right)}{g}_{t,22}^{\left(n\right)})& {\sigma }_{11}{\beta }^2+2{\sigma }_{12}\beta g_{t,22}^{\left(n\right)}+{\sigma }_{22}{g}_{t,22}^{\left(n\right)2}\\ +{\sigma }_{22}\alpha g_{t,22}^{\left(n\right)}\end{array}\right).$$

(21)

It is easy to see that Assumptions (i) and (ii) are verified.

3.2.2. Assumption (iii)

To simplify the example, since a correlation between the two components is already guaranteed by the presence of a non-diagonal covariance matrix $\mathsf{\Sigma }$, we will assume that $B_{12}^{\prime }=0$ so that $B_t^{\left(n\right)[k-1]}\left(\theta \right)$ defined in (18) is diagonal.

To simplify the discussion, we will further suppose, similarly to what we did in Section 3.1, in addition of course to the already introduced parameters ${\eta }_{11}$ and ${\eta }_{22}$, we have only one parameter of each type, e.g., ${\theta }_1=B_{11}^{\prime }$ and ${\theta }_2=B_{22}^{″}$, assuming that $B_{22}^{\prime }\ne 0$ and $B_{11}^{″}$ are fixed constants, respectively denoted $B_{22}^{\prime 0}$ and $B_{11}^{″0}$. To summarize, $\theta ={\left(B_{11}^{\prime }{B}_{22}^{″}{\eta }_{11}{\eta }_{22}\right)}^T$. Contrary to Section 3.1, we recourse to the results of [54] here, although the more lengthy direct analysis is given in Appendix A.2. Indeed, a sufficient condition for (iii) is stated in [54]. It can be shown that the eigenvalues of the MA polynomial are the solutions of the equation $(1-B_{11}^{\prime }exp\left(B_{11}^{″0}L(t,n)\right))(1-B_{22}^{\prime 0}exp\left(B_{22}^{″}L(t,n)\right))=0$. They should be smaller than 1 in absolute value at ${\theta }^0$. Therefore, we assume that the true value ${\theta }^0={\left(B_{11}^{\prime 0}{B}_{22}^{″0}{\eta }_{11}^0{\eta }_{22}^0\right)}^T$ of $\theta$ satisfies $|B_{jj}^{\prime 0}|exp(B_{jj}^{″0}/2)<1$, $j=1,2$. We denote

$${\mathsf{\Phi }}^{1/2}=max\left\{\right|B_{11}^{\prime 0}|exp(B_{11}^{″0}/2),|B_{22}^{\prime 0}|exp(B_{22}^{″0}/2)\}<1.$$

(22)

Assumption (iii) is, therefore, satisfied. In practice, we don’t assume zero initial values for the process, but that it is invertible before time 1. This leads to additional conditions that $|B_{11}^{\prime 0}|exp\left(B_{11}^{″0}L(0,n)\right)<1$ and $|B_{22}^{\prime 0}|exp\left(B_{22}^{″0}L(0,n)\right)<1$. If we were interested in forecasting up to time $n+h$, we should replace above $L(t,n)$ with $L(t+h,n)$ to have stronger conditions in (22).

3.2.3. Assumption (iv)

Now, we consider the existence of V in (4) with (5) for the 4-parameter model described in the previous paragraphs. Let us start with the elements $V_{ij}$, $i,j=3,4$, corresponding to the parameters ${\eta }_{11}$ and ${\eta }_{22}$. The formulas in [51] (Theorem 3) are applicable for computing the terms $V_{t,ij}^{\left(n\right)}$, $i,j=3,4$. Given the expression of $g_{t,\ell \ell }^{\left(n\right)}$ in (20), the terms in these formulas are all of the forms

$$v_t^{\left(n\right)}={\displaystyle \frac{L{(t,n)}^2{g}_{t,\xi }^{\left(n\right)}}{{\left(\alpha \beta -g_{t,\eta }^{\left(n\right)}\right)}^2}},$$

(23)

where $\xi \ge 0$ is equal to $\rho {\eta }_{\ell \ell }$, $\rho =0,1,2$, and $\ell =1,2$, or ${\eta }_{11}{\eta }_{22}$, and $\eta ={\eta }_{11}+{\eta }_{22}>0$. See Appendix A.3 for indications on how to prove the existence of the limit of averages of these terms, and even their computation using integration.

Using (7), the elements of $V_{t,ij}^{\left(n\right)}$, $i,j=1,2$ are sums for $k=1$ to $t-1$ of terms of the kind ${\tau }_{t,ij}^{\left(n\right)}{({\psi }_{tik}^{\left(n\right)})}_{ii}{({\psi }_{tjk}^{\left(n\right)})}_{jj}{\sigma }_{t-k}^{\left(n\right)}$, where ${\tau }_{t,ij}^{\left(n\right)}$ are elements of ${\mathsf{\Sigma }}_t^{\left(n\right)-1}$ and ${\sigma }_{t-k}^{\left(n\right)}$ are elements of ${\mathsf{\Sigma }}_{t-k}^{\left(n\right)}$. Given (21), the special form of $g_t^{\left(n\right)}\left(\theta \right)$, ${\sigma }_t^{\left(n\right)}$ is a quadratic polynomial of the diagonal elements $g_{t,\ell \ell }^{\left(n\right)}$, $\ell =1,2$. More precisely, ${\sigma }_t^{\left(n\right)}$ can be written as a finite sum of the form $c_{{\delta }_1{\delta }_2}{(g_{t,11}^{\left(n\right)})}^{{\delta }_1}{(g_{t,22}^{\left(n\right)})}^{{\delta }_2}$, with ${\delta }_1$ and ${\delta }_2$ integers such that $0\le {\delta }_1\le 2$, $0\le {\delta }_2\le 2$, and ${\delta }_1+{\delta }_2\le 2$, and $c_{{\delta }_1{\delta }_2}$ is a constant. However, $g_{t-k,\ell \ell }^{\left(n\right)}$ is equal to $g_{t,\ell \ell }^{\left(n\right)}exp(-{\eta }_{\ell \ell }^{0}k/(n-1))$. Consequently, ${\sigma }_{t-k}^{\left(n\right)}$ is composed of terms $c_{{\delta }_1{\delta }_2}{\left(g_{t-k,11}^{\left(n\right)}\right)}^{{\delta }_1}{\left(g_{t-k,22}^{\left(n\right)}\right)}^{{\delta }_2}$ equal to $c_{{\delta }_1{\delta }_2}{\left(g_{t,11}^{\left(n\right)}\right)}^{{\delta }_1}{\left(g_{t,22}^{\left(n\right)}\right)}^{{\delta }_2}exp(-({\eta }_{11}^0{\delta }_1+{\eta }_{22}^0{\delta }_2)k/(n-1))$. Hence $V_{t,ij}^{\left(n\right)}$, $i,j=1,2$, is a sum for $k=1$ to $t-1$ of terms of the kind

$$c_{{\delta }_1{\delta }_2}{\tau }_{t,ij}^{\left(n\right)}{\left(g_{t,11}^{\left(n\right)}\right)}^{{\delta }_1}{\left(g_{t,22}^{\left(n\right)}\right)}^{{\delta }_2}{u}_{t,ij{\delta }_1{\delta }_2}^{\left(n\right)},$$

where we denote

$$u_{t,ij{\delta }_1{\delta }_2}^{\left(n\right)}=\sum _{k=1}^{t-1}{\left({\psi }_{tik}^{\left(n\right)}\right)}_{ii}{\left({\psi }_{tjk}^{\left(n\right)}\right)}_{jj}exp(-({\eta }_{11}^0{\delta }_1+{\eta }_{22}^0{\delta }_2)k/(n-1)),$$

(24)

where ${\left({\psi }_{tik}^{\left(n\right)}\right)}_{ii}$ and ${\left({\psi }_{tjk}^{\left(n\right)}\right)}_{jj}$ are obtained using (A2) or/and (A3). The terms in (24) are not all positive because of $L(t-k+1,n)$. However, $L(t-k+1,n)<1/2$ for all $t=1,\dots ,n$ and all n, and, therefore, ${\sum }_{\ell =1}^{k-1}L(t-\ell ,n)\le k/2$ so that the absolute value of the product ${\left({\psi }_{tik}^{\left(n\right)}\right)}_{ii}{\left({\psi }_{tjk}^{\left(n\right)}\right)}_{jj}$ can be bounded by ${\mathsf{\Phi }}^k$, given (22). We want to show that the triangular array $\{u_{t,ij{\delta }_1{\delta }_2}^{\left(n\right)},t=1,\dots ,n\}$ converges to a constant (dependent on i, j, ${\delta }_1$, and ${\delta }_2$) when $t\to \infty$, and hence $n\to \infty$. First, the absolute value of the kth term of (24) is bounded by ${\mathsf{\Phi }}^{k}exp(-({\eta }_{11}^0{\delta }_1+{\eta }_{22}^0{\delta }_2)k/(n-1))<{\mathsf{\Phi }}^k$, since ${\eta }_{11}^0{\delta }_1+{\eta }_{22}^0{\delta }_2>0$, and the limit of the sum of these terms is a convergent geometric series, ensuring convergence of the sequence (24) when $t\to \infty$. Now, for $i,j=1,2$, $V_{t,ij}^{\left(n\right)}$ is a sum of terms that are proportional to $u_{t,ij{\delta }_1{\delta }_2}^{\left(n\right)}{v}_{t,ij{\delta }_1{\delta }_2}^{\left(n\right)}$, where

$$v_{t,ij{\delta }_1{\delta }_2}^{\left(n\right)}={\tau }_{t,ij}^{\left(n\right)}{\left(g_{t,11}^{\left(n\right)}\right)}^{{\delta }_1}{\left(g_{t,22}^{\left(n\right)}\right)}^{{\delta }_2}.$$

We have defined ${\tau }_{t,ij}^{\left(n\right)}$ as elements of ${\mathsf{\Sigma }}_t^{\left(n\right)-1}$, so as elements of the matrix of cofactors of the matrix ${\mathsf{\Sigma }}_t^{\left(n\right)}$ divided by its determinant $D_t^{\left(n\right)}$, where $D_t^{\left(n\right)}$ is $({\sigma }_{11}{\sigma }_{22}-{\sigma }_{12}^2){(\alpha \beta -g_{t,11}^{\left(n\right)}{g}_{t,22}^{\left(n\right)})}^2$. Hence, $v_{t,ij{\delta }_1{\delta }_2}^{\left(n\right)}$ is a finite sum with terms that are proportional to

$$v_{t,ij{\delta }_3{\delta }_4}^{\left(n\right)}={\displaystyle \frac{{\left(g_{t,11}^{\left(n\right)}\right)}^{{\delta }_3}{\left(g_{t,22}^{\left(n\right)}\right)}^{{\delta }_4}}{{(\alpha \beta -g_{t,11}^{\left(n\right)}{g}_{t,22}^{\left(n\right)})}^2}},$$

where ${\delta }_3\ge {\delta }_1$ and ${\delta }_4\ge {\delta }_2$ are integers. We obtain $V_{t,ij}^{\left(n\right)}$ for $i,j=3,4$ to show that

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}\sum _{t=1}^n{v}_{t,ij{\delta }_3{\delta }_4}^{\left(n\right)}$$

exists, is finite, and can be evaluated using an integral. Therefore, we can invoke Theorem 2.5 of [55] to show that

$$\underset{n\to \infty }{lim}{\displaystyle \frac{1}{n}}\sum _{t=1}^n{u}_{t,ij{\delta }_1{\delta }_2}^{\left(n\right)}{v}_{t,ij{\delta }_3{\delta }_4}^{\left(n\right)}$$

exists and is finite. This proves the existence of $V_{ij}$ for $i,j=1,2$.

3.2.4. Assumption (v)

The existence of W in (4) with (6) can be studied similarly using ${\kappa }_t$, depending on the distribution of ${ϵ}_t$, see [52] (Section 3.5).

3.2.5. Assumption (vi)

It is also easy to check Assumption (vi), for example for $i=1$. We have to show that $(1/n^2)$ multiplied by ${\sum }_{d=1}^{n-1}{\sum }_{t=1}^{n-d}{\sum }_{k=1}^{t-1}\parallel g_{t-k}^{\left(n\right)}{\parallel }_F^2\parallel {\psi }_{t1k}^{\left(n\right)}{\parallel }_F{\parallel {\psi }_{t+d,1,k+d}^{\left(n\right)}\parallel }_F$ is $O(1/n)$. First, it can be shown that $\parallel g_{t-k}^{\left(n\right)}{\parallel }_F^2$ is equal to $2+exp\left(2{\eta }_{11}^{0}L(t-k,n)\right)+exp\left(2{\eta }_{22}^{0}L(t-k,n)\right)$, and this is less than or equal to $2+exp\left({\eta }_{11}^0\right)+exp\left({\eta }_{22}^0\right)$. Then, we are exactly in the same situation as in Section 3.1.

Example 2. 

In [43] (Section 4.1.2), simulations results were shown for artificial Gaussian or Student (with 5 degrees of freedom) time series generated by model Equations (17)–(20) with the following values for ${\theta }^0={(B_{11}^{\prime 0},B_{22}^{″0},{\eta }_{22}^0)}^T={(0.8,0.4,0.7)}^T$ but with a linear expression for $B_t^{\left(n\right)}$ that we keep here instead of the exponential that was easier for the analytic developments: $B_{t,11}^{\left(n\right)}=B_{11}^{\prime }+B_{11}^{″}L(t,n)$ and $B_{t,22}^{\left(n\right)}=B_{22}^{\prime }+B_{22}^{″}L(t,n)$. Moreover, $B_{11}^{″}=0$, $B_{22}^{\prime }=0.25$, and ${\eta }_{11}=0$ are taken as constants, as well as $\alpha =\beta =-0.6$. First, note that these values satisfy the requirements after a linear approximation of the exponentials. Indeed, ${\mathsf{\Phi }}^{1/2}=max\left(\right|B_{11}^{\prime 0}|exp(B_{11}^{″}/2),|B_{22}^{\prime 0}|exp(B_{22}^{″}/2))<1$ and ${\eta }_{22}>0$.

Let us consider 1000 time series of length $n=25,50,100,200,\mathrm{and}400$ obtained using multivariate Laplace distribution. The empirical estimation results are shown in

Table 3. Estimation results for the tdVM${\mathrm{A}}^{\left(n\right)}$(1) model (8) under (10) but with linear expressions for $B_{t,11}^{\left(n\right)}$ and $B_{t,22}^{\left(n\right)}$, and with Laplace errors. (a) emp.est. is the empirical estimate (average over the simulation estimates), (b) emp.s.e. is the empirical standard error (standard deviation over the simulation estimates), (c) est.s.e. is the estimated standard error (average of the standard errors as a by-product of estimation), (d) theor.s.e. is the (approximate) theoretical standard error (finite approximation of standard error using (4)), and (e) % rej. is the percentage of rejection of the test of the hypothesis $H_0:\theta ={\theta }^0$ across the simulations.

Parameter:	Sample	(a)	(b)	(c)	(d)	(e)
True Value:	Size $\mathit{n}$	emp.est.	emp.s.e.	est.s.e.	theor.s.e.	% rej.
$B_{11}^{\prime }$: $0.8$	25	0.7827	0.1045	0.0989	0.0913	11.5
	50	0.8014	0.0696	0.0658	0.0614	9.0
	100	0.8043	0.0453	0.0435	0.0423	9.0
	200	0.8026	0.0310	0.0301	0.0295	5.9
	400	0.8012	0.0216	0.0208	0.0207	7.3
$B_{22}^{″}$: $0.4$	25	0.3850	0.4856	0.4473	0.4722	15.6
	50	0.4082	0.3412	0.3069	0.3207	10.9
	100	0.3934	0.2308	0.2149	0.2223	7.8
	200	0.3943	0.1606	0.1508	0.1557	6.2
	400	0.4004	0.1127	0.1078	0.1095	6.2
${\eta }_{22}$: $0.7$	25	0.7458	0.5624	0.3567	0.4424	24.4
	50	0.7349	0.3492	0.2626	0.3200	16.7
	100	0.7168	0.2415	0.1973	0.2289	14.1
	200	0.7115	0.1658	0.1455	0.1627	10.0
	400	0.6950	0.1190	0.1078	0.1154	8.7

. In all cases, the estimates in column (a)  (emp.est.) are often close to the true value, and closer when n increases; the sample standard errors in column (b)  (est.s.e.) decrease with n and are very close to the averages across simulations of the estimated standard errors (obtained using the sandwich formula and estimates $\widehat{V}$ and $\widehat{W}$ of V and W, respectively, see [52] for details) shown in column (c)  (emp.s.e.) and the approximate standard errors in column (d)  (theor.s.e.), also based on a sandwich formula but now on the two finite averages of (4) evaluated at ${\theta }^0$; the percentages of simulations where the hypothesis $H_0({\theta }_i={\theta }_i^0)$ is rejected at the 5% level in column (e)   (% rej.) are close to $0.05$ at least for large n. Like in Example 1, for n large enough, the estimates become close to the true value, the estimated standard errors are close to the empirical results (but also for the parameter ${\eta }_{22}$ related to heteroscedasticity) and the approximated values derived from the theory, and, finally, the percentage of rejection in the tests are slightly closer to 5%. Thanks to the sandwich correction, the standard errors are not underestimated. For instance, for ${\eta }_{22}$ and $n=400$, the average of the standard errors using ${\widehat{V}}^{-1}$ instead of ${\widehat{V}}^{-1}\widehat{W}{\widehat{V}}^{-1}$ would lead to $0.0746$ instead of $0.1078$ shown in column (c)   and $0.0731$ instead of $0.1154$ in column (d)   , far from the empirical standard error of $0.1190$ shown in column (b). Note that the results are slightly better for a normal distribution of the errors. In

Table 4. Estimation results for the tdVM${\mathrm{A}}^{\left(n\right)}$(1) model, $n=400$, with normal errors. (a) emp.est. is the empirical estimate (average over the simulation estimates), (b) emp.s.e. is the empirical standard error (standard deviation over the simulation estimates), (c1) and (c2) est.s.e. are the estimated standard errors (average of the standard errors as a by-product of estimation), using $\widehat{V}$ or ${\widehat{V}}^{-1}\widehat{W}{\widehat{V}}^{-1}$, respectively; (d) theor.s.e. is the (approximate) theoretical standard error (finite approximation of standard error using (4)), and (e) % rej. is the percentage of rejection of the test of the hypothesis $H_0:\theta ={\theta }^0$ across the simulations.

Parameter ${\mathit{\theta }}_{\mathit{i}}$	${\mathit{B}}_{\mathbf{11}}^{\mathbf{\prime }}$	${\mathit{B}}_{\mathbf{22}}^{\mathbf{″}}$	${\mathit{\eta }}_{\mathbf{22}}$
True value ${\theta }_i^0$	0.8000	0.4000	0.7000
(a) emp.est.	0.8010	0.4007	0.6977
(b) emp.s.e.	0.0216	0.1134	0.0747
(c1) est.s.e., based on $\widehat{V}$	0.0210	0.1105	0.0744
(c2) est.s.e., based on ${\widehat{V}}^{-1}\widehat{W}{\widehat{V}}^{-1}$	0.0208	0.1095	0.0729
(d) theor.s.e.	0.0207	0.1095	0.0731
(e) % rej.	7.1	6.0	5.5

, we present the normal results obtained for $n=400$ as we had only shown them for $n=100$ in [43]. The agreement is better for ${\eta }_{22}$ with $0.0747$ in row (b)  (emp.s.e.), $0.0744$ in row (c1)   (est.s.e.) or $0.0729$ in row (c2)   (est.s.e.) or $0.0731$ in row (d)   (theor.s.e.). In the normal case, we have computed an estimate of $V_{33}$ using the integration approach in [55] (Procedure 6). Using this, refer to the details in Appendix A.3, $V_{33}=0.465316$ and the corresponding standard error for ${\eta }_{22}$ when $n=400$ is $0.0732986$, in full agreement with the empirical, estimated, and theoretical approximation standard errors in the last column of Table 4.

3.3. Treatment of a tdVARM$A^{\left(n\right)}$(1,1) Model

Starting from here, we consider only homoscedastic models. Let $\{x_t^{\left(n\right)};t=1,\dots ,n,n\in \mathbf{N}\}$ be an r-vector time series satisfying

$$x_t^{\left(n\right)}=A_t^{\left(n\right)}\left(\theta \right)x_{t-1}^{\left(n\right)}+e_t^{\left(n\right)}\left(\theta \right)+B_t^{\left(n\right)}\left(\theta \right)e_{t-1}^{\left(n\right)}\left(\theta \right),$$

and $g_t^{\left(n\right)}\left(\theta \right)=I_r$. Hence, (ii) and (v) have no object since ${\mathsf{\Sigma }}_t^{\left(n\right)}\left(\theta \right)=\mathsf{\Sigma }$ does not depend on $\theta$ and $W=V$. We will not cover the tdVARM${\mathrm{A}}^{\left(n\right)}(1,1)$ model in detail, simply show how the coefficients ${\psi }_{tik}^{\left(n\right)}$ can be computed. It is a special case of the model defined in (1) with $p=1$ and $q=1$. Now, following [44], in this special case, the coefficients of the pure moving average representation are given by:

$${\psi }_{tk}^{\left(n\right)}\left(\theta \right)=\left\{\prod _{l=0}^{k-2}{A}_{t-l}^{\left(n\right)}\left(\theta \right)\right\}\left\{B_{t-k+1}^{\left(n\right)}\left(\theta \right)+A_{t-k+1}^{\left(n\right)}\left(\theta \right)\right\},\mathrm{for}k=1,2,\dots ,t-1,$$

where a product for $l=0$ to $-1$ is set to $I_r$. The coefficients of the pure autoregressive form are

$${\pi }_{tk}^{\left(n\right)}\left(\theta \right)=\left\{{(-1)}^{k-1}\prod _{l=0}^{k-2}{B}_{t-l}^{\left(n\right)}\left(\theta \right)\right\}\left\{A_{t-k+1}^{\left(n\right)}\left(\theta \right)+B_{t-k+1}^{\left(n\right)}\left(\theta \right)\right\},$$

so, for $i=1,\dots ,m$, their derivatives are given by

$${\displaystyle \frac{\partial {\pi }_{t1}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}={\displaystyle \frac{\partial A_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}+{\displaystyle \frac{\partial B_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}},$$

$${\displaystyle \frac{\partial {\pi }_{t2}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}=-{\displaystyle \frac{\partial B_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}\{A_{t-1}^{\left(n\right)}\left(\theta \right)+B_{t-1}^{\left(n\right)}\left(\theta \right)\}-B_t^{\left(n\right)}\left(\theta \right)\left\{{\displaystyle \frac{\partial A_{t-1}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}+{\displaystyle \frac{\partial B_{t-1}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}\right\},$$

$$\begin{array}{ccc}\hfill {\displaystyle \frac{\partial {\pi }_{t3}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}& =& {\displaystyle \frac{\partial B_t^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}{B}_{t-1}^{\left(n\right)}\left(\theta \right)\left\{A_{t-2}^{\left(n\right)}\left(\theta \right)+B_{t-2}^{\left(n\right)}\left(\theta \right)\right\}\hfill \\ \hfill & +& B_t^{\left(n\right)}\left(\theta \right){\displaystyle \frac{\partial B_{t-1}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}\left\{A_{t-2}^{\left(n\right)}\left(\theta \right)+B_{t-2}^{\left(n\right)}\left(\theta \right)\right\}\hfill \\ & +& B_t^{\left(n\right)}\left(\theta \right)B_{t-1}^{\left(n\right)}\left(\theta \right)\left\{{\displaystyle \frac{\partial A_{t-2}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}+{\displaystyle \frac{\partial B_{t-2}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}\right\},\dots \hfill \end{array}$$

Consequently

$${\displaystyle \frac{\partial {\pi }_{tk}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}={(-1)}^{k-1}\sum _{l=1}^k\left(\prod _{h=1}^k{\chi }_{t+1-h,k,l,h,i}^{\left(n\right)}\left(\theta \right)\right),$$

where

$${\chi }_{t,k,l,h,i}^{\left(n\right)}\left(\theta \right)=\left\{\begin{array}{ccc}{\displaystyle \frac{\partial {\chi }_{t,k,l,h}^{\left(n\right)}\left(\theta \right)}{\partial {\theta }_i}}\hfill & \mathrm{if}\hfill & h=l,\hfill \\ {\chi }_{t,k,l,h}^{\left(n\right)}\left(\theta \right)\hfill & \mathrm{if}\hfill & h\ne l,\hfill \end{array}\right.$$

and

$${\chi }_{t,k,l,h}^{\left(n\right)}\left(\theta \right)=\left\{\begin{array}{cc}{B}_t^{\left(n\right)}\left(\theta \right)\hfill & ifh<k,\hfill \\ A_t^{\left(n\right)}\left(\theta \right)+B_t^{\left(n\right)}\left(\theta \right)\hfill & ifh=k.\hfill \end{array}\right.$$

Then

$${\psi }_{tik}^{\left(n\right)}\left(\theta \right)=\sum _{u=1}^k\left\{\sum _{l=1}^u\left(\prod _{h=1}^u{\chi }_{t+1-h,k,l,h,i}^{\left(n\right)}\left(\theta \right)\right)\right\}\left\{\prod _{h=u+1}^k{\tilde{\chi }}_{t+1-h,k,h}^{\left(n\right)}\left({\theta }^0\right)\right\},$$

$${\tilde{\chi }}_{t+1-h,k,h}^{\left(n\right)}\left(\theta \right)=\left\{\begin{array}{cc}{A}_t^{\left(n\right)}\left(\theta \right)\hfill & ifh<k,\hfill \\ A_t^{\left(n\right)}\left(\theta \right)+B_t^{\left(n\right)}\left(\theta \right)\hfill & ifh=k.\hfill \end{array}\right.$$

These results correct the findings presented in the univariate case by [12].

From this expression, it is possible to check (iii) and (vi) relatively easily. Indeed, again let $A_t^{\left(n\right)}=A_t^{\left(n\right)}\left({\theta }^0\right)$ and $B_t^{\left(n\right)}=B_t^{\left(n\right)}\left({\theta }^0\right)$. If we assume that $\parallel A_t^{\left(n\right)}{\parallel }_F <\mathsf{\Phi }$ and $\parallel B_t^{\left(n\right)}{\parallel }_F <\mathsf{\Phi }$, for all t, where $0<\mathsf{\Phi }<1$, since only a few factors involve sums like $A_t^{\left(n\right)}+B_t^{\left(n\right)}$ and the others are bounded by $\mathsf{\Phi }$, the Frobenius norm of the ${\psi }_{tk}^{\left(n\right)}$ and the ${\psi }_{tik}^{\left(n\right)}$ are bounded by ${\mathsf{\Phi }}^k$. Of course, checking (iii), the existence of V, depends heavily on the parametrization. For example, if it exists, V will not be invertible if $A_t^{\left(n\right)}\left({\theta }^0\right)=B_t^{\left(n\right)}\left({\theta }^0\right)$ for all t (or even for most t).

3.4. Treatment of a More General tdVARM${\mathrm{A}}^{\left(n\right)}$$(p,q)$ Model

Ref. [45] indicates how to handle more generally homoscedastic tdVARM${\mathrm{A}}^{\left(n\right)}$($p,q$) models, with $g_t^{\left(n\right)}\left(\theta \right)=I_r$. Indeed, it is shown in [54] that (iii) and (vi) are valid if the determinants of the tdVAR and tdVMA polynomials, respectively, $I_r-{\sum }_{i=1}^p{A}_{ti}^{\left(n\right)}{z}^i$ and $I_r+{\sum }_{j=1}^q{B}_{tj}^{\left(n\right)}{z}^j$, do not vanish when $\left|z\right|\le 1$. Of course, it is only a sufficient condition. That argument was used in Section 3.2.2 to simplify the treatment, whereas Appendix A.2 does not use that argument for the same model and is therefore lengthier. Again, (ii) and (v) have no object, while (i) and (iii) depend on the specific parametrization.

4. Discussion

The results presented in Section 3.1 and Section 3.2 confirm that the theory exposed in [43] is applicable and that its assumptions can be checked analytically, at least in the two simple bivariate tdVA${\mathrm{R}}^{\left(n\right)}$(1) and tdVM${\mathrm{A}}^{\left(n\right)}$(1) models. The results have already been exploited in the simulation experiments in [43] (Sections 4.1.2 and 4.2.2) as the values of the parameters used there meet the conditions stated here. The treatment of other models by analytical methods is certainly more challenging, although we discussed it in some detail for a tdVARM${\mathrm{A}}^{\left(n\right)}$(1, 1) model. For more complex models, the approach of [54] can also be considered, as it allows us to put the model in tdVAR form, albeit with a higher dimension.

As mentioned in Remark 1, we have taken linear functions of time for illustrative purposes, but it should be clear that the theory works in whole generality. First, a linear function of time with the divisor $n-1$ appearing in (10) is compatible with Dahlhaus’ LSP theory (see [16]), since the (Frobenius norm of the) coefficient can easily be bounded from above by 1. Anyway, we can consider that linearity is a first attempt if constancy cannot be retained. Second, in [19], the authors have shown a univariate tdA${\mathrm{R}}^{\left(n\right)}$process of order 1, where the coefficient can be greater than 1 during an interval of time that shrinks to 0 as n increases. We can have the same change here, meaning that the process does not need to be locally stationary. What is essential is that the coefficients of the pure tdMA representation are bounded by a decreasing exponential function of the lag. Third, the tdA${\mathrm{R}}^{\left(n\right)}$ coefficient does not need to be differentiable with respect to time. The theory in [43] is valid when there are any number of breaks, provided they do not add too many parameters, for instance, if there is a periodicity of 2, with a coefficient equal to ${\theta }_1$ for t odd, and ${\theta }_2$ for t even. Indeed, it is a generalization of [12] where such an example is shown. We can even have a periodic behavior, even with an incommensurable period for the different matrix entries, as shown by [44].

The lag of one can be replaced with another integer without any change. For instance, for quarterly data, we can replace it with 4, or 12 for monthly data. The results in Section 3.1 and Section 3.2 are easily adapted. Because the inference is infill, meaning that more and more observations are assumed to be made between the first and last ones, the LSP theory is no longer valid in the case of periodicity. On the contrary, the inference in [43,44] is of type outfill or increasing domain, that is to say, that more years are supposed to be observed, therefore preserving the period of the periodic behavior. Ref. [19] has also shown a tdA${\mathrm{R}}^{\left(n\right)}$(1) process with a coefficient that varies linearly with time but where heteroscedasticity (so $g_t^{\left(n\right)}$) is a periodic function of time. Finally, ref. [19] indicates that the so-called multiplicative seasonal ARMA models of [41] cannot be generalized in the LSP framework for the same reason. On the contrary, there is no problem working with those models in our context. That remark made here for the tdVA${\mathrm{R}}^{\left(n\right)}$(1) model extends to the tdVMA(1) model and all the tdVARM${\mathrm{A}}^{\left(n\right)}$ models, as well.

It is possible to obtain the coefficients of the pure MA representations by simple recursive relations (see [54] for algorithms). Ref. [39] has even proposed an explicit solution, although it is limited to univariate processes. With these coefficients, everything can be computed, at least for finite n, and the assumptions can be checked. Of course, when we have a finite multivariate time series, it is not immediately apparent to suggest a tdVARMA model. What we propose in practice (see the examples treated in [43,53]) is first fitting a VARMA model, and then adding slopes for the linearly varying coefficients and, possibly, heteroscedasticity, before removing non-significant parameters one by one. At this time, there is no large-scale study on real data using time-dependent multivariate models. Still, the results for univariate models in [21] (with marginal heteroscedasticity only) and [56] seem promising.

We have specified in the introduction that our approach is not adequate for high-dimensional tdVARM${\mathrm{A}}^{\left(n\right)}$ models, i.e., when r is greater than a very small integer because the number of parameters grows too quickly. Here, we have the entries in the VAR and VMA polynomials but also the coefficients of polynomials if we extend the linear dependence in (10) or (20) to a polynomial dependence, for instance. In principle, the concept in [46] should be valid in our framework of time-dependent VARMA models, including the case of polynomial dependence, multiple breaks, or threshold models with multiple regimes, although bigtime in [47] should be strongly changed for that purpose. This is left for future research.

It should be noted that, although it is interesting to see that the assumptions of [43] can be verified, they are not particularly helpful in the face of a real multivariate time series. We note that, unfortunately, verifying the same assumptions using the data (one realization of length n) is impossible. The same criticism, however, holds for the entire literature on models with time-dependent coefficients.