# Autoregressive Models with Time-Dependent Coefficients—A Comparison between Several Approaches

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. The AM Theory for Time-Dependent Autoregressive Processes

**Definition**

**1.**

**Example**

**1.**

**Theorem**

**1.**

**Proof.**

**Remark**

**1.**

^{(n)}processes. Additionally, [16] provided a better foundation for the asymptotic theory for array processes, a theorem for a reduction of the order of moments from 8 to slightly more than 4 and tools for obtaining the asymptotic covariance matrix of the estimator. In [17], there was an example of vector tdAR and tdMA models on monthly log returns of IBM stock and the S&P500 index from January 1926 to December 1999, treated first in [18].

## 3. The Theory of Locally Stationary Processes

## 4. The Theory of Cyclically Time-Dependent Models

## 5. A Comparison with the Theory of Locally Stationary Processes

## 6. A Comparison with the Theory of Cyclically Time-Dependent Models

## 7. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A. Theorem A1 (Theorem 2’ of [1])

- there exists an estimator ${\widehat{\beta}}_{n},$ such that ${\widehat{\beta}}^{\left(n\right)}\to {\beta}^{0}$ in probability;
- ${n}^{\frac{1}{2}}\left({\widehat{\beta}}^{\left(n\right)}-{\beta}^{0}\right)\stackrel{L}{\to}N\left(0,{V}^{-1}W{V}^{-1}\right),$ where there exists a matrix $W$ whose elements are defined by (9).

## Appendix B. Alternative Assumptions under a Mixing Condition

**Definition**

**A1.**

**Lemma**

**A1.**

**Theorem**

**A1.**

**Proof.**

**Remark**

**A1.**

**Remark**

**A2.**

## Appendix C. tdAR(2) Monte Carlo Simulations

**Figure A1.**Variations of ${\varphi}_{t1}^{\left(n\right)}$ (horizontal) and ${\varphi}_{t2}^{\left(n\right)}$ (vertical) with respect to time t for n = 50; inside the triangle corresponds to the causality condition, and the curve separates complex roots (below) from real roots (above).

**Table A1.**Theoretical values of the parameters, averages and standard deviations of the estimates across simulations and averages across simulations of the estimated standard errors ${\varphi}_{1}^{\u2019}$, ${\varphi}_{1}^{\u2033}$, ${\varphi}_{2}^{\u2019}$ and ${\varphi}_{2}^{\u2033}$ for the tdAR(2) model described above for $n$ = 400 and 999 replications (out of 1000).

Parameter | Standard | Average of | |
---|---|---|---|

True Value | Average | Deviation | Standard Error |

${\varphi}_{1}^{\u2019}=0.0$ | 0.007306 | 0.050587 | 0.043869 |

${\varphi}_{1}^{\u2033}=0.002551$ | 0.002422 | 0.000322 | 0.000333 |

${\varphi}_{2}^{\u2019}=-0.2$ | $-$0.193960 | 0.048853 | 0.043537 |

${\varphi}_{2}^{\u2033}=0.003571$ | 0.003421 | 0.000332 | 0.000325 |

**Table A2.**Theoretical values of the parameters, averages and standard deviations of the estimates across simulations and averages across simulations of the estimated standard errors ${\varphi}_{1}^{\u2019}$, ${\varphi}_{1}^{\u2033}$, ${\varphi}_{2}^{\u2019}$ and ${\varphi}_{2}^{\u2033}$ for the tdAR(2) model described above for $n$ = 50 and 934 replications (out of 1000).

Parameter | Standard | Average of | |
---|---|---|---|

True Value | Average | Deviation | Standard Error |

${\varphi}_{1}^{\u2019}=0.0$ | 0.01419 | 0.14260 | 0.13620 |

${\varphi}_{1}^{\u2033}=0.020408$ | 0.01640 | 0.00900 | 0.00957 |

${\varphi}_{2}^{\u2019}=-0.2$ | $-$0.19510 | 0.13972 | 0.12436 |

${\varphi}_{2}^{\u2033}=0.028571$ | 0.02355 | 0.00852 | 0.00749 |

## References

- Azrak, R.; Mélard, G. Asymptotic properties of quasi-maximum likelihood estimators for ARMA models with time-dependent coefficients. Stat. Inference Stoch. Process.
**2006**, 9, 279–330. [Google Scholar] [CrossRef] - Alj, A.; Azrak, R.; Ley, C.; Mélard, G. Asymptotic properties of QML estimators for VARMA models with time-dependent coefficients. Scand. J. Stat.
**2017**, 44, 617–635. [Google Scholar] [CrossRef] - Wegman, E.J. Some results on non stationary first order autoregression. Technometrics
**1974**, 16, 321–322. [Google Scholar] [CrossRef] - Kwoun, G.H.; Yajima, Y. On an autoregressive model with time-dependent coefficients. Ann. Inst. Stat. Math.
**1986**, 38, 297–309. [Google Scholar] [CrossRef] - Tjøstheim, D. Estimation in Linear Time Series Models I: Stationary Series; Department of Mathematics: Bergen, Norway; Department of Statistics, University of Bergen: Bergen, Norway; University of North Carolina: Chapel Hill, NC, USA, 1984. [Google Scholar]
- Hamdoune, S. Étude des Problèmes d’estimation de Certains Modèles ARMA Évolutifs. Ph.D. Thesis, Université Henri Poincaré, Nancy, France, 1995. [Google Scholar]
- Dahlhaus, R. Fitting time series models to nonstationary processes. Ann. Stat.
**1997**, 25, 1–37. [Google Scholar] [CrossRef] - Bibi, A.; Francq, C. Consistent and asymptotically normal estimators for cyclically time-dependent linear models. Ann. Inst. Stat. Math.
**2003**, 55, 41–68. [Google Scholar] [CrossRef] - Dahlhaus, R. Locally Stationary Processes. In Handbook of Statistics; Rao, T.S., Rao, S.S., Rao, C.R., Eds.; Elsevier: Amsterdam, The Netherlands, 2012; Volume 30. [Google Scholar]
- Dahlhaus, R. Asymptotic statistical inference for nonstationary processes with evolutionary spectra. In Athens Conference on Applied Probability in Time Series Analysis; Robinson, P.M., Rosenblatt, M., Eds.; Springer: New York, NY, USA, 1996; Volume 2, pp. 145–159. [Google Scholar] [CrossRef]
- Mélard, G. The likelihood function of a time-dependent ARMA model. In Applied Time Series Analysis; Anderson, O.D., Perryman, M.R., Eds.; North-Holland: Amsterdam, The Netherlands, 1982; pp. 229–239. [Google Scholar]
- Azrak, R.; Mélard, G. The exact quasi-likelihood of time-dependent ARMA models. J. Stat. Plan. Inference
**1998**, 68, 31–45. [Google Scholar] [CrossRef] - Mélard, G.; Pasteels, J.-M. User’s Manual of Time Series Expert (TSE version 2.3); Institut de Statistique et Faculté des Sciences sociales, politiques et économiques, Université libre de Bruxelles: Bruxelles, Belgium, 1998; Available online: https://dipot.ulb.ac.be/dspace/retrieve/829842/TSE23E.PDF (accessed on 23 May 2022).
- Box, G.E.P.; Jenkins, G.M.; Reinsel, G.C.; Ljung, G.S. Time Series Analysis, Forecasting and Control, 5th ed.; Prentice-Hall: Hoboken, NJ, USA, 2015. [Google Scholar]
- Alj, A.; Azrak, R.; Mélard, G. General Estimation Results for tdVARMA Array Models; ECARES Working Paper 2022-25; Université Libre de Bruxelles: Bruxelles, Belgium, 2022. [Google Scholar]
- Azrak, R.; Mélard, G. Asymptotic properties of conditional least-squares estimators for array time series. Stat. Inference Stoch. Process.
**2021**, 24, 525–547. [Google Scholar] [CrossRef] - Alj, A.; Jónasson, K.; Mélard, G. The exact Gaussian likelihood estimation of time-dependent VARMA models. Comput. Stat. Data Anal.
**2016**, 100, 633–644. [Google Scholar] [CrossRef] - Tsay, R.S. Analysis of Financial Time Series, 2nd ed.; John Wiley: New York, NY, USA, 2005. [Google Scholar] [CrossRef]
- Dahlhaus, R. Maximum likelihood estimation and model selection for locally stationary processes. J. Nonparametric Stat.
**1996**, 6, 171–191. [Google Scholar] [CrossRef] - Barigozzi, M.; Hallin, M.; Soccorsi, S.; von Sachs, R. Time-varying general dynamic factor models and the measurement of financial connectedness. J. Econ.
**2020**, 222, 324–343. [Google Scholar] [CrossRef] - Hu, L.; Huang, T.; You, J. Estimation and identification of a varying-coefficient additive model for locally stationary processes. J. Am. Stat. Assoc.
**2019**, 114, 1191–1204. [Google Scholar] [CrossRef] - Hu, L.; Huang, T.; You, J. Two-step estimation of time-varying additive model for locally stationary time series. Comput. Stat. Data Anal.
**2019**, 130, 94–110. [Google Scholar] [CrossRef] - von Sachs, R. Nonparametric Spectral Analysis of Multivariate Time Series. Annu. Rev. Stat. Its Appl.
**2020**, 7, 361–386. [Google Scholar] [CrossRef] - Kreiss, J.P.; Paparoditis, E. Bootstrapping locally stationary processes. J. R. Stat. Soc. Ser. B
**2015**, 77, 267–290. [Google Scholar] [CrossRef] - Niedźwiecki, M.; Ciołek, M. Identification of nonstationary multivariate autoregressive processes–Comparison of competitive and collaborative strategies for joint selection of estimation. Digit. Signal Process.
**2018**, 78, 72–81. [Google Scholar] [CrossRef] - Zhou, Z. Inference for non-stationary time-series autoregression. J. Time Ser. Anal.
**2013**, 34, 508–516. [Google Scholar] [CrossRef] - Roueff, F.; Sanchez-Perez, A. Prediction of weakly locally stationary processes by auto-regression. ALEA Lat. Am. J. Probab. Math. Stat.
**2018**, 15, 1215–1239. [Google Scholar] [CrossRef] - Das, S.; Politis, D.N. Predictive inference for locally stationary time series with an application to climate data. J. Am. Stat. Assoc.
**2021**, 116, 919–934. [Google Scholar] [CrossRef] - Paparoditis, E.; Preuß, P. On local power properties of frequency domain-based tests for stationarity. Scand. J. Stat.
**2016**, 43, 664–682. [Google Scholar] [CrossRef] - Puchstein, R.; Preuß, P. Testing for stationarity in multivariate locally stationary processes. J. Time Ser. Anal.
**2016**, 37, 3–29. [Google Scholar] [CrossRef] - Richter, S.; Dahlhaus, R. Cross validation for locally stationary processes. Ann. Stat.
**2019**, 47, 2145–2173. [Google Scholar] [CrossRef] - Niedźwiecki, M.; Ciołek, M.; Kajikawa, Y. On adaptive covariance and spectrum estimation of locally stationary multivariate processes. Automatica
**2017**, 82, 1–12. [Google Scholar] [CrossRef] - Kapetanios, G.; Marcellino, M.; Venditti, F. Large time-varying parameter VARs: A nonparametric approach. J. Appl. Econom.
**2019**, 34, 1027–1049. [Google Scholar] [CrossRef] - Cardinali, A.; Nason, G.P. Costationarity of locally stationary time series using costat. J. Stat. Softw.
**2013**, 55, 1–22. [Google Scholar] [CrossRef] - Dahlhaus, R.; Richter, S.; Wu, W.B. Towards a general theory for nonlinear locally stationary processes. Bernoulli
**2019**, 25, 1013–1044. [Google Scholar] [CrossRef] - Birr, S.; Volgushev, S.; Kley, T.; Dette, H.; Hallin, M. Quantile spectral analysis for locally stationary time series. J. R. Stat. Soc. Ser. B
**2017**, 79, 1619–1643. [Google Scholar] [CrossRef] - Gorrostieta, C.; Ombao, H.; von Sachs, R. Time-dependent dual-frequency coherence in multivariate non-stationary time series. J. Time Ser. Anal.
**2019**, 40, 3–22. [Google Scholar] [CrossRef] - Bardet, J.-M.; Doukhan, P. Non-parametric estimation of time varying AR(1) processes with local stationarity and periodicity. Electron. J. Stat.
**2018**, 12, 2323–2354. [Google Scholar] [CrossRef] - Francq, C.; Gautier, A. Estimation de modèles ARMA à changements de régime récurrents. C. R. L’académie Sci. Paris Ser. I
**2004**, 339, 55–58. [Google Scholar] [CrossRef] - Francq, C.; Gautier, A. Large sample properties of parameter least squares estimates for time-varying ARMA models. J. Time Ser. Anal.
**2004**, 25, 765–783. [Google Scholar] [CrossRef] - Francq, C.; Gautier, A. Estimation of time-varying ARMA models with Markovian changes in regime. Stat. Probab. Lett.
**2004**, 70, 243–251. [Google Scholar] [CrossRef] - Gautier, A. Influence asymptotique de la correction par la moyenne sur l’estimation d’un modèle AR périodique. C. R. L’académie Sci. Paris Ser. I
**2005**, 340, 315–318. [Google Scholar] [CrossRef] - Bibi, A.; Oyet, A.J. Estimation of some bilinear time series models with time varying coefficients. Stoch. Anal. Appl.
**2004**, 22, 355–376. [Google Scholar] [CrossRef] - Bibi, A.; Ghezal, A. On periodic time-varying bilinear processes: Structure and asymptotic inference. Stat. Methods Appl.
**2016**, 25, 395–420. [Google Scholar] [CrossRef] - Bibi, A.; Ghezal, A. QMLE of periodic bilinear models and of PARMA models with periodic bilinear innovations. Kybernetika
**2018**, 54, 375–399. [Google Scholar] [CrossRef] - Regnard, N.; Zakoïan, J.M. Structure and estimation of a class of nonstationary yet nonexplosive GARCH models. J. Time Ser. Anal.
**2010**, 31, 348–364. [Google Scholar] [CrossRef] - Regnard, N.; Zakoian, J.M. A conditionally heteroskedastic model with time-varying coefficients for daily gas spot prices. Energy Econ.
**2011**, 33, 1240–1251. [Google Scholar] [CrossRef] - Boubacar Maïnassara, Y.; Rabehasaina, L. Estimation of weak ARMA models with regime changes. Stat. Inference Stoch. Process.
**2020**, 23, 1–52. [Google Scholar] [CrossRef] - Mélard, G. On an alternative model for intervention analysis. In Time Series Analysis; Anderson, O.D., Perryman, M.R., Eds.; North-Holland: Amsterdam, The Netherlands, 1981; pp. 345–354. [Google Scholar]
- Bosq, D. Nonparametric Statistics for Stochastic Processes: Estimation and Prediction, 2nd ed.; Springer: New York, NY, USA, 1998. [Google Scholar] [CrossRef]
- Kolmogorov, A.N.; Rozanov, Y.A. On strong mixing conditions for stationary Gaussian processes. Theory Probab. Appl.
**1960**, 5, 204–208. [Google Scholar] [CrossRef] - Rio, E. Théorie Asymptotique des Processus Aléatoires Faiblement Dépendants; Springer: Berlin, Germany, 2000. [Google Scholar]
- Andrews, D.W.K. Laws of large numbers for dependent non-identically distributed random variables. Econom. Theory
**1988**, 4, 458–467. [Google Scholar] [CrossRef] - Hall, P.; Heyde, C.C. Martingale Limit Theory and Its Application; Academic Press: New York, NY, USA, 1980. [Google Scholar]
- Alj, A.; Azrak, R.; Mélard, G. On conditions in central limit theorems for martingale difference arrays. Econ. Lett.
**2014**, 123, 305–307. [Google Scholar] [CrossRef] - Chanda, K.C. Strong mixing properties of linear stochastic processes. J. Appl. Probab.
**1974**, 11, 401–408. [Google Scholar] [CrossRef] - Pham, D.; Tran, T. Some mixing properties of time series models. Stoch. Process. Appl.
**1985**, 19, 297–303. [Google Scholar] [CrossRef]

**Figure 1.**Schematic presentation on how to interpret asymptotics in AM and LSP theories (see the text for details).

**Figure 2.**Artificial series produced using the process defined by (1) and (13) (see the text for details).

**Figure 3.**True values of ${\varphi}_{t}^{\left(n\right)}$ (that go above 1!) (solid line) and their fit (discontinuous line).

**Figure 5.**${w}_{t}^{\left(128\right)}$ as a function of ${w}_{t-1}^{\left(128\right)}$ (crosses: $t\le 64$, stars: t > 64).

**Figure 6.**${w}_{t}^{\left(128\right)}$ as a function of ${w}_{t-1}^{\left(128\right)}$ (plusses: high scatter, when g

_{t}= 2; circles: small scatter, when g

_{t}= 0.5).

**Table 1.**Estimates of a (homoscedastic) tdAR(1) model defined by (1) for p = 1 with a constant (denoted MEAN 1). The parameters ${\varphi}_{1}^{\u2019}$ and ${\varphi}_{1}^{\u2033}$ are, respectively, denoted as AR 1 and TDAR 1.

Final Values of the Parameters | With 95% Confidence Limits | |||||
---|---|---|---|---|---|---|

Name | Value | Std Error | t-Value | Lower | Upper | |

1 | MEAN 1 | 9.3223 | 0.10676 | 87.3 | 9.1 | 9.5 |

2 | AR 1 | 0.84640 | 2.97198 × 10^{−2} | 28.5 | 0.79 | 0.90 |

3 | TDAR 1 | −7.25476 × 10^{−4} | 2.71992 × 10^{−4} | −2.7 | −1.26 × 10^{−3} | −1.92 × 10^{−4} |

**Table 2.**Theoretical values of the parameters, averages and standard deviations of the estimates across simulations and medians across simulations of the estimated standard errors ${\varphi}^{\prime}$ (true value: 0.15), ${\varphi}^{\u2033}$ (true value: 0.015), ${\varphi}^{\u2034}$ (true value: 0) and $g$ (true value 0.5) for the tdAR(1) model described above for $n$ = 128 and 964 replications (out of 1000).

Parameter | Standard | Median of | |
---|---|---|---|

True Value | Average | Deviation | Standard Error |

${\varphi}^{\prime}=0.15$ | 0.23554 | 0.14611 | 0.10380 |

${\varphi}^{\u2033}=0.015$ | 0.01282 | 0.00222 | 0.00160 |

${\varphi}^{\u2034}=0.0$ | $-$0.00000 | 0.00005 | 0.00005 |

$g=0.5$ | 0.54054 | 0.07857 | 0.08157 |

**Table 3.**Theoretical values of the parameters, averages and standard deviations of estimates across simulations, and averages across simulations of estimated standard errors ${\varphi}^{\prime}$ (true value: 0.15), ${\varphi}^{\u2033}$ (true value: 0.015) and $g$ (true value 0.5) for the tdAR(1) model described above for $n$ = 128 and 999 replications (out of 1000).

Parameter | Standard | Average of | |
---|---|---|---|

True Value | Average | Deviation | Standard Error |

${\varphi}^{\prime}=0.15$ | 0.22023 | 0.12577 | 0.06683 |

${\varphi}^{\u2033}=0.015$ | 0.01305 | 0.00202 | 0.00146 |

$g=0.5$ | 0.54290 | 0.07847 | 0.06984 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Azrak, R.; Mélard, G.
Autoregressive Models with Time-Dependent Coefficients—A Comparison between Several Approaches. *Stats* **2022**, *5*, 784-804.
https://doi.org/10.3390/stats5030046

**AMA Style**

Azrak R, Mélard G.
Autoregressive Models with Time-Dependent Coefficients—A Comparison between Several Approaches. *Stats*. 2022; 5(3):784-804.
https://doi.org/10.3390/stats5030046

**Chicago/Turabian Style**

Azrak, Rajae, and Guy Mélard.
2022. "Autoregressive Models with Time-Dependent Coefficients—A Comparison between Several Approaches" *Stats* 5, no. 3: 784-804.
https://doi.org/10.3390/stats5030046