# Nonlinear Time Series Modeling: A Unified Perspective, Algorithm and Application

^{1}

^{2}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

- (1)
- Marginal modeling: The identification of the marginal probability law (in particular, the heavy-tailed marginal densities) of a time series plays a vital role in financial econometrics. Notations: common quantile Q, inverse of the distribution function F, respectively denoted $Q(u;Y),\text{}0u1$ and $F(y;Y)$. The mid-distribution is defined as ${F}^{\mathrm{mid}}(y;Y)=F(y;Y)-0.5\mathrm{Pr}(Y\left(t\right)=y)$.
- (2)
- Correlation modeling: Covariance function (defined for positive and negative lag h) $R(h;Y)=\mathrm{Cov}\left[Y\right(t),Y(t+h\left)\right]$. $R(0;Y)=\mathrm{Var}\left[Y\right(t\left)\right]$, $\mu =\mathbb{E}\left[Y\right(t\left)\right]$ assumed zero in our prediction theory. Correlation function $\rho \left(h\right)=\mathrm{Cor}\left[Y\right(t),Y(t+h\left)\right]=R(h;Y)/R(0;Y)$.
- (3)
- Frequency-domain modeling: When covariance is absolutely summable, define spectral density function $f(\omega ;Y)=\sum R(h;Y)\phantom{\rule{0.166667em}{0ex}}{e}^{-2\pi i\omega h},\text{}-1/2\omega 1/2$.
- (4)
- Time-domain modeling: The time domain model is a linear filter relating $Y\left(t\right)$ to white noise $\u03f5\left(t\right)$, $\mathcal{N}(0,1)$ independent random variables. Autoregressive scheme of order m, a predominant linear time series technique for modeling conditional mean, is defined as (assuming $\mathbb{E}\left[Y\right(t\left)\right]=0$):$$Y\left(t\right)-a(1;m)Y(t-1)-\dots -a(m;m)Y(t-m)\phantom{\rule{0.166667em}{0ex}}=\phantom{\rule{0.166667em}{0ex}}{\sigma}_{m}\u03f5\left(t\right),$$$$f(\omega ;Y)\phantom{\rule{3.33333pt}{0ex}}=\phantom{\rule{3.33333pt}{0ex}}{\displaystyle \frac{{\sigma}_{m}^{2}}{|1-{\sum}_{k=1}^{m}a(k;m){e}^{2\pi i\omega k}{|}^{2}}}.$$$${Y}^{\mu ,m}\left[t\right]\phantom{\rule{0.166667em}{0ex}}=\phantom{\rule{0.166667em}{0ex}}\mathbb{E}\left[Y\left(t\right)\mid Y(t-1),\dots ,Y(t-m)\right]\phantom{\rule{0.166667em}{0ex}}=\phantom{\rule{0.166667em}{0ex}}a(1;m)Y(t-1)+\cdots +a(m;m)Y(t-m).$$$$\mathrm{AIC}\left(m\right)\phantom{\rule{0.166667em}{0ex}}=\phantom{\rule{0.166667em}{0ex}}2\mathrm{log}{\sigma}_{m}+2m/n.$$

## 2. From Linear to Nonlinear Modeling

## 3. Nonparametric LPTime Analysis

- (a)
- The algorithmic modeling aspect (how it works).
- (b)
- The required theoretical ideas and notions (why it works).
- (c)
- The application to daily S&P 500 return data between 2 January 1963 and 31 December 2009 (empirical proof-of-work).

#### 3.1. The Data and LP-Transformation

#### 3.2. Marginal Modeling

#### Non-Normality Diagnosis

#### 3.3. Copula Dependence Modeling

#### 3.3.1. Nonparametric Serial Copula

#### 3.3.2. LP-Comoment of Lag h

#### 3.3.3. LP-Correlogram, Evidence and Source of Nonlinearity

`acf()`R function on $\mathrm{Vec}\left(\mathrm{YS}\right)\left(t\right)$ generates the graphical display of our proposed LP-correlogram plot.

#### 3.3.4. AutoLPinfor: Nonlinear Correlation Measure

#### 3.3.5. Nonparametric Estimation of Blomqvist’s Beta

#### 3.3.6. Nonstationarity Diagnosis, LP-Comoment Approach

#### 3.4. Local Dependence Modeling

#### 3.4.1. Quantile Correlation Plot and Test for Asymmetry

`QCF`) as the following in terms of the copula distribution function of $\left(Y\right(t),Y(t+h\left)\right)$ denoted by $\mathrm{Cop}(u,v;Y(t),Y(t+h\left)\right):=\mathrm{Cop}(u,v;h)$,

`QCF`under the independence assumption. Deviation from this line helps us to better understand the nature of asymmetry. We compute ${\widehat{\lambda}}_{\mathrm{G}}[u;Y\left(t\right),Y(t+h)]$ using the fitted Gaussian copula:

#### 3.4.2. Conditional LPinfor Dependence Measure

#### 3.5. Non-Crossing Conditional Quantile Modeling

#### 3.6. Nonlinear Spectrum Analysis

#### 3.7. Nonparametric Model Specification

`LPVAR`) is given by:

## 4. Conclusions

- From the theoretical standpoint, the unique aspect of our proposal lies in its ability to simultaneously embrace and employ the spectral domain, time domain, quantile domain and information domain analyses for enhanced insights, which to the best of our knowledge has not appeared in the nonlinear time series literature before.
- From a practical angle, the novelty of our technique is that it permits us to use the techniques from linear Gaussian time series to create non-Gaussian nonlinear time series models with highly interpretable parameters. This aspect makes
`LPTime`computationally extremely attractive for data scientists, as they can now borrow all the standard time series analysis machinery from`R`libraries for implementation purposes. - From the pedagogical side, we believe that these concepts and methods can easily be augmented with the standard time series analysis course to modernize the current curriculum so that students can handle complex time series modeling problems (McNeil et al. 2010) using the tools with which they are already familiar.

`LPTime`technology to create a realistic general-purpose algorithm for empirical time series modeling. In addition, many new theoretical results and diagnostic measures were presented, which laid the foundation for the algorithmic implementation of

`LPTime`. We showed how LPTime can systematically explore the data to discover empirical facts hidden in time series. For example, LPTime empirical modeling of S&P 500 return data reproduces the ‘stylized facts’—(a) heavy tails; (b) non-Gaussian; (c) nonlinear serial dependence; (d) tail correlation; (e) asymmetric dependence; (f) volatility clustering; (g) long-memory volatility structure; (h) efficient market hypothesis; (i) leverage effect; (j) excess kurtosis—in a coherent manner under a single general unified framework. We have emphasized how the statistical parameters of our model can be interpreted in light of established economic theory.

`R`package

`LPTime`(Mukhopadhyay and Nandi 2015), which is available on

`CRAN`.

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Adrian, Tobias, and Markus K Brunnermeier. 2011. Covar. Technical Report. Cambridge: National Bureau of Economic Research. [Google Scholar]
- Brillinger, David R. 1977. The identification of a particular nonlinear time series system. Biometrika 64: 509–15. [Google Scholar] [CrossRef]
- Brillinger, David R. 2004. Some data analyses using mutual information. Brazilian Journal of Probability and Statistics 18: 163–83. [Google Scholar]
- Engle, Robert F. 1982. Autoregressive conditional heteroscedasticity with estimates of the variance of united kingdom inflation. Econometrica: Journal of the Econometric Society 50: 987–1007. [Google Scholar] [CrossRef]
- Engle, Robert F., and Simone Manganelli. 2004. Caviar: Conditional autoregressive value at risk by regression quantiles. Journal of Business & Economic Statistics 22: 367–81. [Google Scholar]
- Granger, Clive, and Jin-Lung Lin. 1994. Using the mutual information coefficient to identify lags in nonlinear models. Journal of Time Series Analysis 15: 371–84. [Google Scholar] [CrossRef]
- Granger, Clive W. J. 1993. Strategies for modelling nonlinear time-series relationships. Economic Record 69: 233–38. [Google Scholar] [CrossRef]
- Granger, Clive W. J. 1998. Overview of nonlinear time series specification in economics. Paper presented at the NSF Symposium on Nonlinear Time Series Models, University of California, Berkeley, CA, USA, 22 May 1998. [Google Scholar]
- Granger, Clive W. J. 2003. Time series concepts for conditional distributions. Oxford Bulletin of Economics and Statistics 65: 689–701. [Google Scholar] [CrossRef]
- Granger, Clive W. J., and Roselyne Joyeux. 1980. An introduction to long-memory time series models and fractional differencing. Journal of Time Series Analysis 1: 15–29. [Google Scholar] [CrossRef]
- Guo, Xin, Howard Shek, Tze Leung Lai, and Samuel Po-Shing Wong. 2017. Quantitative Trading: Algorithms, Analytics, Data, Models, Optimization. New York: Chapman and Hall/CRC. [Google Scholar]
- Hendry, David F. 2011. Empirical economic model discovery and theory evaluation. Rationality, Markets and Morals 2: 115–45. [Google Scholar]
- McNeil, Alexander J., Rüdiger Frey, and Paul Embrechts. 2010. Quantitative Risk Management: Concepts, Techniques, and Tools. Princeton: Princeton University Press. [Google Scholar]
- Mukhopadhyay, Subhadeep. 2016. Large scale signal detection: A unifying view. Biometrics 72: 325–34. [Google Scholar] [CrossRef] [PubMed]
- Mukhopadhyay, Subhadeep. 2017. Large-scale mode identification and data-driven sciences. Electronic Journal of Statistics 11: 215–40. [Google Scholar] [CrossRef]
- Mukhopadhyay, Subhadeep, and Douglas Fletcher. 2018. Generalized Empirical Bayes Modeling via Frequentist Goodness-of-Fit. Nature Scientific Reports 8: 1–15. [Google Scholar] [CrossRef] [PubMed]
- Mukhopadhyay, Subhadeep, and Shinjini Nandi. 2015. LPTime: LP Nonparametric Approach to Non-Gaussian Non-Linear Time Series Modelling. CRAN, R Package Version 1.0-2. Ithaca: Cornell University Library. [Google Scholar]
- Mukhopadhyay, Subhadeep, and Shinjini Nandi. 2017. LPiTrack: Eye movement pattern recognition algorithm and application to biometric identification. Machine Learning. [Google Scholar] [CrossRef]
- Mukhopadhyay, Subhadeep, and Emanuel Parzen. 2014. LP approach to statistical modeling. arXiv, arXiv:1405.2601. [Google Scholar]
- Parzen, Emanuel. 1967. On empirical multiple time series analysis. In Statistics, Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. Berkeley: University of California Press, vol. 1, pp. 305–40. [Google Scholar]
- Parzen, Emanuel. 1979. Nonparametric statistical data modeling (with discussion). Journal of the American Statistical Association 74: 105–31. [Google Scholar] [CrossRef]
- Parzen, Emanuel. 1992. Time series, statistics, and information. In New Directions in Time Series Analysis. Edited by Emanuel Parzen, Murad Taqqu, David R. Brillinger, Peter Caines, John Geweke and Murray Rosenblatt. New York: Springer Verlag, pp. 265–86. [Google Scholar]
- Parzen, Emanuel. 1997. Comparison distributions and quantile limit theorems. Paper presented at the International Conference on Asymptotic Methods in Probability and Statistics, Carleton University, Ottawa, ON, Canada, July 8–13. [Google Scholar]
- Parzen, Emanuel, and Subhadeep Mukhopadhyay. 2012. Modeling, Dependence, Classification, United Statistical Science, Many Cultures. arXiv, arXiv:1204.4699. [Google Scholar]
- Parzen, Emanuel, and Subhadeep Mukhopadhyay. 2013a. United Statistical Algorithms, LP comoment, Copula Density, Nonparametric Modeling. Paper presented at the 59th ISI World Statistics Congress (WSC) of the International Statistical Institute, Hong Kong, China, August 25–30. [Google Scholar]
- Parzen, Emanuel, and Subhadeep Mukhopadhyay. 2013b. United Statistical Algorithms, Small and Big Data, Future of Statisticians. arXiv, arXiv:1308.0641. [Google Scholar]
- Salmon, Felix. 2012. The formula that killed wall street. Significance 9: 16–20. [Google Scholar] [CrossRef]
- Terasvirta, Timo, Dag Tjøstheim, and Clive W. J. Granger. 2010. Modelling Nonlinear Economic Time Series. Kettering: OUP Catalogue. [Google Scholar]
- Tsay, Ruey S. 2010. Analysis of Financial Time Series. Hoboken: Wiley. [Google Scholar]
- Tukey, John. 1980. Can we predict where “time series” should go next. In Directions in Times Series. Hayward: IMS, pp. 1–31. [Google Scholar]
- Woodward, Wayne A., Henry L. Gray, and Alan C. Elliott. 2011. Applied Time Series Analysis. Boca Raton: CRC Press. [Google Scholar]

1 | The LP nomenclature: In nonparametric statistics, the letter L plays a special role to denote robust methods based on ranks and order statistics such as quantile-domain methods. With the same motivation, we use the letter L. On the other hand, P simply stands for Polynomials. Our custom-constructed basis functions are orthonormal polynomials of mid-rank transform instead of raw y-values; for more details see Mukhopadhyay and Parzen (2014). |

**Figure 1.**LP-transformed S&P 500 daily stock returns between October 1986 and October 1988. This is just a small part of the full time series from 2 January 1963–31 December 2009 (cf. Section 3.1).

**Figure 2.**(

**a**) The marginal distribution of daily returns; (

**b**) plots the histogram of $\Phi \left({y}_{i}\right)$ and display the LP-estimated comparison density curve. and (

**c**) shows the associated comparison density estimate with G as t-distribution with 2 degrees of freedom.

**Figure 3.**

**Top**: Nonparametric smooth serial copula density (lag one) estimate of S&P return data.

**Bottom**: BIC plot to select the significant LP-comoments computed in Equation (11).

**Figure 4.**LP-correlogram: Sample autocorrelations of LP-transformed time series. The decay rate of the sample autocorrelations of ${\mathrm{YS}}_{2}\left(t\right)$ appears to be much slower than the exponential decay of the ARMA process, implying possible long-memory behavior.

**Figure 7.**Estimated Quantile Correlation Function (

`QCF`) ${\widehat{\lambda}}_{\mathrm{LP}}[u;Y\left(t\right),Y(t+1)]$. It detects asymmetry in the tail dependence between the lower-left quadrant and upper-right quadrant for S&P 500 return data. The red dotted line denotes the quantile correlation function under dependence. The dark green line shows the quantile correlation curve for the fitted Gaussian copula.

**Figure 8.**(

**a**) The conditional LPinfor curve is shown for the pair $\left[Y\right(t),Y(t+1\left)\right]$. The asymmetric dependence in the tails is clearly shown, and almost nothing is going on in between. (

**b**,

**c**) Display of how the mean and volatility levels of conditional distribution $f[y;Y(t+1\left)\right|Y\left(t\right)=Q(u;Y(t\left)\right)]$ change with respect to the unconditional marginal distribution $f(y;Y(t\left)\right)$ at different quantiles.

**Figure 9.**Each row displays the estimated conditional comparison density and the corresponding conditional distribution for $u=0.01,0.5,0.99$.

**Figure 10.**The figure shows estimated non-parametric conditional quantile curves for S&p 500 return data. The red solid line, which represents $\widehat{Q}(0.001;Y(t+1)|Y\left(t\right))$, is popularly known as the one-day 0.1% Conditional Value at Risk measure (CoVaR).

**Figure 11.**LPSpectrum: AR spectral density estimate for S&P 500 return data. Order selected by the BIC method. This provides a diagnostic tool for providing evidence of hidden periodicities in non-Gaussian nonlinear time series.

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Mukhopadhyay, S.; Parzen, E.
Nonlinear Time Series Modeling: A Unified Perspective, Algorithm and Application. *J. Risk Financial Manag.* **2018**, *11*, 37.
https://doi.org/10.3390/jrfm11030037

**AMA Style**

Mukhopadhyay S, Parzen E.
Nonlinear Time Series Modeling: A Unified Perspective, Algorithm and Application. *Journal of Risk and Financial Management*. 2018; 11(3):37.
https://doi.org/10.3390/jrfm11030037

**Chicago/Turabian Style**

Mukhopadhyay, Subhadeep, and Emanuel Parzen.
2018. "Nonlinear Time Series Modeling: A Unified Perspective, Algorithm and Application" *Journal of Risk and Financial Management* 11, no. 3: 37.
https://doi.org/10.3390/jrfm11030037