An Experiment on Autoregressive and Threshold Autoregressive Models with Non-Gaussian Error with Application to Realized Volatility

Abstract

This article explores the fitting of Autoregressive (AR) and Threshold AR (TAR) models with a non-Gaussian error structure. This is motivated by the problem of finding a possible probabilistic model for the realized volatility. A Gamma random error is proposed to cater for the non-negativity of the realized volatility. With many good properties, such as consistency even for non-Gaussian errors, the maximum likelihood estimate is applied. Furthermore, a non-gradient numerical Nelder–Mead method for optimization and a penalty method, introduced for the non-negative constraint imposed by the Gamma distribution, are used. In the simulation experiments, the proposed fitting method found the true model with a rather insignificant bias and mean square error (MSE), given the true AR or TAR model. The AR and TAR models with Gamma random error are then tested on empirical realized volatility data of 30 stocks, where one third of the cases are fitted quite well, suggesting that the model may have potential as a supplement for current Gaussian random error models with proper adaptation.

1. Introduction

As the financial market and investment instruments grow more sophisticated, the need for the proper risk management of financial activities and the modeling of financial volatility has become more crucial. As there is no unique and unambiguous definition for volatility, observable quantities (such as daily high-lows or intra-day price changes) are used to approximate the quantity, thus dividing volatility modeling techniques into two sub-groups: Parametric and non-parametric (Anderson et al. 2002; Zheng et al. 2014). The first group are the traditional parametric latent volatility models, such as the Generalized Autoregressive Conditional Heteroscedastic (GARCH) model or the Stochastic Variance (SV) model. However, these parametric models have become increasingly restrictive in use, due to growing complexity. As mentioned in McAleer and Medeiros (2008), as the traditional standard latent volatility models cannot adequately describe the slowly decreasing auto-correlations of squared returns and as the usage of Gaussian standardized error has been criticized by many, the Realized Volatility (RV) model, as an alternative non-parametric method, has received increasing attention. In its simplest form, the RV model can be simply defined as

$$R{V}_t=\sum _{i=0}^{n_t}{r}_{t,i}^2,$$

(1)

where $R{V}_t$ denotes the realized volatility at day t, $r_{t,i}$ denotes the ith intra-period return at day t, and $n_t$ is the number of high frequency data observed. It has been shown that the RV model more accurately measures the ’true volatility’ than daily squared returns (Anderson et al. 1999; Kambouroudis et al. 2016) and it is among the best for modeling the volatilities of the U.S. and E.U. stock indices (Kambouroudis et al. 2016). It is also a good measure for market risk, due to its ability to show clustering and fat-tail behavior for price fluctuations (Zheng et al. 2014).

A lot of work has been done towards constructing the realized volatility; see McAleer and Medeiros (2008) for a review. The focus of this article is to consider possible probabilistic models, given $R{V}_t$; particularly if it is possible to model it with a non-Gaussian random error structure. As models based on the Wishart distribution have been proposed for multi-variate realized volatility (Golosnoy et al. 2012) and multi-variate stochastic volatility (Gouriéroux et al. 2009), and as the Wishart distribution is the multi-variate analog of the chi-square distribution (which is a member of the Gamma distribution family), a Gamma random error structure in the univariate case has become of interest. Thus, traditional Autoregressive (AR) and Threshold-type non-linear AR (TAR) models with Gamma random error are explored. This article can be regarded as an extension of Li and McLeod (1988).

2. Materials and Methods

This section aims to provide the specification for the proposed model and the fitting methodology. It will also briefly touch on the materials and methods for conducting the empirical data analysis.

2.1. Model Specification

Time-series models with non-Gaussian error were previously considered, in some detail, by Li and McLeod (1988), and earlier in Lawrance and Lewis (1980) and Ledolter (1979). In this article, specifically, the AR and TAR models are further explored. The AR(p) model is defined as follows:

$$R{V}_t=\sum _{i=1}^p{\phi }_i\ast R{V}_{t-i}+{\epsilon }_t,$$

(2)

where ${\epsilon }_t$ is the random error, assumed to follow a Gamma distribution; thus, ${\epsilon }_t\sim \Gamma (\alpha ,\beta )$, where the density function is defined as

$$f(x)=\frac{1}{\Gamma (\alpha )\ast {\beta }^{\alpha }}\ast x^{\alpha -1}\ast e^{-\frac{x}{\beta }}.$$

(3)

It should be noted that it is assumed that there is no drift term in the AR model; yet, the drift term could be easily incorporated into the model. The TAR(p) model, similar to that introduced in Tong (1978) but with a modification in the random error term, is defined as follows:

$$\begin{array}{c}\hfill R{V}_t=\sum _{i=1}^p{\phi }_{1,i}\ast R{V}_{t-i}+{\epsilon }_{1,t}ifR{V}_{t-d}\le T,\\ \hfill R{V}_t=\sum _{i=1}^p{\phi }_{2,i}\ast R{V}_{t-i}+{\epsilon }_{2,t}ifR{V}_{t-d}>T,\end{array}$$

(4)

where d ≥ 1 is the lag of the model and T is the threshold, such that the model is divided into two regimes, according to the observations at d time periods earlier. The pivot element $R{V}_{t-d}$ determines which regime $R{V}_t$ falls into, with $R{V}_t$ falling into the first regime if $R{V}_{t-d}$ is less than or equal to the threshold and into the second regime, otherwise. Each regime follows an AR(p) model, as defined above, with different AR and Gamma parameters.

2.2. Model Estimation

The fitting of the AR(p) model is introduced in this part, followed by the extension of the procedure to the fitting of the TAR(p) model. Both procedures are fitted with the maximum likelihood procedure, as it has been shown that the maximum likelihood estimators (MLE) are consistent for Gamma random error (Li and McLeod 1988).

The MLE for AR(p) model are derived by $l(\widehat{\alpha },\widehat{\beta },\widehat{\phi })=argmin(-l)$, where l denotes the log-likelihood function, in the form of

$$l(\alpha ,\beta ,\phi )=-n\ast ln(\Gamma (\alpha ))-n\ast \alpha \ast ln(\beta )+(\alpha -1)\ast \sum _{t=1}^{n}ln({\epsilon }_t)-\frac{{\sum }_{t=1}^n{\epsilon }_t}{\beta },$$

(5)

where

$${\epsilon }_t=R{V}_t-\sum _{i=1}^p{\phi }_i\ast R{V}_{t-i}$$

(6)

is the random error.

To further reduce the dimension of estimation, a profile likelihood method is used. The Gamma parameters $\alpha$ and $\beta$ are replaced by the MLE of $\alpha$ and $\beta$, using the result of Wilk et al. (1962) and the approximation $\frac{dln(\gamma (\alpha ))}{d\alpha }\approx ln(\alpha -\frac{1}{2})$. Thus, the final estimates of $\alpha$ and $\beta$ are as follows:

$$\widehat{\alpha }=\frac{A}{2\ast (A-G)}and\widehat{\beta }=\frac{A}{\widehat{\alpha }},$$

(7)

where A stands for the arithmetic mean of the random error and G is the geometric mean. Thus, the estimation of the model is achieved by estimating $\widehat{\phi }=argmin(-\widehat{l})$, where

$$\widehat{l}(\phi )=-n\ast ln(\Gamma (\widehat{\alpha }))-n\ast \widehat{\alpha }\ast ln(\widehat{\beta })+(\widehat{\alpha }-1)\ast \sum _{t=1}^{n}ln({\epsilon }_t)-\frac{{\sum }_{t=1}^n{\epsilon }_t}{\widehat{\beta }},$$

(8)

where $\widehat{\alpha }$ and $\widehat{\beta }$ are estimated by Equation (7) and, by Equation (6), thus depend on $\phi$.

The Nelder–Mead method, which is a non-gradient optimization method, is proposed to optimize the negative log-likelihood function. Although such a procedure is heuristic and may converge to non-stationary points, its performance is much more stable than traditional gradient methods, such as the Hessian matrix method, which may not be easily calculated (even numerically) given the dependency of the log-likelihood function and as $\phi$ is quite complicated.

Additionally, before simply applying the method and carrying out the optimization, it should be noticed that, as the random error ${\epsilon }_t$ is assumed to be Gamma, it is required to be greater than zero, which is also evident from the term $ln({\epsilon }_t)$ in the expression of the log-likelihood function. To reflect this non-negativity constraint, a penalty method is applied and the log-likelihood function becomes:

$$\widehat{l}=(-n\ast ln(\Gamma (\widehat{\alpha }))-n\ast \widehat{\alpha }\ast ln(\widehat{\beta })+(\widehat{\alpha }-1)\ast \sum _{t=1}^{n}ln({\epsilon }_t)-\frac{{\sum }_{t=1}^n{\epsilon }_t}{\widehat{\beta }})\ast I_{all{\epsilon }_t\ge 0}-M\ast I_{some{\epsilon }_t<0},$$

(9)

where M is some large-enough number.

As the Nelder–Mead method is a heuristic search method, the choice of initial point may greatly affect the result and, thus, the estimation process takes various initial points and returns the result that yields a best fit, using the AIC or BIC. Furthermore, a candidate set of AR order p is given and the procedure searches for the best AR order within the set, again by AIC and BIC. Specifically, in the scope of the simulation study in this report, the initial points for $\phi$ are set uniformly within [0,1] and the initial points for T are set within [$\mu -n\ast \sigma$, $\mu +n\ast \sigma$], where $\mu$ the sample mean of the RV, $\sigma$ is the sample variance, and n is a pre-determined number to control the range, here set as 0.5. The step size of $\phi$ is set to be 0.25 and that of T to be 0.05$\sigma$. For empirical data analysis, values of $\phi$ in the ranges [0,0.5] and [0.5,1] are tested, with step size 0.125, and the results showed that the outcome from [0,0.5] almost always dominated that from [0.5,1] and, thus, the range [0,0.5] and step size 0.125 were used for $\phi$.

The fitting of the TAR(p) model is essentially the same, except that the random errors are classified into two different regimes. Thus, the log-likelihood function is expressed as:

$$\begin{array}{c}\hfill \widehat{l}=(-n_1\ast ln(\Gamma (\widehat{{\alpha }_1}))-n_1\ast \widehat{{\alpha }_1}\ast ln(\widehat{{\beta }_1})+(\widehat{{\alpha }_1}-1)\ast \sum _{t=1}^{n_1}ln({\epsilon }_{1,t})-\frac{{\sum }_{t=1}^{n_1}{\epsilon }_{1,t}}{\widehat{{\beta }_1}}\\ \hfill -n_2\ast ln(\Gamma (\widehat{{\alpha }_2}))-n_2\ast \widehat{{\alpha }_2}\ast ln(\widehat{{\beta }_2})+(\widehat{{\alpha }_2}-1)\ast \sum _{t=1}^{n_2}ln({\epsilon }_{2,t})-\frac{{\sum }_{t=1}^{n_2}{\epsilon }_{2,t}}{\widehat{{\beta }_2}})\\ \hfill \ast I_{all{\epsilon }_{1,t},{\epsilon }_{2,t}\ge 0}-M\ast I_{some{\epsilon }_{1,t},{\epsilon }_{2,t}<0},\end{array}$$

(10)

where ${\epsilon }_{1,t}$ are the random errors corresponding to the observations in the first regime, $n_1$ is the number of observations in the first regime, and ${\epsilon }_{2,t}$ and $n_2$ the corresponding counterparts in the second regime, respectively.

A final concern regarding the model estimation would be that, for the first few observations, the AR model may not be properly initiated, as there are no earlier observations. Therefore, the sample estimates are essentially estimated by a sample, with the first few observations serving only as the independent variable, but not the dependent variable; that is,

$$R{V}_{t+n}=\sum _{i=1}^p{\phi }_i\ast R{V}_{t+n-i}+{\epsilon }_{t+n},$$

(11)

with n being the truncated size. Additionally, as the AIC and BIC are typically applied on the same sample with the same sample size, to allow for the comparison between models of different AR order and lag, a common truncation of size 10 is applied in the scope of this study, as the AR order and lag investigated did not exceed this reasonably.

As with the process of fitting the AR(p) model, the fitting for TAR(p) searches for the best model of AR order p and lag d, where p and d are given in the pre-determined candidate set and the threshold T.

2.3. Empirical Data Analysis Preparation

The data used in this paper were the consolidated realized volatility data from Shen et al. (2018), which are the realized volatilities for 30 stocks traded on the New York Stock Exchange (NYSE).

Graphs of PACF and the corresponding naive 95% confidence bound, proposed by Quenouille (1949), were first examined for the stock data, which showed that the PACF of the stocks were mostly significant within a lag of 5 and demonstrated a somewhat cut-off property; thus suggesting the fitting the AR model was potentially a good starting point. Non-linear threshold type AR models were also considered as a supplement to the AR model.

After considering the practical reasonableness of the model and the computational power available, an AR order up to 5 and lag order up to 3 were considered.

The final model for each stock was determined by both considering the AIC and BIC and the associated Ljung–Box test for each criterion. If the model selected by the two criteria differed with a similar goodness of fit, a simpler model was preferred. Otherwise, the model that gave a better goodness of fit result was preferred.

The data set and R code used for the study are available upon request, from either author.

3. Results

This section aims to briefly describe how the proposed AR and Threshold AR (TAR) models were fitted with a simulation study and some empirical data.

3.1. Simulation Study

A simulation study was conducted, by running the model-fitting process on batches of randomly generated AR or TAR models of observation length 500 and batch size 50 for all the results in this section (i.e., 50 simulated observations of length 500 were considered in each simulation experiment). The completion of each simulation took around half a day on a laptop. The following tables give the results for the bias and mean square error in the simulation study.

Table 1. Simulation results for the threshold Autoregressive (TAR) (2) model with $d=2$.

True Model	${\mathit{\alpha }}_1$	${\mathit{\beta }}_1$	${\mathit{\alpha }}_2$	${\mathit{\beta }}_2$	${\mathit{\phi }}_{1,1}$	${\mathit{\phi }}_{1,2}$	${\mathit{\phi }}_{2,1}$	${\mathit{\phi }}_{2,2}$	T
	5	2	5	2	0.5	0.3	0.3	0.2	30
AIC	Proportion of correct estimation of Autoregressive (AR) order and Lag: 44/50
AIC Bias	0.032	0.023	0.341	−0.022	0.013	−0.007	0.003	−0.004	0.001
AIC MSE	1.245	0.083	2.759	0.123	0.002	0.003	0.002	0.002	0.001
BIC	Proportion of correct estimation of AR order and Lag: 50/50
BIC Bias	0.015	0.022	0.384	−0.015	0.012	−0.006	0.004	−0.005	0.001
BIC MSE	1.198	0.08	3.887	0.143	0.002	0.003	0.002	0.002	0.001

and

Table 2. Simulation results for the TAR (1) model with d = 1.

True Model	${\mathit{\alpha }}_1$	${\mathit{\beta }}_1$	${\mathit{\alpha }}_2$	${\mathit{\beta }}_2$	${\mathit{\phi }}_{1,1}$	${\mathit{\phi }}_{2,1}$	T
	4	2	4	2	0.7	0.3	15
AIC	Proportion of correct estimation of AR order and Lag: 36/50
AIC Bias	0.34	−0.074	0.068	−0.006	−0.008	0.007	0.019
AIC MSE	0.929	0.086	0.722	0.078	0.003	0.001	0.002
BIC	Proportion of correct estimation of AR order and Lag: 50/50
BIC Bias	0.199	−0.035	0.016	0.024	−0.002	0.007	0.019
BIC MSE	0.825	0.079	0.711	0.082	0.003	0.001	0.002

give the results for the threshold models and

Table 3. Simulation results for the AR (2) model.

True Model	$\mathit{\alpha }$	$\mathit{\beta }$	${\mathit{\phi }}_1$	${\mathit{\phi }}_2$
	5	2	0.6	0.2
AIC	Proportion of correct estimation of AR order and Lag: 7/50
AIC Bias	0.627275	−0.08103	0.020833	−0.02968
AIC MSE	2.004022	0.087312	0.001747	0.002992
BIC	Proportion of correct estimation of AR order and Lag: 46/50
BIC Bias	0.137581	−0.02197	0.006732	−0.00646
BIC MSE	0.605966	0.037434	0.001058	0.001104

gives the result for AR models. The correct estimation of AR order and lag meant that the estimation of both the AR order p and the lag d were in line with the true parameters. The bias and MSE were calculated with the results in the simulations which gave the correct estimation of AR order and lag. The parameter for the true TAR model was selected such that the TAR structure was reasonably demonstrated (i.e., there were not too few observations in any regime).

The estimates of the AR order and lag were generally good, except the AIC criterion for the AR model, as the AIC tends to pick a more complicated model. In fact, the AIC estimated the AR order correctly in 36 out of 50 cases; yet, in most of these cases, it preferred a threshold structure.

The simulation results show that the model could identify the correct AR order p and the lag d with good accuracy in general, the estimate for the threshold T was very consistent; and the results for the AR parameters $\phi$ were rather accurate when p and d were estimated correctly. It should be noted that the accuracy here is defined as the probability of identifying the correct AR model order and correct threshold, given that the underlying model was indeed an AR/TAR model.

3.2. Empirical Data Analysis

The realized volatilities of 30 stocks traded on the New York Stock Exchange (NYSE) were tested by the proposed models. Please kindly refer to Appendix A, Appendix B and Appendix C for the best model selected by the AIC, BIC, and the final model.

From the results, the AR/TAR model seemed to be a good fit for around 33% of the cases, with almost all of the final models having a threshold structure and a marginally good fit for another 10% of the cases, where the Ljung–Box test was marginally significant. This demonstrates that, overall, the proposed model has the potential to explain a little less than half of the empirical data, in this case, and further investigation, through other data sets or improved fitting algorithms, is worthwhile.

4. Discussion

This section aims to provide a brief discussion as a supplement to the results found above. It is divided into discussions regarding the simulation study and the empirical data, respectively.

4.1. Simulation Study

While, as mentioned before, the bias and MSE were acceptable overall, with consistent estimates for the AR parameter and threshold, it can be noticed that the estimates for Gamma parameters were more volatile. This is possibly due to the profile likelihood methodology adopted for estimation, which increases the complexity in estimating the Gamma parameters.

Additionally, as the simulation study was constructed in such a way that the true model was within the set of candidate models, the BIC would select the true model with probability tending to one and, thus, outperformed the AIC. However, in practice, the true model may not reside within the candidate set, and the AIC may give a better result, yet may also choose a more complicated model (as mentioned above), while the BIC would prefer a simpler model. Therefore, in terms of forecasting MSE, both criteria are considered, in practice, for model selection.

4.2. Empirical Data Analysis

A residual analysis was conducted by looking at the PACF plots for the models with significant goodness of fit test results. It was observed that, in some cases, the PACF still demonstrated a rather clear cut-off at a higher order, suggesting that the AR order of the model could be further increased. Thus, it is suggested that, in this case, it is possible that the model was not a good enough fit, as it did not select a high enough order. This was possible, as the model fitting limited the highest AR order to be less than five, for practical concerns, and as the optimization process was sensitive to the selection of initial points and the initial points were evaluated in a sparser set at higher AR orders, thus resulting in a less-than-ideal fit.

Alternative models with non-Gaussian error provide another perspective of improvement. A Buffered Threshold Autoregressive (BAR) model, as described in Li et al. (2015), has been examined, using a fitting methodology similar to that of the TAR model. However, as the goodness of fit did not improve much, and as it is natural to choose a simpler model given similar goodness of fit, the results of BAR model have not yet been reported. However, other models (such as the Autoregressive Moving-Average model (ARMA)) could still be considered.

5. Conclusions

In this article, the model fitting of a non-Gaussian model on the realized volatility is explored. As the definition of realized volatility requires it to be positive, previous works established a Wishart model (a multi-variate analog of the chi-square distribution) that belongs to the Gamma family; considering this selection, a univariate Gamma random error is proposed and the AR and TAR models are explored. MLE estimation, based on the AIC and BIC, and with some adjustment, is proposed. A profile likelihood method, which replaces the Gamma parameters with their MLE counterparts, is used to reduce the dimension of the estimation and a non-gradient numerical optimization method is employed, as the calculation of gradient may not be feasible. A penalty method is introduced into the likelihood function, to enforce the non-negative constraint imposed by Gamma random error. The proposed process manages to find the true model with a rather insignificant bias and MSE, when the true model is AR or TAR. Finally, the model is tested on the empirical realized volatility data of 30 stocks and managed to fit one third of the cases quite well, suggesting that the model may have the potential to be further generalized, in order to act as a good supplement for current Gaussian random error models. The lack of fit may be improved by considering higher AR orders or a denser initial point selection for the Nelder–Mead method, which requires more computational time. Other possible directions of improvement include using a better method (instead of AIC or BIC) to reduce the ambiguity in choosing the model and possibly using other AR structures, such as the Heterogeneous Auto-Regressive (HAR) model. Other time-series models with non-Gaussian error may also be considered and the model fitting methodology proposed in this article could possibly be extended to these models without difficulty.