- freely available
Econometrics 2016, 4(1), 3; doi:10.3390/econometrics4010003
Abstract: Financial asset returns are known to be conditionally heteroskedastic and generally non-normally distributed, fat-tailed and often skewed. These features must be taken into account to produce accurate forecasts of Value-at-Risk (VaR). We provide a comprehensive look at the problem by considering the impact that different distributional assumptions have on the accuracy of both univariate and multivariate GARCH models in out-of-sample VaR prediction. The set of analyzed distributions comprises the normal, Student, Multivariate Exponential Power and their corresponding skewed counterparts. The accuracy of the VaR forecasts is assessed by implementing standard statistical backtesting procedures used to rank the different specifications. The results show the importance of allowing for heavy-tails and skewness in the distributional assumption with the skew-Student outperforming the others across all tests and confidence levels.
Value-at-Risk (VaR) is a quantitative tool used to measure the maximum potential loss in value of a portfolio of assets over a defined period for a given probability. Specifically, VaR construction requires a quantile estimate of the far-left tail of the unconditional returns distribution. Though widely-used as a risk measure in the past, standard methods of VaR construction assuming iid-ness and normality have come under criticism due to their failure to incorporate three stylized facts of financial returns, namely the presence of volatility clustering, indicated by high autocorrelation of absolute and squared returns, excess kurtosis (fat tails) and skewness in the density of the unconditional returns distribution.
The ability to account for volatility clustering is one of the key strengths of the ARCH modelling approach developed in Engle  and extended in Bollerslev . By combining this approach with a non-normal conditional distribution assumption for the returns, several papers have shown that univariate GARCH models can produce reliable out-of-sample volatility forecasts. For example, Angelidis et al.  combine three GARCH specifications with the univariate skew-Student and skew-GED (Generalized Error) distributions to show that these are able to produce superior VaR forecasts compared to the normal. Specifically, they apply the exponential GARCH (EGARCH) model of Nelson  and the threshold ARCH (TARCH) model to five univariate returns series and find that while the choice of a skewed, heavy-tailed distribution significantly improves the forecasting performance, the choice of the volatility model appears to be irrelevant. These findings are echoed in Mittnik and Paolella  who combine the Asymmetric Power ARCH (APARCH) model of Ding et al.  with an asymmetric generalised Student distribution. Within the univariate framework the most complete study of VaR prediction methods is provided by Kuester et al.  who compare fully parametric models with VaR constructed using historical simulation, extreme-value theory and quantile regression. Their results show that considerable improvement over normality is achieved when using innovation distributions that allow for skewness and fat tails.
Another salient feature of financial returns series is the fact that comovements between markets increase during periods of high volatility, as shown for example by Longin and Solnik  and Brooks et al. . In light of this, a number of studies predict the VaR of a portfolio using multivariate models for the system of individual asset returns in order to achieve forecast improvements due to the use of more information. However, as mentioned in Bauwens et al. , in high dimensional frameworks these models can suffer from the “curse of dimensionality” problem thus being more computationally intensive.
In order to shed some light on this issue, several papers focus on direct comparison of the predictive performance of univariate and multivariate GARCH (MGARCH) models under various distributional assumptions with the aim of providing evidence in favour of one of the two approaches. Key studies in this literature include Giot and Laurent  and Santos et al. . In the first, the authors compare the univariate APARCH model with a multivariate TVC-APARCH combined with the normal, Student and skew-Student distributions, showing that in both the univariate and multivariate settings the latter produced superior VaR forecasts. In the second, the authors study a number of univariate and multivariate volatility models with normal and Student distributions, finding that the multivariate models combined with the Student offer superior out-of-sample performances.
This paper builds on their approach by widening the set of distributions used to model the error term in both the univariate and multivariate frameworks while maintaining a generic specification for the conditional volatility. Specifically, we consider three symmetric distributions, i.e., normal, Student and Multivariate Exponential Power (MEP), and their corresponding skewed counterparts obtained by applying the transformation of Bauwens and Laurent  1. By incorporating skewness in the corresponding symmetric densities by means of new parameters, we can explicitly analyse its marginal contribution as well as its joint effect with heavy-tails in the model forecasting performance.
As for the choice of the volatility models, within the MGARCH literature we employ the Rotated BEKK (RBEKK) model of Noureldin et al.  as it is easy to estimate using covariance targeting even for moderately large cross sections, while in the univariate setting GARCH(1,1) specifications are used to modeling the conditional portfolio variance. Our choice of a relatively simple volatility model allows us to limit our focus entirely on whether the chosen distribution contributes to adequately capture features of the return data. Within the univariate scenario we also consider the NCT-APARCH model proposed by Krause and Paolella , as the authors developed an extremely fast method for parameter estimation that is found to outperform highly competitive models and can be easily compared to ours 2.
Both univariate and multivariate models are estimated employing the aforementioned set of distributional assumptions and their accuracy in producing out-of-sample VaR forecasts is assessed by means of statistical backtesting procedures. The selected tests include the Unconditional Coverage (UC), Independence (IND) and Conditional Coverage (CC) tests of Christoffersen , the Duration-Based Test of Independence (DBI) of Dumitrescu et al. , the Time Until First Failure (TUFF) test of Kupiec  and the Dynamic Quantile (DQ) test of Engle and Manganelli . The results of the tests are summarized using a grading scheme based on the number of acceptances of the null hypothesis which determines the distributional assumption providing the most accurate VaR forecasts.
Results from VaR backtesting show that in the multivariate setup the skew-Student clearly outperforms all other distributions. Moreover, its univariate version produces more accurate VaR forecasts than the NCT-GARCH and is able to compete with the NCT-APARCH which incorporates asymmetry into the conditional volatility specification. Overall, our results show that allowing for heavy-tails and skewness produces the most accurate VaR forecasts in both the univariate and multivariate setups. By comparison, specifications including only heavy tails underperform relative to their skewed counterparts with the difference being more pronounced at 5% VaR. Interestingly, the univariate skew-normal distribution produces Var forecasts comparable to the more heavily-parametrized skew-Student and skew-MEP. As regards comparing the performance of univariate and multivariate models in general, our results do not allow us to clearly advocate the use of one methodology over the other. However, given that the hierarchy of distributions according to VaR forecast accuracy is preserved under both frameworks, the univariate approach may be preferred due to its lower computational burden.
The paper is organized as follows: Section 2 reviews the GARCH modelling framework, the theoretical methodology used for constructing the skewed distributions and the Maximum Likelihood (ML) estimation of the models with the selected distributional assumptions. Section 3 introduces the empirical methodology, comprising the portfolio construction, VaR estimation and backtesting procedures. Section 4 provides estimation results and outcomes of the VaR tests and Section 5 concludes with some final remarks.
2. Theoretical Framework
This section illustrates the key points of our theoretical framework. Namely, we outline the alternative approaches to obtain portfolio VaR forecasts using univariate and multivariate models, we describe the procedure used to construct skewed distributions from the corresponding symmetric counterparts and finally we provide an overview of the set of employed distributions, comprising their likelihood derivation.
Two things are worth mentioning. First, we focus on the portfolio VaR for a long position, implying that the predictive power of the models is linked to their ability in modelling large negative returns. Second, we define the asset allocation scheme via an N-dimensional vector of equal weights , where N denotes the number of assets in the portfolio and , with , . As shown in DeMiguel et al. , Tu and Zhou , Brown et al.  and Fugazza et al. , this “naive” diversification rule is able to consistently outperform more sophisticated methods. Moreover, it has the advantage of not being affected by the specified target return as in the Markowitz framework, being only driven by the number of assets.
2.1. VaR Estimation
Let denote the N-dimensional discrete time vector of de-meaned daily returns at time t and the vector of equal weights known at time . The portfolio return is obtained as , and the portfolio VaR at time t is equal to
The specification of the portfolio standard deviation depends on whether we consider a univariate or multivariate approach, with the difference between the two occurring in the conditioning set used.
In the univariate case, the portfolio standard deviation is obtained as the standard deviation of the portfolio returns conditional on past portfolio returns, i.e., as the square root of
In the multivariate setup, the conditioning set is made up of the entire vector of past returns:
2.2. Constructing Skew Densities
As far as financial applications are concerned, modelling and inference based on the normal distribution have often been proven to be of limited usefulness, as it is possible to gain statistical efficiency by allowing for more involved distributions featuring heavy tails and skewness. As a way to capture higher moments, the literature offers several alternatives. For example, the multivariate noncentral t distribution has fat tails and is skewed; however, the skewness is linked directly to the location parameter, making it somewhat inflexible. The lognormal distribution has also been used to model asset returns, but its skewness is a function of its mean and variance, not a separate parameter. Others, such as the generalized hyperbolic (GH) or mixtures of distributions have also been employed in financial applications, despite being more computationally demanding (see Barndorff-Nielsen  for an introduction and Paolella and Polak  for a recent application).
In this respect, Fernández and Steel  in the univariate case and Bauwens and Laurent  in the multivariate one developed a practical procedure for constructing skewed densities from their symmetric unimodal counterparts. These densities can be defined by introducing skewness in the corresponding symmetric densities by means of new parameters, such that the symmetric density results as a particular case. We build on their findings in order to enlarge the set of distributions to be used for VaR forecasting. In the following, we briefly recall the main steps of the procedure, being the discussion restricted to the more general multivariate framework for sake of space.
We begin by defining the notion of symmetry of a standardized density used hereafter. In the univariate case, symmetry corresponds to , where is a unimodal probability density function with zero mean. In the multivariate case, we rely on the general notion of M-symmetry stated in Definition 1 of Bauwens and Laurent , which encompasses the class of spherically symmetric densities. These can be obtained as a special case of the general family of multivariate elliptical distributions, denoted as
The idea of introducing skewness into an M-symmetric standardized distribution revolves around scaling it differently for negative (positive) values by multiplying (dividing) by a positive constant. The value of this scaling parameter (hereafter referred to as ξ) determines whether the resulting distribution is skewed to the left () or to the right ().
Definition 1. Given a random vector with multivariate symmetric standardized distribution following Equation (6), the standardized skewed density with vector of asymmetry parameters , is obtained as:
The marginal -order moment of the obtained skewed distribution can be computed directly from the standardized moment of the symmetric density . This is accomplished by applying the following transformation function:
As shown in Appendix A, can also be expressed directly as a function of . Note that the resulting skewed distribution, from Definition 1, is not centered at 0 and the variance is a function of ξ (and, where is the case, of the shape parameter η). Given that the elements of are uncorrelated (since those of x are uncorrelated by assumption), standardization of is achieved by the following transformation:
As already mentioned, three symmetric and three asymmetric distributions are considered in the univariate and multivariate framework. Again, for sake of brevity, we only report the multivariate log-likelihood functions as the univariate can be obtained as special cases 3. The algebraic derivations of the formulas of the moments can be found in Appendix A.2. In all cases, estimation of the parameters is performed in one step by Maximum Likelihood (ML). Namely, the log-likelihood function for T observations is expressed as
Multivariate normal distribution This is the most commonly employed distribution in the literature as it is uniquely identified by its conditional first and second moments, which renders ML estimation much simpler from a computational point of view. In addition, given that the score of the normal log-likelihood function has the martingale difference property when the first two conditional moments are correctly specified, the Quasi Maximum Likelihood (QML) estimates are still consistent and asymptotically normal even if the true DGP is not normally-distributed Bollerslev and Wooldridge . The log-likelihood function for T observations is expressed as follows
Multivariate Student distribution The Student distribution is a symmetric and bell-shaped distribution, with heavier tails than the normal. Under the multivariate Student assumption, the log-likelihood function is obtained as
Multivariate Exponential Power (MEP) distribution This distribution belongs to the Kotz family of distributions (a particular class of symmetric and elliptical distributions discussed extensively in Fang et al. ) and is known to have several equivalent definitions in the literature. It can also include both the normal and the Laplace as special cases, as a function of the value of the non-normality parameter β dictating the tail-behaviour of the distribution. Given its simple implementation, in this paper we consider the pdf given in Solaro , which gives rise to the following log-likelihood function:
Multivariate skew-normal distribution The multivariate skew-normal is the first non-symmetric distribution we consider herein; it accounts for the skewness of the return distribution without taking into account its kurtosis (as it does not involve a tail parameter). Applying Definition 1 we derive the skew-normal density function, with corresponding log-likelihood function equal to
Multivariate skew-Student distribution Applying the same procedure as for the skew-normal, the log-likelihood function of the skew-Student distribution is given by the following expression:
Multivariate skew-MEP distribution The log-likelihood function to be maximized is given by
3. Empirical Application
3.1. Data and Forecasting Scheme
Our dataset (used in the paper of Noureldin et al. ) 5 comprises daily open-to-close returns of 10 stocks from the Dow Jones Industrial Average: Bank of America (BAC), JP Morgan (JPM), International Business Machines (IBM), Microsoft (MSFT), Exxon Mobil (XOM), Alcoa (AA), American Express (AXP), Du Pont (DD), General Electric (GE) and Coca Cola (KO). Each univariate vector of returns is calculated as and covers a period of 2200 days, from February 2001 to November 2009. Univariate descriptive statistics over the period of interest are provided in Table 1.
|Stock||Mean||Std.dev.||Skewness||Kurtosis||KS Test||JB Test|
|Estimation sample: 1 February 2001 to 23 January 2007 (1500 observations)|
|Forecasting sample: 24 January 2007 to 30 October 2009 (700 observations)|
|Full sample: 1 February 2001 to 30 October 2009 (2200 observations)|
Descriptive statistics of the stock return time series used in the empirical application. The three panels report the statistics for the in-sample period, the out-of-sample period and the full sample period, respectively. “KS test” and “JB test” denotes the Kolmogorov-Smirnov test and Jarque Bera test, with corresponding p-values in column.
Across the three panels, the values of skewness and kurtosis show that the assets are far from being unconditionally normally distributed, thus supporting the conjecture that more flexible distributional assumptions can be conducive to enhanced model performance.
To this extent, one-step ahead forecasts of the conditional portfolio variance (in the univariate case) and of the conditional covariance matrix of returns (in the multivariate one) are recursively obtained as:
For each model, the portfolio VaR forecast at α% confidence level is then obtained as
For the symmetric distributions in our analysis (normal, Student and MEP), one can easily compute the long VaR of the portfolio by applying Equation (27) and the inverse of each CDF at α%. However, for the non-symmetric distributions this is not straightforward. In order to bypass this complication, for each non-symmetric distribution we apply a simple Monte-Carlo simulation approach, as widely used in VaR computations. Namely, we draw random vectors (numbers) from each symmetric multivariate (univariate) standardized distribution and then we use the estimated skewness parameters to construct the corresponding skewed distribution . By assuming () as the true DGP, we obtain a set of 10,000 simulated returns over the period of interest. Finally, the simulated return distribution is used to derive the 5% and 1% quantiles for the one-step-ahead VaR.
3.2. Testing the Accuracy of VaR Forecasts
The models accuracy in predicting VaR is assessed using multiple statistical backtesting methods. A common starting point for this procedure is the so-called hit function, or indicator function, which is equal to
This represents the key foundation to many of the backtesting procedures developed in recent years and particularly to the accuracy tests being used in this paper. We focus on tests included in the following three categories:
Evaluation of the Frequency of Violations
Evaluation of the Independence of Violations
Evaluation of the Duration between Violations.
Frequency of Violations The first way of testing the VaR accuracy is to test the number or the frequency of margin exceedances. A test designed to this aim is the Kupiec test (Kupiec ), also known as the Unconditional Coverage (UC) test. Its null hypothesis is simply that the percentage of violated VaR forecasts or failure rate p is consistent with the given confidence level α, i.e., .
Denoting by F the length of the forecasting period and with v the number of violations occurred throughout this period, the log-likelihood ratio test statistic is defined as
A similar useful test is the TUFF (Time Until First Failure) test (Kupiec ). Under the null, the probability of an exception is equal to the inverse probability of the VaR confidence level, namely . Its basic assumptions are similar to those of the Kupiec test and the t-statistic under the null is obtained as
Independence of Violations A limitation of the Kupiec test is that it is only concerned with the coverage of the VaR estimates without accounting for any clustering of the violations. This aspect is crucial for VaR practitioners, as large losses occurring in rapid succession are more likely to lead to disastrous events than individual exceptions.
The Independence test (IND) of Christoffersen  uses the same likelihood ratio framework as the previous tests but is designed to explicitly detect clustering in the VaR violations. Under the null hypothesis of independence, the IND test assumes that the probability of an exceedance on a given day t is not influenced by what happened the day before. Formally, , where denotes the probability of an i event on day being followed by a j event on day t.
The relevant IND test statistic can be derived as
Although the aforementioned test has received support in the literature, Christoffersen  noted that it was not complete on its own. For this reason, he proposed a joint test, the Conditional Coverage (CC) test, which combines the properties of both UC and IND tests.
Formally, the CC ratio statistic can be proven to be the sum of the UC and the IND statistics:
A second test belonging to this class is the Regression-based test of Engle and Manganelli , also known as Dynamic Quantile (DQ) test. Instead of directly considering the hit sequence, the test is based on its associated quantile process which assumes the following values:
The idea of this approach is to regress current violations on past violations in order to test for different restrictions on the parameters of the model. That is, we estimate the linear regression model and then we test the joint hypothesis . This assumption coincides with the null of Christoffersen’s CC test. It is also possible to split the test and separately test the independence hypothesis and the unconditional coverage hypothesis, respectively as and . and are asymptotically distributed with respectively , K and one degrees of freedom.
Duration between Violations One of the drawbacks of Christoffersen’s CC test is that it is not capable of capturing dependence in all forms, since it only considers the dependence of observations between two successive days. To address this, Christofferson and Pelletier  introduced the Duration-Based test of independence (DBI), which is an improved test for both independence and coverage. Its basic intuition is that if exceptions are completely independent of each other, then the upcoming VaR violations should be independent of the time that has elapsed since the occurrence of the last exceedance (Campbell ). The duration (in days) between two exceptions is defined via the no-hit-duration , where is the day of violation.
A correctly specified model should have an expected conditional duration of days and the no-hit duration should have no memory. The authors construct the ratio statistic considering different distributions for the null and the alternative hypotheses, namely the exponential, since it is the only memory-free (continuous) random distribution, and the Weibull, which allows for duration dependence. The likelihood ratio statistic is derived as
Under the null hypothesis of independent violations, and a is estimated via numerical maximization of . Whenever , the Weibull function has a decreasing path which corresponds to an excessive number of very long durations (very calm period) while corresponds to an excessive number of very short durations, namely very volatile periods.
4.1. Parameter Estimates
Before turning to the out of sample analysis, it is worth first looking at some parameter estimates obtained by fitting the models on the data. Table 2 reports full sample estimation results over the period January 2001 to February 2009 for a total of 2200 daily observations.
Note: The table reports test statistics and robust standard errors obtained from full sample parameter estimation, for T = 2200. Note that denotes the value of the skewness parameters averaged across univariate series with Mean Asymptotic Square Errors (MASE) reported in brackets. For both the NCT-GARCH and NCT-APARCH models , see Appendix D. The AIC is rescaled by T.
A common feature emerging from both univariate and multivariate panels is that the use of skewed distribution assumptions seems to be justified, as all asymmetric coefficients are significant at standard levels. Moreover, the direct comparison of models fit via the Akaike Information Criteria (AIC) highlights that the models incorporating skewed distributions consistently outperform their symmetric counterparts, with the skew-Student achieving the best fit. In both cases the Student has the lowest AIC among the symmetric densities, while the normal possesses the highest.
In the univariate setting, the NCT-APARCH model substantially improves over the NCT-GARCH due to the introduction of skewness into the volatility model, but it performs slightly worse than its closest competitor, the skew-Student GARCH model. This performance is even more impressive considering that the AIC penalizes the NCT-based models for only three parameters, given that remaining ones are fixed prior to maximization of the likelihood function according to the fast procedure of Krause and Paolella  (see Appendix D).
The evolution of the parameters across re-estimations allows us to discern their sensitivity to the financial crisis by comparing their dynamics against the key dates outlined in Table A1 in Appendix B. From Figure 1, which looks at the tail parameter estimates for the MEP and Student distributions along with their skewed counterparts, a number of commonalities are immediately apparent. Panel (a) reveals a small increase followed by a sharp drop in β between and 22 for both the univariate and multivariate MEP, indicating a thickening of the tails. Since the time frame corresponds to the windows including the events leading up to the collapse of Lehman Brothers on 15 September 2008, we interpret our results as β adjusting to incorporate the extreme negative events associated to this period.
Panel (b) exhibits the same dynamics as before albeit at a reduced magnitude. This occurs due to the introduction of skewness which is able to capture some of the negative returns associated to the crisis, thereby reducing the need for such a large increase in tail thickness. Evidence of such compensating effects between skewness and heavy-tails is given in Figure 2 which provides the evolution of the skew-parameter ξ across re-estimations (given as an average over the assets in the multivariate case for ease of exposition). Specifically, the dynamics of ξ for the univariate skew-MEP in Panel (a) appear to track the tail dynamics: as β decreases on average between τ equal 1 and 18, ξ exhibits a gradual increase, indicating a reduction of negative skewness. Moreover, the aforementioned drop is accompanied by ξ attaining its maximum value.
Turning to the Student and skew-Student distributions, the dynamics are somewhat different along a number of dimensions. First, there is now a marked difference between the univariate and multivariate cases. In the former, the dynamics are similar to the MEP and skew-MEP results wherein the tails thicken at around as evidenced by the sharp drop in ν. This is then followed by a short-lived increase after which the tails thicken gradually over the remaining estimation period. The reasoning for this, whereby the tails thicken to accommodate the negative returns associated to the crisis, is the same as the MEP-case.
Unlike the MEP and skew-MEP cases, where the multivariate specification exhibits thinner tails than the univariate but both exhibit congruent dynamics, the Student and skew-Student exhibit fewer similarities between specifications. In contrast, moving from the Student to skew-Student setting reveals almost identical tail parameter dynamics. Combined with the positive skew exhibited in Panel (b) of Figure 2, this indicates that in this case tail thickness does not adjust to incorporate the addition of skewness. The effect of these features on VaR forecast accuracy will be seen in Section 4.2.
4.2. VaR Backtesting Results
In the multivariate setting, the out-of-sample covariance matrix predictions are used to construct equally-weighted portfolios for the computation of the one-step-ahead VaR. Table 3 compares portfolios standard deviation obtained using the univariate and multivariate approaches over both the in- and out-of-sample periods.
|In-sample: 1 February 2001 to 23 January 2007 (1500 observations)|
|Forecasting sample: 24 January 2007 to 30 October 2009 (700 observations)|
|In-sample: 1 February 2001 to 23 January 2007 (1500 observations)|
|Forecasting sample: January 24, 2007 to October 30, 2009 (700 observations)|
Note: The table reports average, minimum and maximum value of portfolio standard deviation over the in- and out-of-sample periods.
As already noted, the financial crisis features heavily in the summary statistics. Since this period is included in the forecasting sample (starting from observation 1921 according to Table A1), we notice a sharp increase in the portfolio standard deviation of all the models (see also the figures reported in Appendix C). On a general basis, heavy-tailed or skewed distributions do not lead to remarkable gains over the in-sample period, as all the models report very similar values on average. This pattern is partly reversed in the forecasting period: while the normal and skew-normal remain very close, the skew-Student and skew-MEP exhibit a lower portfolio standard deviation than their symmetric counterparts. Remarkably good is the performance of the models featuring the NCT distribution; the NCT-APARCH achieves the lowest standard deviation among the univariate alternatives and compares favorably also to many multivariate models, being only outperformed by the skew-Student RBEKK. It appears that the uncertainty due to the larger number of parameters to be estimated is not fully compensated by the gain coming from a better representation of the volatility dynamics in multivariate models.
Overall, according to this table, the univariate and multivariate approaches deliver quite similar portfolio summary statistics. If the focus was on the predicted portfolio variance alone, then the ideal choice would be to use a univariate volatility model coupled with either the skew-Student or the NCT distribution, as they are easy to estimate and computationally faster than the multivariate specifications. Ultimately, we are interested in the models accuracy in forecasting the one-step-ahead portfolio VaR, so we move to analyse the outcomes of the statistical backtesting procedures.
All statistical tests are computed for the 5% and 1% VaR confidence level. For each portfolio we report test statistics along with their corresponding p-values in brackets. Since the applied tests measure the models accuracy in forecasting VaR along several dimensions (as detailed in Section 3.2), the overall results are summarized using a performance measure which considers the percentage of acceptances of the null hypothesis across the different tests at the standard significance level. To this end, rejections of the null are highlighted in bold with the total grade across distributions reported in Table 6.
The first distinguishing feature from the VaR backtesting results is the clear predominance of the skew-Student distribution. This holds for both the univariate and multivariate frameworks with the former producing VaR forecasts that outperform the NCT-APARCH at 5% VaR. Similarly, the skew-MEP distribution produces highly accurate VaR forecasts across the board. These findings exemplify a second feature of the results namely, the impact on VaR forecast accuracy of introducing skewness into a heavy-tailed distribution. Clearly, the performance of both the skew-Student and skew-MEP distributions improves compared to their symmetric counterparts, but this effect is less pronounced at 1% VaR. Indeed, the performance of the Student and MEP distributions improves when moving from the 5 % to the 1% VaR scenario, suggesting that despite the improvement arising from the introduction of skewness, heavy-tails remain useful in capturing larger swings in returns, i.e., those events located further out in the tails. As a final remark on the effect of skewness, we observe the large difference in performance between the NCT-APARCH and NCT-GARCH at 5% VaR. Though both feature a skewed distributional specification, recall that the NCT-APARCH also features asymmetry in the conditional volatility model which explains its superior performance. An analogous result is obtained in Giot and Laurent .
Analysis of the differences between the univariate and multivariate models reveals two key points. First, while the skew-Student and skew-MEP retain their dominance under both frameworks, the performance of their symmetric counterparts at 1% VaR is worse under the multivariate specification. Second, within the univariate framework, the skew-normal is capable of producing VaR forecasts comparable to the high performance skewed and heavy-tailed distributions at both VaR confidence levels. This does not hold in the multivariate setup where the skew-normal offers no improvements in VaR accuracy over the normal. Besides, we observe that in both frameworks the empirical failure rate (i.e., the frequency of violations) of the skewed distributions is closer to the nominal value than their symmetric counterparts, which are oversized. The NCT-GARCH represents the only exception, being considerably more conservative than the NCT-APARCH at both 5% and 1% VaR confidence levels.
With respect to the previous backtesting methods, the DQ test takes into account a more general temporal dependence between the series of violations and is considered the most reliable in assessing VaR accuracy 6. The DQ test results are reported in Table 7 and Table 8. In order to compare the different distributional assumptions, Table 9 summarises the percentage of null hypothesis acceptances over the two lag-lengths for each confidence level.
The DQ test results tell a similar story to the VaR backtesting procedures. As before, the skew-Student outperforms its competitors at both 5% and 1% VaR under both univariate and multivariate setups. The univariate skew-normal continues to produce VaR forecasts with an accuracy comparable to the top-performers. Again, this does not extend to the multivariate case.
Finally, forecast accuracy at 5% VaR again reveals a more pronounced improvement when moving from symmetric to skewed distributions than in the 1% case. Overall, introducing skewness into heavy-tailed distributions continues to offer the highest VaR forecast accuracy.
|K = 1|
|K = 3|
|K = 1|
|K = 3|
|K = 1|
|K = 3|
|K = 1|
|K = 3|
Given its importance in risk management, practitioners must be capable of forecasting the Value-at-Risk of their asset portfolios to a high degree of accuracy. This requires taking into account a number of properties of financial returns namely, non-normality, heavy-tails, skewness and the possibility of comovements between assets. In this article, we focus primarily on the effect of varying the distributional assumption used to forecast VaR. Moreover, we addressed the still open question of whether univariate or multivariate models are most appropriate for the problem of portfolio VaR forecasting.
The distributions treated in the paper comprised three symmetric and three skewed distributional assumptions (i.e., normal, Student, MEP and their skewed counterparts) which were coupled with the RBEKK model in the multivariate framework and GARCH in the univariate. In addition, we compared our specification with a novel method for fast estimation of the (univariate) NCT-GARCH and NCT-APARCH models. We then proceeded to the models’ accuracy in predicting equally-weighted portfolio VaR.
Employing a series of standard backtesting methods to compare the distribution-based model performance, the results reveal that the skew-Student specification produces the most accurate one-step ahead VaR forecast across all multivariate specifications and is able to compete with the high-performance NCT-APARCH in the univariate setup. This finding is echoed in the univariate MEP results wherein the skewed version outperforms the symmetric across all tests and confidence levels. By contrast, the multivariate skew-MEP exhibits thinner tails than the normal and thus, performs poorly at the 1% confidence level. More generally, The test results reveal a clear hierarchy of distributional assumptions within the univariate and multivariate setups, with that hierarchy being preserved when moving from one to the other. However, attempting to compare the performance of univariate to multivariate distributions in general does not reveal any key differences with the exception of certain cases in the skew-normal and skew-MEP specifications. Consequently, the additional computational burden of estimating multivariate models does not seem to be justified.
There are several possible avenues of research extending from this work. Given the focus on distributions, we limited our attention to relatively simple parametric models namely, the RBEKK and GARCH models. As the NCT-GARCH/APARCH results showed, capturing asymmetries in returns volatility improves forecast accuracy. Considering their multivariate versions would provide a useful contribution and allow for a detailed study of the optimal model-distribution-dimension combination. Another possibility would be to consider higher forecast horizons for the VaR in order to check if the inclusion of skewness and asymmetric forms of dependence can lead to significant improvements in the long run.
Manuela Braione and Nicolas K. Scholtes acknowledge support of the “Communauté française de Belgique” through contracts “Projet d’Actions de Recherche Concertées” “12/17-045” and “13/17-055”, respectively granted by the Académie universitaire Louvain. We thank seminar participants at the 13th Journée d’économétrie at the Université Paris Ouest Nanterre La Défense along with Luc Bauwens and Florian Ielpo for useful comments. We also thank two anonymous referees and the academic editors for their comments and advice, all of which led to considerable improvements in the paper.
Manuela Braione and Nicolas K. Scholtes contributed equally to the writing of the MATLAB code as well as the preparation of the current manuscript.
Conflicts of Interest
The authors declare no conflict of interest.
The transformation is incorporated into the symmetric, standardised pdfs as follows:
A.2. Distributions Moments
We report the first two moments of the univariate symmetric normal, Student and MEP distributions used to compute the log-likelihood function as given in Section 2.3.
Note that the following relation holds:
|Rolling Fixed-Window||Forecast Horizon|
|1||1–1500||1 Febraury 2001–23 January 2007||1501–1520||24 January 2007–21 Febraury 2007|
|2||21–1520||2 March 2001–21 Febraury 2007||1521–1540||22 Febraury 2007–21 March 2007|
|3||41–1540||30 March 2001–21 March 2007||1541–1560||22 March 2007–19 April 2007|
|4||61–1560||30 April 2001–19 April 2007||1561–1580||20 April 2007–17 May 2007|
|5||81–1580||29 May2001–17 May 2007||1581–1600||18 May 2007–15 June 2007|
|6||101–1600||26 June 2001– 15 June 2007||1601–1620||18 June 2007–16 July 2007|
|7||121–1620||25 July 2001–16 July 2007||1621–1640||17 July 2007–13 August 2007|
|8||141–1640||22 August 2001–13 August 2007||1641–1660||14 August 2007–11 September 2007|
|9||161–1660||26 September 2001–11 September 2007||1661–1680||12 September 2007–9 October 2007|
|10||181–1680||24 October 2001–9 October 2007||1681–1700||10 October 2007–6 November 2007|
|11||201–1700||21 November 2001–6 Novembe 2007||1701–1720||7 November 2007–5 December 2007|
|12||221–1720||20 December 2001–5 December 2007||1721–1740||6 December 2007–4 January 2008|
|13||241–1740||22 January 2002–4 January 2008||1741–1760||7 January 2008–4 Febraury 2008|
|14||261–1760||20 Febraury 2002–4 Febraury 2008||1761–1780||5 Febraury 2008– 4 March 2008|
|15||281–1780||20 March 2002–4 March 2008||1781–1800||5 March 2008–2 April 2008|
|16||301–1800||18 April 2002–2 April 2008||1801–1820||3 April 2008–30 April 2008|
|17||321–1820||16 May 2002–30 May 2008||1821–1840||1 May 2008–29 May 2008|
|18||341–1840||14 June 2002–29 May 2008||1841–1860||30 May 2008–26 June 2008|
|19||361–1860||15 July 2002–26 June 2008||1861–1880||27 June 2008–25 July 2008|
|20||381–1880||12 August 2002–25 July 2008||1881–1900||28 July 2008– 22 August 2008|
|21||401–1900||10 September2002–22 August 2008||1901–1920||25 August 2008– 22 September 2008|
|22||421–1920||8 October 2002–22 September 2008||1921–1940||23 September 2008–20 October 2008|
|23||441–1940||5 November 2002–20 October 2008||1941–1960||21 October 2008–17 November 2008|
|24||461-1960||4 December 2002–17 November 2008||1961–1980||18 November 2008–16 December 2008|
|25||481–1980||3 January 2003–16 December 2008||1981–2000||17 December 2008–15 January 2009|
|26||501–2000||3 Febraury 2003–15 January 2009||2001–2020||16 January 2009–13 Febraury 2009|
|27||521–2020||4 March 2003–13 Febraury 2009||2021–2040||17 Febraury 2009–16 March 2009|
|28||541–2040||1 April 2003–16 March 2009||2041–2060||17 March 2009–14 April 2009|
|29||561–2060||30 April 2003–14 April 2009||2061–2080||15 April 2009–12 May 2009|
|30||581–2080||29 May 2003–12 May 2009||2081–2100||13 May 2009–10 June 2009|
|31||601–2100||26 June 2003–10 June 2009||2101–2120||11 June 2009–9 July 2009|
|32||621–2120||25 July 2003–9 July 2009||2121–2140||10 July 2009–6 August 2009|
|33||641–2140||22 August 2003–6 August 2009||2141–2160||7 August 2009–3 September 2009|
|34||661–2160||22 September 2003–3 September 2009||2161–2180||4 September 2009–2 October 2009|
|35||681-2180||20 October 2003–2 October 2009||2181–2200||5 October 2009–30 October 2009|
D. The Univariate NCT-APARCH Model
The main ingredients of the NCT-APARCH model discussed in the paper by Krause and Paolella  are, as the name may suggest, the use of a univariate noncentral t-distribution assumption for the error term and of an APARCH specification for the conditional variance equation.
The NCT density function is given as follows:
The evolution of the conditional variance is modeled according to the APARCH model, which allows for both heavy-tails and asymmetry. It is defined as follows:
E. Backtesting VaR: Augmented Independence and Conditional Coverage Tests
Table A2 and Table A3 report test-statistics and corresponding p-values obtained from the Independence (IND) and Conditional Coverage (CC) tests of Christoffersen (see Section 4.2) computed with a number of lagged observations .
The tests are built in an extended Markov framework that allows for higher, or K-th order, dependence in the VaR observations, thus highlighting possible clustering in the series of violations. The distribution of the generalized IND and CC tests is asymptotically and , respectively. For further details on the construction of the tests we remand the interested reader to theoretical framework developed in Pajhede .
Results for both univariate and multivariate specifications are reported below.
|IND (K = 3)|
|CC (K = 3)|
|IND (K = 3)|
|CC (K = 3)|
|IND (K = 3)|
|CC (K = 3)|
|IND (K = 3)|
|CC (K = 3)|
Overall, adding more lags in the regression equation does not seem to crucially affect the outcome of the tests, as the new results are still in line with those obtained under in Table 4 and Table 5. The main difference registered for both univariate and multivariate approaches is in the outcome of the IND test, which now all models in the tables pass at the standard level for both the 5% and 1% VaR. Apparently, violations are not clustered in time for more than one or two lags.
As for the CC test, results concerning the multivariate models are basically the same obtained by setting , while in the univariate case we observe an improvement in the performance of the models (excluding the normal) for the VaR.
Ultimately, univariate distributions featuring skewness and heavy tails confirm their predominance over the remaining alternatives in terms of better predictive ability. In the multivariate case this is particularly true for the skew-Student and the skew-MEP, as the skew-normal is not leading to remarkable improvements over the corresponding normal assumption.
- Engle, R. Autoregressive Conditional Heteroskedasticity with Estimates of United Kindgom Heteroskedasticity. Econometrica 1982, 50, 987–1007. [Google Scholar] [CrossRef]
- Bollerslev, T. Generalized Autoregressive Conditional Heteroskedasticity. J. Econom. 1986, 31, 307–327. [Google Scholar] [CrossRef]
- Angelidis, T.; Benos, A.; Degiannakis, S. The Use of GARCH Models in VaR Estimation. Stat. Methodol. 2004, 1, 105–128. [Google Scholar] [CrossRef]
- Nelson, D.B. Conditional heteroskedasticity in asset returns: A new approach. Econometrica 1991, 59, 347–370. [Google Scholar] [CrossRef]
- Mittnik, S.; Paolella, M. Conditional density and value-at-risk prediction of Asian currency exchange rates. J. Forecast. 2000, 19, 313–333. [Google Scholar] [CrossRef]
- Ding, Z.; Granger, C.W.; Engle, R.F. A long memory property of stock market returns and a new model. J. Empir. Financ. 1993, 1, 83–106. [Google Scholar] [CrossRef]
- Kuester, K.; Mittnik, S.; Paolella, M. Value-at-risk prediction: A comparison of alternative strategies. J. Financ. Econom. 2006, 4, 53–89. [Google Scholar] [CrossRef]
- Longin, F.; Solnik, B. Is the correlation in international equity returns constant: 1960–1990? J. Int. Money Financ. 1995, 14, 3–26. [Google Scholar] [CrossRef]
- Brooks, C.; Persand, G. Value-at-risk and market crashes. J. Risk 2000, 2, 5–26. [Google Scholar]
- Bauwens, L.; Laurent, S.; Rombouts, J. Multivariate GARCH models: A Survey. J. Appl. Econom. 2006, 21, 79–109. [Google Scholar] [CrossRef]
- Giot, P.; Laurent, S. Value-at-risk for long and short trading positions. J. Appl. Econom. 2003, 18, 641–663. [Google Scholar] [CrossRef]
- Santos, A.A.; Nogales, F.J.; Ruiz, E. Comparing univariate and multivariate models to forecast portfolio value-at-risk. J. Financ. Econom. 2013, 11, 400–441. [Google Scholar] [CrossRef]
- Bauwens, L.; Laurent, S. A New Class of Multivariate Skew Densities, with Application to GARCH Models. J. Bus. Econ. Stat. 2005, 23, 346–354. [Google Scholar] [CrossRef]
- Ley, C.; Paindaveine, D. Multivariate skewing mechanisms: A unified perspective based on the transformation approach. Stat. Probab. Lett. 2010, 80, 1685–1694. [Google Scholar] [CrossRef]
- Noureldin, D.; Shephard, N.; Sheppard, K. Multivariate rotated ARCH models. J. Econom. 2014, 179, 16–30. [Google Scholar] [CrossRef]
- Krause, J.; Paolella, M.S. A fast, accurate method for value-at-risk and expected shortfall. Econometrics 2014, 2, 98–122. [Google Scholar] [CrossRef]
- Christoffersen, P.F. Evaluating interval forecasts. Int. Econ. Rev. 1998, 39, 841–862. [Google Scholar] [CrossRef]
- Dumitrescu, E.; Hurlin, C.; Pham, V. Backtesting Value-at-Risk: From Dynamic Quantile to Dynamic Binary Tests. Finance 1995, 33, 79–111. [Google Scholar]
- Kupiec, P.H. Techniques for verifying the accuracy of risk measurement models. J. Deriv. 1995, 3, 73–84. [Google Scholar] [CrossRef]
- Engle, R.F.; Manganelli, S. CAViaR: Conditional autoregressive value at risk by regression quantiles. J. Bus. Econ. Stat. 2004, 22, 367–381. [Google Scholar] [CrossRef]
- DeMiguel, V.; Garlappi, L.; Uppal, R. Optimal versus naive diversification: How inefficient is the 1/N portfolio strategy? Rev. Financ. Stud. 2009, 22, 1915–1953. [Google Scholar] [CrossRef]
- Tu, J.; Zhou, G. Markowitz meets Talmud: A combination of sophisticated and naive diversification strategies. J. Financ. Econ. 2011, 99, 204–215. [Google Scholar] [CrossRef]
- Brown, S.J.; Hwang, I.; In, F. Why Optimal Diversification Cannot Outperform Naive Diversification: Evidence from Tail Risk Exposure. 2013. [Google Scholar] [CrossRef]
- Fugazza, C.; Guidolin, M.; Nicodano, G. Equally Weighted vs. Long-Run Optimal Portfolios. Eur. Financ. Manag. 2014, 21, 742–789. [Google Scholar] [CrossRef]
- Bauwens, L.; Omrane, W.; Rengifo, E. Intra-Daily FX Optimal Portfolio Allocation; CORE Discussion Paper 2006/10; CORE: Louvain-la-Neuve, Belgium, 2006. [Google Scholar]
- Engle, R.F.; Kroner, K.F. Multivariate simultaneous generalized ARCH. Econometr. Theory 1995, 11, 122–150. [Google Scholar] [CrossRef]
- Barndorff-Nielsen, O. Exponentially decreasing distributions for the logarithm of particle size. Proc. R. Soc. Lond. A Math. Phys. Eng. Sci. 1977, 353, 401–419. [Google Scholar] [CrossRef]
- Paolella, M.S.; Polak, P. COMFORT: A Common Market Factor Non-Gaussian Returns Model. J. Econom. 2015, 187, 593–605. [Google Scholar] [CrossRef]
- Fernández, C.; Steel, M.F. On Bayesian modeling of fat tails and skewness. J. Am. Stat. Assoc. 1998, 93, 359–371. [Google Scholar]
- Bollerslev, T.; Wooldridge, J. Quasi-Maximum Likelihood Estimation and Inference in Dynamic Models with Time-Varying Covariances. Econom. Rev. 1992, 1, 143–172. [Google Scholar] [CrossRef]
- Fang, K.; Kotz, S.; Ng, K. Symmetric Multivariate and Related Distributions; Chapman and Hall: London, UK, 1990. [Google Scholar]
- Solaro, N. Random variate generation from multivariate exponential power distribution. Stat. Appl. II 2004, 2, 25–44. [Google Scholar]
- Noureldin, D.; Shephard, N.; Sheppard, K. Multivariate high-frequency-based volatility (HEAVY) models. J. Appl. Econom. 2012, 27, 907–933. [Google Scholar] [CrossRef]
- Christoffersen, P. Evaluating Interval Forecasts. Int. Eco. Rev. 1998, 39, 841–862. [Google Scholar] [CrossRef]
- Kupiec, P. Techniques for Verifying the Accuracy of Risk Management Models. J. Deriv. 1995, 23, 73–84. [Google Scholar] [CrossRef]
- Campbell, S. A Review of Backtesting and Backtesting Procedures. J. Risk 2006, 9, 1–17. [Google Scholar]
- Engle, R.; Manganelli, S. CaViaR: Conditional Autoregressive Value at Risk by Regression Quantiles. J. Bus. Econ. Stat. 2004, 22, 367–381. [Google Scholar] [CrossRef]
- Christofferson, P.; Pelletier, D. Backtesting Value-at-Risk: A Duration-Based Approach. J. Financ. Econom. 2004, 2, 84–108. [Google Scholar] [CrossRef]
- Pajhede, T. Backtesting Value-at-Risk: The generalized Markov Test; U. Copenhagen Economics Discussion Paper 15-18; University of Copenhagen: Copenhagen, Denmark, 2015. [Google Scholar]
- 1.The interested reader is referred to Ley and Paindaveine  who provide a detailed overview of this approach.
- 2.We thank the authors for kindly providing us their MATLAB codes.
- 3.In Appendix D we provide a brief overwiew of the noncentral t distribution and its likelihood derivation.
- 4.In the univariate case it corresponds to .
- 5.Downloaded from http://realized.oxford-man.ox.ac.uk/data/download.
- 6.We thank an anonymous referee who pointed out the possibility of better comparing the outcomes of the DQ and Christoffersen’s CC and IND tests by allowing the latter to be computed in an extended framework than the standard one described in Section 3.2. As results did not lead to significant improvements, this issue is briefly covered in Appendix E.
© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).