Next Article in Journal / Special Issue
A Deep Learning Model for Forecasting Velocity Structures of the Loop Current System in the Gulf of Mexico
Previous Article in Journal
Improving Hotel Room Demand Forecasts for Vienna across Hotel Classes and Forecast Horizons: Single Models and Combination Techniques Based on Encompassing Tests
Previous Article in Special Issue
Bootstrapped Holt Method with Autoregressive Coefficients Based on Harmony Search Algorithm
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Model-Free Time-Aggregated Predictions for Econometric Datasets

1
Department of Mathematics, University of California San Diego, La Jolla, CA 92093, USA
2
Department of Statistics, University of Florida, Gainesville, FL 32611, USA
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Forecasting 2021, 3(4), 920-933; https://doi.org/10.3390/forecast3040055
Submission received: 3 November 2021 / Revised: 2 December 2021 / Accepted: 6 December 2021 / Published: 8 December 2021
(This article belongs to the Special Issue Feature Papers of Forecasting 2021)

Abstract

:
Forecasting volatility from econometric datasets is a crucial task in finance. To acquire meaningful volatility predictions, various methods were built upon GARCH-type models, but these classical techniques suffer from instability of short and volatile data. Recently, a novel existing normalizing and variance-stabilizing (NoVaS) method for predicting squared log-returns of financial data was proposed. This model-free method has been shown to possess more accurate and stable prediction performance than GARCH-type methods. However, whether this method can sustain this high performance for long-term prediction is still in doubt. In this article, we firstly explore the robustness of the existing NoVaS method for long-term time-aggregated predictions. Then, we develop a more parsimonious variant of the existing method. With systematic justification and extensive data analysis, our new method shows better performance than current NoVaS and standard GARCH(1,1) methods on both short- and long-term time-aggregated predictions. The success of our new method is remarkable since efficient predictions with short and volatile data always carry great importance. Additionally, this article opens potential avenues where one can design a model-free prediction structure to meet specific needs.

1. Introduction

Accurate and robust volatility forecasting is a central focus in financial econometrics. This type of forecasting is crucial for practitioners and traders to make decisions in risk management, asset allocation, pricing of derivative instruments and strategic decisions regarding fiscal policies, etc. Standard methods to perform volatility forecasting are typically built upon applying GARCH-type models to predict squared financial log-returns. With the model-free prediction principle, first proposed by Politis [1], a model-free volatility prediction method—NoVaS—has been proposed recently for efficient forecasting without the assumption of normality. Some previous studies have shown that the NoVaS method possesses better predictive performance than GARCH-type models when forecasting squared log-returns, e.g., Gulay and Emec [2] showed that the NoVaS method could overcome GARCH-type models (GARCH, EGARCH and GJR-GARCH) with generalized error distributions by comparing the pseudo-out-of-sample (POOS) forecasting performance on S&P500 and BIST 100 return series (here the pseudo-out-of-sample forecasting analysis means using data up to and including the current time to predict future values). Chen and Politis [3] showed that the “time-varying” NoVaS method is robust against possible non-stationarities in the data. Furthermore, Chen and Politis [4] extended this NoVaS approach to perform multi-step-ahead predictions of squared log-returns.
However, to the best of our knowledge, such methods have not been evaluated for time-aggregated prediction. Time-aggregated prediction here stands for the prediction of Y n + 1 + + Y n + h after observing { Y t } t = 1 n . Such predictions remain crucial for strategic decisions implemented by commodity or service providers, ([5,6]), trust funds, pension management, insurance companies, portfolio management of specific derivatives ([7]) and assets ([8]). Time-aggregated forecasting is also able to provide some degree of confidence in understanding the general trend in the near future, potentially for the entire following week or months ahead, which is definitely more meaningful than merely understanding what might happen for any single step ahead (predicting Y n + h for one value of h) in the time horizon. In fact, the quality of forecasts for econometric data has been evaluated through such time-aggregated metrics in [9,10]. In this article, we continue utilizing these time-aggregated metrics to challenge the ability of the NoVaS method for short- and long-term time-aggregated predictions on squared log-returns series. For exploring such capabilities of the existing NoVaS method, we set up comprehensive data analyses to substantiate the efficiency of the NoVaS method and also address the lack of data experiments in NoVaS studies. Apart from this, we also attempt to improve the existing one further by proposing a more parsimonious model. Based on extensive data analysis, our new method shows more stable performance than the state-of-the-art NoVaS method regardless of whether simulation or real-world data are used. We also find that the state-of-the-art NoVaS method is even surpassed by the standard GARCH(1,1) model sometimes. On the other hand, our new method returns consistently excellent forecasting. Notably, our method achieves a remarkable improvement when the dataset at hand is short and volatile.
The rest of this article is organized as follows. In Section 2, we firstly introduce the theoretical background and structure of the existing NoVaS method. Then, our new method is proposed and a simple comparison is made to show the stability of our new method. In Section 3, we substantiate our proposal by extensive simulations and data analysis. Moreover, we utilize the CW test to support our parsimonious model. Finally, a summary and discussion are given in Section 4 and Section 5, respectively.

2. Method

2.1. The Existing NoVaS Method

The NoVaS method is a model-free prediction principle. The main idea lies in applying an invertible transformation H, which can map the non- i . i . d . vector { Y i } i = 1 t to a vector { ϵ i } i = 1 t that has i . i . d . components. This leads to the prediction of Y t + 1 by inversely transforming the prediction of ϵ t + 1 [11]. The starting point to build the transformation of the existing NoVaS method is the ARCH model [12]. Then, Politis [1] made some adjustments to determine the final form of H as:
W t = Y t α s t 1 2 + a ˜ 0 Y t 2 + i = 1 p a i Y t i 2 for t = p + 1 , , n .
In Equation (1), { Y t } t = 1 n is the log-returns vector in this article; { W t } t = p + 1 n is the transformed vector, which we hope to transform to i . i . d . ; α is a fixed-scale invariant constant; s t 1 2 is calculated by ( t 1 ) 1 i = 1 t 1 ( Y i μ ) 2 , with μ being the mean of { Y i } i = 1 t 1 ; a ˜ 0 is the coefficient corresponding with the currently observed value Y t 2 . For reaching a qualified transformation function, Equation (2) is required to stabilize the variance.
α ( 0 , 1 ) , a ˜ 0 0 , a i 0 for all i 1 , α + a ˜ 0 + i = 1 p a i = 1
Then, α and a ˜ 0 , a 1 , , a p are finally determined by minimizing | K u r t o s i s ( W t ) 3 | . In practice, the transformed { W t } is usually uncorrelated; see [11] for additional processes for correlated { W t } . This method is model-free in the sense that we do not assume any particular distribution for the innovation { W t } except for matching its kurtosis to 3. Once H is found, H 1 can be obtained immediately. For example, H 1 corresponding with Equation (1) is:
Y t = W t 2 1 a ˜ 0 W t 2 ( α s t 1 2 + i = 1 p a i Y t i 2 ) for t = p + 1 , , n .
To obtain the prediction of Y n + 1 2 , Politis [11] defined two types of optimal predictors under L 1 (Mean Absolute Deviation) and L 2 (Mean Squared Error) criteria after observing historical information set F n = { Y t , 1 t n } :
L 1 - optimal predictor of Y n + 1 2 : Median Y n + 1 , m 2 : m = 1 , , M | F n = Median W n + 1 , m 2 1 a ˜ 0 W n + 1 , m 2 ( α s n 2 + i = 1 p a i Y n + 1 i 2 ) : m = 1 , , M | F n = ( α s n 2 + i = 1 p a i Y n + 1 i 2 ) Median W n + 1 , m 2 1 a ˜ 0 W n + 1 , m 2 : m = 1 , , M L 2 - optimal predictor of Y n + 1 2 : Mean Y n + 1 , m 2 : m = 1 , , M | F n = Mean W n + 1 , m 2 1 a ˜ 0 W n + 1 , m 2 ( α s n 2 + i = 1 p a i Y n + 1 i 2 ) : m = 1 , , M | F n = ( α s n 2 + i = 1 p a i Y n + 1 i 2 ) Mean W n + 1 , m 2 1 a ˜ 0 W n + 1 , m 2 : m = 1 , , M
where { W n + 1 , m } m = 1 M are generated M times from its empirical distribution or a normal distribution. Here, the normal distribution is an asymptotic limit of the empirical distribution of { W n + 1 } . More details about this procedure and multi-step prediction are presented in Section 2.2. { Y n + 1 , m 2 } m = 1 M are given by plugging { W n + 1 , m } m = 1 M into Equation (3) and setting t as n + 1 . During the optimization process, different forms of unknown parameters in Equation (2) are applied so that various NoVaS methods are established. Chen [13] pointed out that the Generalized Exponential NoVaS (GE-NoVaS) method with exponentially decayed unknown parameters presented in Equation (5) is superior to other NoVaS-type methods.
α 0 , a ˜ 0 = c , a i = c e c i for all 1 i p , c = 1 α i = 0 p e c i

2.2. A New Method with Less Parameters

However, during our investigation, we found that the GE-NoVaS method returns extremely large predictions under the L 2 criterion sometimes. The reason for this phenomenon is that the denominator of Equation (3) will be quite small when the generated { W * } (from empirical or normal distribution) is very close to 1 / a ˜ 0 . In this situation, the prediction error will be amplified. Moreover, when the long-term ahead prediction is desired, this amplification will be accumulated and the final prediction will be dampened. Therefore, a removing- a ˜ 0 idea is proposed to avoid such issues in this article. H and H 1 of the GE-NoVaS-without- a ˜ 0 method can be rewritten as below:
W t = Y t α s t 1 2 + i = 1 p a i Y t i 2 ; Y t = W t 2 ( α s t 1 2 + i = 1 p a i Y t i 2 ) ; for t = p + 1 , , n .
We should notice that even without the a ˜ 0 term, the causal prediction rule is still satisfied. It is easy to obtain the analytical form of the first-step-ahead Y n + 1 , which can be expressed as below:
Y n + 1 = W n + 1 2 ( α s n 2 + i = 1 p a i Y n + 1 i 2 )
More specifically, when the first-step GE-NoVaS-without- a ˜ 0 prediction is performed, { W n + 1 * } are generated M (i.e., 5000 in this article) times from a standard normal distribution by the Monte Carlo method or bootstrapped from its empirical distribution F ^ w which is calculated from Equation (1). Then, plugging these { W n + 1 , m * } m = 1 M into Equation (7), M pseudo-predictions { Y ^ n + 1 , m * } m = 1 M are obtained. According to the strategy implied by Equation (4), we choose L 1 and L 2 risk optimal predictors Y ^ n + 1 2 as the sample median and mean of { Y ^ n + 1 , 1 * , , Y ^ n + 1 , M * } , respectively. We can even predict the general form of Y n + h , such as g ( Y n + h ) , by adopting the sample mean or median of { g ( Y ^ n + 1 , 1 * ) , , g ( Y ^ n + 1 , M * ) } . Similarly, the two-steps-ahead Y n + 2 can be expressed as:
Y n + 2 = W n + 2 2 ( α s n + 1 2 + a 1 Y n + 1 2 + i = 2 p a i Y n + 2 i 2 )
When the prediction of Y n + 2 is required, M pairs of { W n + 1 * , W n + 2 * } are still generated by bootstrapping or Monte Carlo method from empirically or standard normal distributions, respectively. Y n + 1 2 is replaced by the predicted value Y ^ n + 1 2 which is derived from running the first-step GE-NoVaS-without- a ˜ 0 prediction with simulated { W n + 1 , m * } m = 1 M under the L 1 or L 2 criterion. Subsequently, we choose L 1 and L 2 risk optimal predictors of Y n + 2 as the sample median and mean of { Y ^ n + 2 , 1 * , , Y ^ n + 2 , M * } .
Finally, iterating the process described above, we can accomplish multi-step-ahead NoVaS predictions. Y n + h , h 3 can be expressed as:
Y n + h = W n + h 2 ( α s n + h 1 2 + i = 1 p a i Y n + h i 2 )
To obtain the prediction of Y n + h , we generate M number of { W n + 1 * , , W n + h * } and plug { Y n + k } k = 1 h 1 with NoVaS predicted values { Y ^ n + k } k = 1 h 1 , which are computed iteratively. L 1 and L 2 risk optimal predictors of Y n + h are computed by the sample median and mean of { Y ^ n + h , 1 * , , Y ^ n + h , M * } . In short, we can summarize that Y n + h is determined by:
Y n + h = f GE - NoVaS - without a ˜ 0 ( W n + 1 , , W n + h , F n )
Since F n is the observed information set, we can simplify the expression of Y n + h as:
Y n + h = f GE - NoVaS - without a ˜ 0 ( W n + 1 , , W n + h )
For applying the GE-NoVaS method, we can still build the relationship between Y n + h and { W n + 1 , , W n + h } as:
Y n + h = f GE - NoVaS ( W n + 1 , , W n + h )
We should notice that simulated { W n + 1 , m * , , W n + h , m * } m = 1 M for obtaining GE-NoVaS method prediction of Y n + h should be generated by the bootstrapping or Monte Carlo method from an empirically or trimmed standard normal distribution. The reason for using the trimmed distribution is | W t | 1 / a ˜ 0 from Equation (1). Here, we summarize Algorithm 1 to perform h-step-ahead time-aggregated prediction using the GE-NoVaS-without- a ˜ 0 method. The algorithm of GE-NoVaS can be written out similarly.
  • Remark (The advantage of removing the a ˜ 0 term): First, after removing the a ˜ 0 term, the prediction of the NoVaS method under the L 2 criterion is more stable. More details will be shown in Section 2.3. Second, the suggestion of removing a ˜ 0 can also lead to less time complexity of our new method. The reason for this phenomenon is simple. If we consider the limiting distribution of { W t } series, 1 / a ˜ 0 is required to be larger than or equal to 3 to ensure that { W t } has a sufficiently large range, i.e., a ˜ 0 is required to be less than or equal to 0.111 (recall that the mass of standard normal data is within [ 3 , 3 ] ). However, the optimal combination of NoVaS coefficients may not render a suitable a ˜ 0 . For this situation, we need to increase the NoVaS transformation order p and repeat the normalizing and variance-stabilizing process till a ˜ 0 in the optimal combination of coefficients is suitable. This repeating process definitely increases the computation workload.
Algorithm 1: The h-step ahead prediction for the GE-NoVaS-without- a ˜ 0 method.
   Step 1 Define a grid of possible α values, { α k ; k = 1 , , K } . Fix α = α k , then calculate the optimal combination of α k , a 1 , , a p of the GE-NoVaS-without- a ˜ 0 method, which minimizes | K u r t o s i s ( W t ) 3 | .
   Step 2 Derive the analytic form of Equation (11) using α k , a 1 , , a p from the first step.
   Step 3 Generate { W n + 1 * , , W n + h * } M times from a standard normal distribution or the empirical distribution F ^ w . Plug { W n + 1 * , , W n + h * } into the analytic form of Equation (11) to obtain M pseudo-values { Y ^ n + h , 1 * , , Y ^ n + h , M * } .
   Step 4 Calculate the optimal predictor of g ( Y n + h ) by taking the sample mean (under L 2 risk criterion) or sample median (under L 1 risk criterion) of the set { g ( Y ^ n + h , 1 * ) , , g ( Y ^ n + h , M * ) } .
   Step 5 Repeat above steps with different α values from { α k ; k = 1 , , K } to get K prediction results.

2.3. The Potential Instability of the GE-NoVaS Method

Next, we provide an illustration to compare the GE-NoVaS and GE-NoVaS-without- a ˜ 0 methods in predicting the volatility of the Microsoft Corporation (MSFT) daily closing price from 8 January 1998 to 31 December 1999 and show an interesting finding that the long-term time-aggregated predictions of the GE-NoVaS method are unstable under the L 2 criterion. Based on the finding of Awartani and Corradi [14], squared log-returns can be used as a proxy for volatility to render a correct ranking of different GARCH models in terms of a quadratic loss function. Log-return series { Y t } can be computed by the equation shown below:
Y t = 100 × l o g ( X t + 1 / X t )
where { X t } is the corresponding MSFT daily closing price series. For achieving a comprehensive comparison, we use 250 financial log-returns as a sliding window to perform POOS 1-step, 5-step and 30-step (long-term) ahead time-aggregated predictions under the L 2 criterion. Then, we roll this window through the whole dataset, i.e., we use { Y 1 , , Y 250 } to predict Y 251 2 , { Y 251 2 , , Y 255 2 } and { Y 251 2 , , Y 280 2 } ; then, we use { Y 2 , , Y 251 } to predict Y 252 2 , { Y 252 2 , , Y 256 2 } and { Y 252 2 , , Y 281 2 } , for 1-step, 5-step and 30-step aggregated predictions, respectively, and so on. We can define all 1-step, 5-step and 30-step-ahead time-aggregated predictions as { Y ^ k , 1 2 } , { Y ^ i , 5 2 } and { Y ^ j , 30 2 } , which are presented as below:
Assume that there are a total of N log - return data points : Y ^ k , 1 2 = Y ^ k + 1 2 , k = 250 , 251 , , N 1 Y ^ i , 5 2 = m = 1 5 Y ^ i + m 2 , i = 250 , 251 , , N 5 Y ^ j , 30 2 = m = 1 30 Y ^ j + m 2 , j = 250 , 251 , , N 30
In Equation (14), Y ^ k + 1 2 , Y ^ i + m 2 , Y ^ j + m 2 are single-step predictions of squared log-returns by the two NoVaS-type methods. To obtain the “Prediction Errors” for the two methods, we can calculate the “loss” by comparing the aggregated prediction results with the realized aggregated values based on Equation (15):
L p , h = p ( Y ^ p , h 2 m = 1 h ( Y p + m 2 ) ) 2 , p { k , i , j } ; h { 1 , 5 , 30 }
where { Y p + m 2 } are realized squared log-returns. To show the potential instability of the GE-NoVaS method under the L 2 criterion, we take α to be 0.5 to build a toy example. In the algorithm when performing the GE-NoVaS method, α could take an optimal value from a discrete set { 0.1 , , 0.8 } based on the prediction performance.
From Figure 1, we can clearly see that the GE-NoVaS-without- a ˜ 0 method can better capture different steps’ true time-aggregated features. On the other hand, the GE-NoVaS method returns unstable results for 30-step-ahead time-aggregated predictions. Besides, we can see that the 1-step-ahead POOS prediction returned by the GE-NoVaS method is almost a flat curve, which is actually meaningless. Similarly, for the 5-step-ahead time-aggregated prediction case, the POOS prediction of the GE-NoVaS method fails to match the true time-aggregated values.

3. Data Analysis and Results

To perform extensive data analysis in a bid to validate our method, we deploy POOS predictions using two NoVaS and standard GARCH(1,1) methods with simulated and real-world data. All results are collated in Table 1. The optimal results for each data cases are highlighted in bold. For controlling the dependence of the prediction performance on the length of the dataset, we build datasets with two fixed lengths—250 or 500—to mimic 1-year or 2-year data, respectively. At the same time, we choose the window size for our rollover forecasting analysis to be 100 or 250 for the 1-year or 2-year datasets.

3.1. Simulation Study

We use the same simulation Models 1–4 from [4], shown below, to mimic four 1-year datasets. Recall that one NoVaS method can generate the L 1 or L 2 predictor and { W * } can be chosen from a normal distribution or empirical distribution; thus, there are four variants of one specific NoVaS method. We take the best-performing result among four variants of a specific NoVaS method to be its final prediction. Finally, we continue applying the formula in Equation (15) to measure the performance of the different methods, as described in Section 2.3.
  • Model 1: Time-varying GARCH(1,1) with Gaussian errors
    X t = σ t ϵ t , σ t 2 = 0.00001 + β 1 , t σ t 1 2 + α 1 , t X t 1 2 , { ϵ t } i . i . d . N ( 0 , 1 )
    α 1 , t = 0.1 0.05 t / n ; β 1 , t = 0.73 + 0.2 t / n , n = 250
  • Model 2: Standard GARCH(1,1) with Gaussian errors
    X t = σ t ϵ t , σ t 2 = 0.00001 + 0.73 σ t 1 2 + 0.1 X t 1 2 , { ϵ t } i . i . d . N ( 0 , 1 )
  • Model 3: (Another) Standard GARCH(1,1) with Gaussian errors
    X t = σ t ϵ t , σ t 2 = 0.00001 + 0.8895 σ t 1 2 + 0.1 X t 1 2 , { ϵ t } i . i . d . N ( 0 , 1 )
  • Model 4: Standard GARCH(1,1) with Student-t errors
    X t = σ t ϵ t , σ t 2 = 0.00001 + 0.73 σ t 1 2 + 0.1 X t 1 2 ,
    { ϵ t } i . i . d . t distribution with five degrees of freedom
Result analysis: From the first block of Table 1, we can read that both NoVaS methods are superior to the GARCH(1,1) model. Although these simulated datasets are generated from GARCH(1,1)-type models, the GE-NoVaS-without- a ˜ 0 method can bring around 66% and 48% improvements compared to the GARCH(1,1) model for 5-step-ahead time-aggregated predictions of Model-4 and Model-1 data, respectively. Notably, GARCH(1,1) brings poor results for the 30-step-ahead time-aggregated predictions of Model-4 simulated data, which implies that such a classical method is impaired by error accumulation problems when long-term predictions are required. On the other hand, the model-free NoVaS method can avoid this issue. Taking a closer look at these results, we can observe that almost all optimal results come from applying the GE-NoVaS-without- a ˜ 0 method. Moreover, the GE-NoVaS method is surpassed by GARCH(1,1) when forecasting 30-step-ahead time-aggregated Model-2 data. On the other hand, the GE-NoVaS-without- a ˜ 0 method provides consistently stable results. These results imply that the GE-NoVaS-without- a ˜ 0 method dominates the GE-NoVaS method when predicting long-term or short-term time-aggregated predictions. Besides, using the same generated models from the previous study of the NoVaS method [4] ensures fairness. Additionally, with simulation implementations, the ability against model misspecification of NoVaS methods is verified in Appendix A.

3.2. A Few Real Datasets

We also present a variety of real-world datasets of different size and intrinsic behavior:
  • 2-year period data: 2018∼2019 stock price data.
  • 1-year period data: 2019 stock price and index data.
  • 1-year period volatile data due to pandemic: 11.2019∼10.2020 stock price, currency and index data.
Taking into account three types of real-world data is necessary to challenge our new method and explore the existing method in different regimes. We also tactically pay more attention to short and volatile data since this is a more challenging task to handle. Equation (13) is continually used to obtain the log-return series of different datasets.
Before comparing in depth the forecasting performance of the NoVaS-type and GARCH methods, we first investigate the properties of the used datasets. From Figure 2, we can see that there were huge variations in the four datasets during 11.2019∼10.2020, which implies the extreme fluctuations in global economics due to the COVID-19 pandemic. We wished to apply such datasets to test whether the NoVaS-type methods can achieve good forecasting performance for such volatile data.
Besides, it is natural to question whether these datasets are stationary. In a comprehensive manner, we choose three statistical tests—Augmented Dickey–Fuller (ADF) Test [15], Phillips–Perron (PP) Unit Root Test [16] and Kwiatkowski–Phillips–Schmidt–Shin (KPSS) Test [17]—to check the stationarity of the squared log-returns series of each selected dataset. One aspect that should be noticed is that the number of lags is crucial for the ADF test. If the included lag is too small, then the remaining serial correlation in the errors will bias the test. If this number is larger, the power of the test will suffer. Here, we consider taking the longest lag that is statistically significant. More specifically, we determine this longest lag by observing the last lag that crosses through the confidence interval lines of the autocorrelation plot. Besides, we apply a long version of the truncation lag parameter on both the PP and KPSS tests. The results of the three tests are tabulated in Table A4. Combining these results, we can argue that most of the squared log-return series in the normal time period are stationary. However, during the volatile time period, the squared log-returns of IBM, SP500 and Dow Jones are thought to be non-stationary by the ADF test. The KPSS test also returns small p-values for these three datasets. These results are consistent with our conjecture that data tend to show non-stationarity during volatile periods. Again, it will be interesting to see if the NoVaS-type methods can offer good forecasting performance for non-stationary data. Recall that Chen and Politis [3] found that the NoVaS methodology generally outperforms the GARCH benchmark on the one-step-ahead point prediction of non-stationary data (involving local stationarity and/or structural breaks). However, they only considered two real-world time series. Here, we extend such empirical study to short- and long-term time-aggregated predictions with sufficient data examples.
  • Remark (One ARCH-type model for non-stationary data): Since our stationarity tests suggest that some series may not be stationary, we can consider applying ARCH-without-intercept, which is a variant of the ARCH model. This variant is non-stationary but stable in the sense that the observed process has non-degenerated distribution. Moreover, it appears to be an alternative to common stationary but highly persistent GARCH models [18]. Inspired by this ARCH-type model, the NoVaS method may be further improved by removing the corresponding intercept term α s t 1 2 in Equations (1) and (6). More empirical experiments could be conducted along this direction.
  • Result analysis: From the last three blocks of Table 1, there is no optimal result that comes from the GARCH(1,1) method. When the target data are short and volatile, GARCH(1,1) gives poor results for 30-step-ahead time-aggregated predictions, such as the volatile Djones, CADJPY and IBM cases. Among the two NoVaS methods, the GE-NoVaS-without- a ˜ 0 method outperforms the GE-NoVaS method for the three types of real-world data. More specifically, around 70% and 30% improvements are created by our new method compared to the existing GE-NoVaS method when forecasting 30-step-ahead time-aggregated volatile Djones and CADJPY data, respectively. We should also notice that the GE-NoVaS method is again surpassed by the GARCH(1,1) model on 30-step-ahead aggregated predictions of 2018∼2019 BAC data. On the other hand, the GE-NoVaS-without- a ˜ 0 method performs stably. These comprehensive prediction comparisons cover the shortage of empirical analyses of NoVaS methods, and imply that NoVaS-type methods are indeed valid and efficient for real-world short- or long-term predictions of three main types of econometric data. See Appendix A for more results.

3.3. Statistical Significance

However, one may suggest that the victory of our new methods is only specific to these samples. Therefore, we challenge this superiority by testing the statistical significance. Noting that the GE-NoVaS-without- a ˜ 0 method is a nested method (taking a ˜ 0 = 0 in the larger model) compared with the GE-NoVaS method, we deploy the CW test [19] to ensure that the removing- a ˜ 0 idea is also statistically reasonable; see the p-value column in Table 1 for the tests’ results. The reason for not performing CW tests on the simulation cases is that the prediction performance of each simulation is the average value of 5 replications. These CW test results imply that the null hypothesis should not be rejected for almost all cases under a 5% level of significance, which confirms the equivalence of the new method to the existing one.

4. Summary

In previous studies of NoVaS methods, only a few real-word data analyses were performed [2,3,4]. Here, we provide extensive data analyses to address the lack of real-world data experiments. Our results are consistent with previous findings and substantiate the effectiveness of the NoVaS method again, i.e., the NoVaS method is more efficient and stable than the classical GARCH method for short-term predictions. Further, we reveal the ability of NoVaS-type methods to perform long-term time-aggregated forecasting. Beyond this, we propose a new NoVaS method that outperforms the state-of-the-art GE-NoVaS method. Our findings in this article are summarized as follows:
  • Existing GE-NoVaS and new GE-NoVaS-without- a ˜ 0 methods provide substantial improvements for time-aggregated prediction, which hints towards the stability of NoVaS-type methods for providing long-horizon inferences.
  • Our new method has superior performance to the GE-NoVaS method, especially for shorter sample sizes or more volatile data. This is significant given that GARCH-type models are difficult to estimate in shorter samples.
  • We provide a statistical hypothesis test that shows that our model provides a more parsimonious fit, especially for long-term time-aggregated predictions.

5. Discussion

In this article, we explored the GE-NoVaS method toward short and long time-aggregated predictions and proposed a new variant that is based on a parsimonious model, has better empirical performance and yet is statistically reasonable. Although our new method is in a parsimonious form, it still obeys the autoregressive prediction rule and it is more stable for performing predictions under L 2 risk criterion than current the GE-NoVaS method. We should note that the unknown coefficients of both the GE-NoVaS ( a ˜ 0 , a 1 , , a p ) and GE-NoVaS-without- a ˜ 0 ( a 1 , , a p ) methods are in exponential form, which implies that the correlations within series data are decreasing in exponential speed with the increasing time order. However, this specific form is not suitable for use for predicting all datasets. In other words, we anticipate performing NoVaS prediction without fixing the unknown coefficients in an invariant form to satisfy the variety of real-world econometric datasets. Therefore, building a NoVaS method with a more arbitrary coefficient form can be a future research direction. In addition, we should also note that there is a high demand to perform efficient forecasting for integer time series data. For example, a relevant topic regarding such integer-value prediction is forecasting COVID-19 cases. It will be beneficial to develop a variant of NoVaS for integer-value data. Moreover, in the financial market, the stock data move together. Thus, it would be exciting to see if one can perform model-free predictions in a multiple time series scenario. We hope that this article will open up avenues where one can explore other specific transformation structures to improve the existing forecasting frameworks and aid in specific tasks.
From a statistical inference point of view, one can also construct prediction intervals for these predictions using bootstrap. Such prediction intervals are well sought after in the econometrics literature and some results on the asymptotic validity of these can be provided. Additionally, we can also explore dividing the dataset into test and training in some optimal way and see if this can improve the performance of these methods.
In addition, there are some model-free methods based on machine learning to perform prediction tasks. These modern techniques enjoy high accuracy, but are time-consuming and lack of statistical inference. On the other hand, our new method and existing NoVaS methods are time-efficient and outperform classical GARCH-type methods significantly. More importantly, NoVaS-type methods can provide concrete statistical inference. Thus, it will be interesting to challenge NoVaS-type methods’ forecasting accuracy with machine-learning-based methods.

Author Contributions

Data curation, K.W. and S.K.; Formal analysis, S.K.; Investigation, K.W. and S.K.; Methodology, S.K.; Software, K.W. and S.K.; Visualization, K.W.; Writing—original draft, K.W.; Writing—review and editing, S.K. All authors have read and agreed to the published version of the manuscript.

Funding

The second author’s research is partially funded by NSF-DMS 2124222.

Data Availability Statement

We collected all data presented here from www.investing.com (accessed on 3 November 2021) manually. Then, we transformed the closing price data into financial log-returns based on Equation (13).

Acknowledgments

We are thankful to the two anonymous referees who helped us to improve the paper significantly. The first author is thankful to Professor Politis for the introduction to the topic and useful discussions.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A. Additional Simulation Study and Data Analysis Results

Appendix A.1. Additional Simulation Study: Model Misspecification

In the real world, it is difficult to convincingly state whether the data obey one particular type of GARCH model, so we wish to provide four more GARCH-type models to simulate one-year datasets to see if our methods are satisfactory regardless of the underlying distribution and GARCH-type model. The simulation study results are presented in Table A1, which implies that the NoVaS-type methods are more robust against model misspecification and GE-NoVaS-without- a ˜ 0 is the best method.
  • Model 5: Another time-varying GARCH(1,1) with Gaussian errors
    X t = σ t ϵ t , σ t 2 = ω 0 , t + β 1 , t σ t 1 2 + α 1 , t X t 1 2 , { ϵ t } i . i . d . N ( 0 , 1 )
    g t = t / n ; ω 0 , t = 4 s i n ( 0.5 π g t ) + 5 ; α 1 , t = 1 ( g t 0.3 ) 2 + 0.5 ; β 1 , t = 0.2 s i n ( 0.5 π g t ) + 0.2 , n = 250
  • Model 6: Exponential GARCH(1,1) with Gaussian errors
    X t = σ t ϵ t , log ( σ t 2 ) = 0.00001 + 0.8895 log ( σ t 1 2 ) + 0.1 ϵ t 1 + 0.3 ( | ϵ t 1 | E | ϵ t 1 | ) ,
    { ϵ t } i . i . d . N ( 0 , 1 )
  • Model 7: GJR-GARCH(1,1) with Gaussian errors
    X t = σ t ϵ t , σ t 2 = 0.00001 + 0.5 σ t 1 2 + 0.5 X t 1 2 0.5 I t 1 X t 1 2 , { ϵ t } i . i . d . N ( 0 , 1 ) I t = 1 if X t 0 ; I t = 0 otherwise
  • Model 8: Another GJR-GARCH(1,1) with Gaussian errors
    X t = σ t ϵ t , σ t 2 = 0.00001 + 0.73 σ t 1 2 + 0.1 X t 1 2 + 0.3 I t 1 X t 1 2 , { ϵ t } i . i . d . N ( 0 , 1 ) I t = 1 if X t 0 ; I t = 0 otherwise
Table A1. Comparisons of different methods’ forecasting performance on simulated 1-year data.
Table A1. Comparisons of different methods’ forecasting performance on simulated 1-year data.
GE-NoVaSGE-NoVaS-without- a ˜ 0 GARCH(1,1)
M5-1step0.915380.831681.00000
M5-5steps0.491690.437721.00000
M5-30steps0.250090.226591.00000
M6-1step0.959390.946611.00000
M6-5steps0.935940.847191.00000
M6-30steps0.844010.703011.00000
M7-1step0.848130.735531.00000
M7-5steps0.508490.466181.00000
M7-30steps0.068320.064791.00000
M8-1step0.795610.765861.00000
M8-5steps0.480280.381071.00000
M8-30steps0.009770.009181.00000

Appendix A.2. Additional Data Analysis: 1-Year Datasets

To make our data analysis more comprehensive, we present more results of predictions on 1-year real-world datasets in Table A2. One interesting finding is that the GE-NoVaS method is significantly overcome by using the GARCH(1,1) model for some cases, such as the BAC, TSLA and Smallcap datasets. The GE-NoVaS-without- a ˜ 0 method still maintains great forecasting performance.
Table A2. Comparisons of different methods’ forecasting performance on real-world 1-year data.
Table A2. Comparisons of different methods’ forecasting performance on real-world 1-year data.
GE-NoVaSGE-NoVaS-without- a ˜ 0 GARCH(1,1)
2019-MCD-1step0.959590.931411.00000
2019-MCD-5steps1.007230.900611.00000
2019-MCD-30steps1.052390.808051.00000
2019-BAC-1step1.042720.977571.00000
2019-BAC-5steps1.227610.895711.00000
2019-BAC-30steps1.450201.011751.00000
2019-MSFT-1step1.033080.984691.00000
2019-MSFT-5steps1.223401.023871.00000
2019-MSFT-30steps1.230200.975851.00000
2019-TSLA-1step1.004280.986461.00000
2019-TSLA-5steps1.066100.975231.00000
2019-TSLA-30steps2.006230.871581.00000
2019-Bitcoin-1step0.899290.867951.00000
2019-Bitcoin-5steps0.623120.556201.00000
2019-Bitcoin-30steps0.007330.006241.00000
2019-Nasdaq-1step0.999600.935581.00000
2019-Nasdaq-5steps1.152820.844591.00000
2019-Nasdaq-30steps0.689940.589241.00000
2019-NYSE-1step0.924860.904071.00000
2019-NYSE-5steps0.862490.698221.00000
2019-NYSE-30steps0.221220.181731.00000
2019-Smallcap-1step1.020410.987311.00000
2019-Smallcap-5steps1.158680.877001.00000
2019-Samllcap-30steps1.304670.888251.00000
2019-BSE-1step0.706670.676941.00000
2019-BSE-5steps0.256750.236651.00000
2019-BSE-30steps0.037640.028901.00000
2019-BIST-1step0.968070.954671.00000
2019-BIST-5steps0.989440.828981.00000
2019-BIST-30steps2.219960.885111.00000

Appendix A.3. Additional Data Analysis: Volatile 1-Year Datasets

Similarly, we consider more volatile 1-year datasets. All prediction results are tabulated in Table A3. It is clear that both NoVaS-type methods still outperform the GARCH(1,1) model for short- and long-term time-aggregated forecasting. Although the GE-NoVaS method yields optimal performance in some cases, we should note that the GE-NoVaS-without- a ˜ 0 method still gives almost the same but slightly worse results. Interestingly, the GE-NoVaS-without- a ˜ 0 method can introduce a significant improvement compared with the GE-NoVaS method for 30-step-ahead predictions. This again hints towards the superior robustness of our new method specifically for long-term aggregated predictions.
Table A3. Comparisons of different methods’ forecasting performance on volatile 1-year data.
Table A3. Comparisons of different methods’ forecasting performance on volatile 1-year data.
GE-NoVaSGE-NoVaS-without- a ˜ 0 GARCH(1,1)
11.2019∼10.2020-MCD-1step0.517550.580181.00000
11.2019∼10.2020-MCD-5steps0.107250.178871.00000
11.2019∼10.2020-MCD-30steps3.32 × 10 5 7.48 × 10 6 1.00000
11.2019∼10.2020-AMZN-1step0.970990.902001.00000
11.2019∼10.2020-AMZN-5steps0.887050.717891.00000
11.2019∼10.2020-AMZN-30steps0.581240.534601.00000
11.2019∼10.2020-SBUX-1step0.682060.699431.00000
11.2019∼10.2020-SBUX-5steps0.242550.305281.00000
11.2019∼10.2020-SBUX-30steps0.004990.002891.00000
11.2019∼10.2020-MSFT-1step0.801330.845021.00000
11.2019∼10.2020-MSFT-5steps0.355670.375281.00000
11.2019∼10.2020-MSFT-30steps0.013420.007321.00000
11.2019∼10.2020-EURJPY-1step0.950930.942061.00000
11.2019∼10.2020-EURJPY-5steps0.761820.767271.00000
11.2019∼10.2020-EURJPY-30steps0.162020.153501.00000
11.2019∼10.2020-CNYJPY-1step0.778120.798771.00000
11.2019∼10.2020-CNYJPY-5steps0.388750.405691.00000
11.2019∼10.2020-CNYJPY-30steps0.083980.062701.00000
11.2019∼10.2020-Smallcap-1step0.581700.609311.00000
11.2019∼10.2020-Smallcap-5steps0.102700.103371.00000
11.2019∼10.2020-Smallcap-30steps7.00 × 10 5 5.96 × 10 5 1.00000
11.2019∼10.2020-BSE-1step0.394930.397451.00000
11.2019∼10.2020-BSE-5steps0.033200.041091.00000
11.2019∼10.2020-BSE-30steps2.45 × 10 5 1.82 × 10 5 1.00000
11.2019∼10.2020-NYSE-1step0.557410.571741.00000
11.2019∼10.2020-NYSE-5steps0.089940.101821.00000
11.2019∼10.2020-NYSE-30steps1.36 × 10 5 6.64 × 10 6 1.00000
11.2019∼10.2020-USDXfuture-1step1.146210.996401.00000
11.2019∼10.2020-USDXfuture-5steps0.610750.548341.00000
11.2019∼10.2020-USDXfuture-30steps0.107230.102781.00000
11.2019∼10.2020-Nasdaq-1step0.713800.753501.00000
11.2019∼10.2020-Nasdaq-5steps0.293320.335191.00000
11.2019∼10.2020-Nasdaq-30steps0.012230.005991.00000
11.2019∼10.2020-Bovespa-1step0.600310.575581.00000
11.2019∼10.2020-Bovespa-5steps0.086030.074471.00000
11.2019∼10.2020-Bovespa-30steps6.87 × 10 6 2.04 × 10 6 1.00000

Appendix B. Stationarity Test Results of Some Real-World Datasets

Table A4. p-values of three stationarity tests.
Table A4. p-values of three stationarity tests.
ADFKPSSPP
2018∼2019 MCD0.010.100.01
2018∼2019 BAC0.010.100.01
2019 AAPL0.010.100.01
2019 Djones0.100.100.01
2019 SP5000.180.100.01
11.2019∼10.2020 IBM0.310.050.01
11.2019∼10.2020 CADJPY0.010.100.01
11.2019∼10.2020 SP5000.230.080.01
11.2019∼10.2020 Djones0.220.080.01
Note: The null hypothesis of the ADF and PP tests is that the tested series is non-stationary. Therefore, if the ADF and PP tests are rejected, it means that this tested series is stationary. On the other hand, the null hypothesis of KPSS is that the series is stationary.

References

  1. Politis, D.N. A Normalizing and Variance-Stabilizing Transformation for Financial Time Series; Elsevier Inc.: Amsterdam, The Netherlands, 2003. [Google Scholar]
  2. Gulay, E.; Emec, H. Comparison of forecasting performances: Does normalization and variance stabilization method beat GARCH (1, 1)-type models? Empirical evidence from the stock markets. J. Forecast. 2018, 37, 133–150. [Google Scholar] [CrossRef]
  3. Chen, J.; Politis, D.N. Time-varying NoVaS Versus GARCH: Point Prediction, Volatility Estimation and Prediction Intervals. J. Time Ser. Econom. 2020, 1. [Google Scholar] [CrossRef]
  4. Chen, J.; Politis, D.N. Optimal multi-step-ahead prediction of ARCH/GARCH models and NoVaS transformation. Econometrics 2019, 7, 34. [Google Scholar] [CrossRef] [Green Version]
  5. Chudỳ, M.; Karmakar, S.; Wu, W.B. Long-term prediction intervals of economic time series. Empir. Econ. 2020, 58, 191–222. [Google Scholar] [CrossRef] [Green Version]
  6. Karmakar, S.; Chudy, M.; Wu, W.B. Long-term prediction intervals with many covariates. arXiv 2020, arXiv:2012.08223. [Google Scholar] [CrossRef]
  7. Kitsul, Y.; Wright, J.H. The economics of options-implied inflation probability density functions. J. Financ. Econ. 2013, 110, 696–711. [Google Scholar] [CrossRef] [Green Version]
  8. Bansal, R.; Kiku, D.; Yaron, A. Risks for the long run: Estimation with time aggregation. J. Monet. Econ. 2016, 82, 52–69. [Google Scholar] [CrossRef] [Green Version]
  9. Starica, C. Is GARCH (1, 1) as Good a Model as the Accolades of the Nobel Prize Would Imply? University Library of Munich: Munich, Germany, 2003; SSRN 637322. [Google Scholar]
  10. Fryzlewicz, P.; Sapatinas, T.; Rao, S.S. Normalized least-squares estimation in time-varying ARCH models. Ann. Stat. 2008, 36, 742–786. [Google Scholar] [CrossRef]
  11. Politis, D.N. The Model-Free Prediction Principle. In Model-Free Prediction and Regression; Springer: Berlin/Heidelberg, Germany, 2015; pp. 13–30. [Google Scholar]
  12. Engle, R.F. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation. Econom. J. Econom. Soc. 1982, 50, 987–1007. [Google Scholar] [CrossRef]
  13. Chen, J. Prediction in Time Series Models and Model-free Inference with a Specialization in Financial Return Data. Ph.D. Thesis, University of California, San Diego, CA, USA, 2018. [Google Scholar]
  14. Awartani, B.M.; Corradi, V. Predicting the volatility of the S&P-500 stock index via GARCH models: The role of asymmetries. Int. J. Forecast. 2005, 21, 167–183. [Google Scholar]
  15. Said, S.E.; Dickey, D.A. Testing for unit roots in autoregressive-moving average models of unknown order. Biometrika 1984, 71, 599–607. [Google Scholar] [CrossRef]
  16. Perron, P. Trends and random walks in macroeconomic time series: Further evidence from a new approach. J. Econ. Dyn. Control 1988, 12, 297–332. [Google Scholar] [CrossRef]
  17. Kwiatkowski, D.; Phillips, P.C.; Schmidt, P.; Shin, Y. Testing the null hypothesis of stationarity against the alternative of a unit root: How sure are we that economic time series have a unit root? J. Econom. 1992, 54, 159–178. [Google Scholar] [CrossRef]
  18. Hafner, C.M.; Preminger, A. An ARCH model without intercept. Econ. Lett. 2015, 129, 13–17. [Google Scholar] [CrossRef]
  19. Clark, T.E.; West, K.D. Approximately normal tests for equal predictive accuracy in nested models. J. Econom. 2007, 138, 291–311. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Curves of the true and predicted time-aggregated squared log-returns from GE-NoVaS and GE-NoVaS-without- a ˜ 0 methods.
Figure 1. Curves of the true and predicted time-aggregated squared log-returns from GE-NoVaS and GE-NoVaS-without- a ˜ 0 methods.
Forecasting 03 00055 g001
Figure 2. Price series of selected 9 datasets.
Figure 2. Price series of selected 9 datasets.
Forecasting 03 00055 g002
Table 1. Comparisons of different methods’ forecasting performance.
Table 1. Comparisons of different methods’ forecasting performance.
GE-NoVaSGE-NoVaS-without- a ˜ 0 GARCH(1,1)p-Value(CW Test)
Simulated-1-year-dataModel-1-1step0.913690.887811.00000
Model-1-5steps0.610010.528721.00000
Model-1-30steps0.772500.736041.00000
Model-2-1step0.977960.946351.00000
Model-2-5steps0.981270.963611.00000
Model-2-30steps1.383530.988721.00000
Model-3-1step0.991830.928291.00000
Model-3-5steps0.770880.674821.00000
Model-3-30steps0.796720.710031.00000
Model-4-1step0.836310.780871.00000
Model-4-5steps0.382960.343961.00000
Model-4-30steps0.001990.002011.00000
2-years-data2018∼2019-MCD-1step0.996310.996141.000000.00053
2018∼2019-MCD-5steps0.954030.921201.000000.03386
2018∼2019-MCD-30steps0.757300.626181.000000.19691
2018∼2019-BAC-1step0.983930.979661.000000.09568
2018∼2019-BAC-5steps0.988850.951241.000000.07437
2018∼2019-BAC-30steps1.141110.874141.000000.03643
1-year-data2019-AAPL-1step0.845330.809481.000000.25096
2019-AAPL-5steps0.854010.681911.000000.06387
2019-AAPL-30steps0.990430.738231.000000.17726
2019-Djones-1step0.967520.963651.000000.34514
2019-Djones-5steps0.987250.895421.000000.24529
2019-Djones-30steps0.863330.803041.000000.23766
2019-SP500-1step0.969780.921831.000000.45693
2019-SP500-5steps0.967040.755791.000000.24402
2019-SP500-30steps0.343890.297961.000000.08148
Volatile-1-year-data11.2019∼10.2020-IBM-1step0.802220.807441.000000.16568
11.2019∼10.2020-IBM-5steps0.389330.407431.000000.03664
11.2019∼10.2020-IBM-30steps0.011430.009181.000000.15364
11.2019∼10.2020-CADJPY-1step0.469400.487121.000000.16230
11.2019∼10.2020-CADJPY-5steps0.116780.135491.000000.06828
11.2019∼10.2020-CADJPY-30steps0.005840.003941.000000.15174
11.2019∼10.2020-SP500-1step0.972940.923491.000000.05536
11.2019∼10.2020-SP500-5steps0.965900.751831.000000.17380
11.2019∼10.2020-SP500-30steps0.343570.297931.000000.16022
11.2019∼10.2020-Djones-1step0.563570.575501.000000.11099
11.2019∼10.2020-Djones-5steps0.098100.115541.000000.45057
11.2019∼10.2020-Djones-30steps4.32 × 10 5 1.24 × 10 5 1.000000.68487
Note: The values presented in the GE-NoVaS and GE-NoVaS-without- a ˜ 0 columns reflect the relative performance compared with the ‘standard’ GARCH(1,1) method. The null hypothesis of the CW test is that parsimonious and larger models have equal mean squared prediction error (MSPE). The alternative is that the larger model has a smaller MSPE.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Wu, K.; Karmakar, S. Model-Free Time-Aggregated Predictions for Econometric Datasets. Forecasting 2021, 3, 920-933. https://doi.org/10.3390/forecast3040055

AMA Style

Wu K, Karmakar S. Model-Free Time-Aggregated Predictions for Econometric Datasets. Forecasting. 2021; 3(4):920-933. https://doi.org/10.3390/forecast3040055

Chicago/Turabian Style

Wu, Kejin, and Sayar Karmakar. 2021. "Model-Free Time-Aggregated Predictions for Econometric Datasets" Forecasting 3, no. 4: 920-933. https://doi.org/10.3390/forecast3040055

Article Metrics

Back to TopTop