A GARCH Model with Artificial Neural Networks

Abstract

In this paper, we incorporate a GARCH model into an artificial neural network (ANN) for financial volatility modeling and estimate the parameters in Tensorflow. Our goal was to better predict stock volatility. We evaluate the performance of the models using the mean absolute errors of powers of the out-of-sample returns between 2 March 2018 and 28 February 2020. Our results show that our modeling procedure with an ANN can outperform the standard GARCH(1,1) model with standardized Student’s t distribution. Our variable importance analysis shows that Net Debt/EBITA is among the six most important predictor variables in all of the neural network models we have examined. The main contribution of this paper is that we propose a Long Short-Term Memory (LSTM) model with a GARCH framework because LSTM can systematically take into consideration potential nonlinearity in volatility structure at different time points. One of the advantages of our research is that the proposed models are easy to implement because our proposed models can be run in Tensorflow, a Python package that enables fast and automatic optimization. Another advantage is that the proposed models enable variable importance analysis.

1. Introduction

GARCH models are popular in finance and economic literature. Bollerslev [1] introduced a GARCH(1,1) model in which the residual term follows a standardized Student’s t distribution. Nikolaev et al. [2] suggested that the GARCH model takes into account the dynamics of time-dependent variance and therefore incorporated a GARCH model into non-linear neural networks. So and Yu [3] compared multiple GARCH models on their performance in value at risk estimation. So and Wong [4] developed expected shortfall and median shortfall estimation under GARCH models. Chen and So [5] extended the GARCH model to have threshold nonlinearity. Chen et al. [6] studied Markov switching GARCH models for volatility forecasting. So and Xu [7] used GARCH models to capture more intraday information. Efimova and Serletis [8] modeled oil, natural gas, and electricity markets using a multivariate GARCH model because it allows for rich dynamics in the variance–covariance structure. Their finding is higher precision in key estimates achieved by using a less computationally demanding model. Kristjanpoller and Minutolo [9] extended their model by applying an artificial neural network (ANN) to the GARCH model. Zhipeng and Shenghong [10] extended the BEKK–GARCH model by applying the Markov regime switching method that incorporates state transformation. Escobar-Anel et al. [11] introduced a class of GARCH models. The daily return at time t can be modeled as follows:

$$\begin{array}{cc}\hfill & R_{t+1}=r+\lambda \left(w_t\right)h_{t+1}+{ϵ}_{t+1},\hfill \\ \hfill & h_{t+1}=g(h_t,w_t,{ϵ}_t),\hfill \end{array}$$

where $R_{t+1}$ is a single period return. $w_t$ introduces flexibility so that non-Normal models can be generated. The variable $\lambda \left(w_t\right)$ measures the impact of market risk from $w_t$ and r is a risk-free rate. $g(.)$ is a non-negative function as that the variances in the model are non-negative. The class of GARCH models can incorporate important empirical characteristics of financial asset returns and derivatives, such as jumps, and have a recursive closed form for a conditional moment generating function. They noted that the class GARCH model can accommodate dynamic volatility, enabling efficient option pricing and an assessment of risks. Mercuri [12] extended the GARCH model for option pricing by defining a tempered stable distribution, which enables lighter tail of real financial data.

A standard GARCH model, however, may not capture the information and predict the distributions of returns effectively because the relationship between the current variance and the variance on the previous day may not be linear. Nikolaev et al. [13] found that for the dataset consisting of DEM/GBP currency exchange rates between 3 January 1984 to 31 December 1991, an RNN–GARCH model’s performance can improve statistical characteristics, where RNN is a recurrence neural network. Nikolaev et al. [2] suggested that their recurrent network has better results and is more accurate than linear models. Kristjanpoller and Minutolo [9] applied an artificial neural network to the GARCH model because with this approach a neural network architecture can help to detect nonlinear patterns. They noted that by incorporating nonlinearity into the GARCH model, an ANN–GARCH model shows a 25% reduction in mean absolute percentage errors for gold spot price and gold future price data collected from Bloomberg. Kristjanpoller and Minutolo [14] showed that an ANN–GARCH model improves the prediction of future oil prices. Liu et al. [15] used an asymmetric GARCH process for deposit insurance pricing in which the conditional covariance depends on the conditional variance, the square root of the conditional variance and the perturbation from the previous period. Kim and Won [16] proposed a hybrid model integrating Long Short-Term Memory (LSTM) with multiple GARCH-type models to incorporate nonlinearities in the model. Ardia et al. [17] defined different tail and asymmetry parameters for different states in a GARCH model in order to incorporate regime changes in their volatility dynamics. Cheikh et al. [18] applied a logistic function in the conditional variance equation of a GARCH model in order to capture the asymmetric behavior of bad news compared to good news.

To summarize, GARCH models are popular in finance and economic literature [1,2,3,5,6,7,8,9,10,11,12]. However, standard GARCH models may not capture the volatility dynamic adequately because the relationship between the current variance and the variance on the previous day may not be linear [2,9,13,14,15,16,17,18]. Kristjanpoller and Minutolo [14] showed that an ANN–GARCH model improves the prediction of future oil prices. Kim and Won [16] proposed a hybrid model integrating LSTM with multiple GARCH-type models to incorporate nonlinearities in the model. These inspired us to develop a hybrid model: LSTM model in the GARCH framework.

There are several notable differences between our proposed models and Kristjanpoller and Minutolo [14]’s model. Firstly, our models systematically incorporate information on the previous days through the use of LSTM, but the approach taken by Kristjanpoller and Minutolo [14] was simply to apply ANN on mostly GARCH forecasts, the square of the oil price return and other financial series to output volatility forecast. Secondly, we use the characteristics of the stock index as inputs. Their first models are built by GARCH forecast and the square of the oil price returns. When their initial model is stable, they include four financial series of data. The list of input variables in their model then changes with time. Consequently, their model building process requires much more time and computation power. Thirdly, the objective functions are deterministic while our objective function takes into account the probabilistic nature of returns. Their objective functions are heteroscedasticity adjusted mean squared error and heteroscedasticity adjusted mean squared error but our objective function is the negative loglikelihood function.

There are obvious differences between our proposed models and Kim and Won [16]’s model. Firstly, their LSTM model inputs include the parameters of GARCH, EGARCH, and EWMA at different times in order to measure the trend factors at different times, indicating that they carried out multiple optimizations for calculation of inputs. This requires more time for training and may accumulate errors in inputs due to a substantial number of objective functions for optimizations. Secondly, their model is more complex than our proposed models, meaning that their model is more prone to overfitting. Thirdly, the objective functions are deterministic while our objective function takes into account the probabilistic nature of returns. Their objective functions are mean absolute error (MAE), mean squared error, heteroscedasticity adjusted MSE and heteroscedasticity adjusted MAE but our objective function is the negative loglikelihood function.

The main contribution of this paper is that it proposes an LSTM model with a GARCH framework. LSTM can systematically take into consideration the potential nonlinearity in a volatility structure at different time points. Therefore, our models are more robust and comprehensive in predicting the volatility of returns. To examine the accuracy of our models, we need to visualize the recurrent network model performances and evaluate metrics of model performances. In order to visualize the recurrent network model performances, we compare the actual returns with the predicted standard deviations and compare the actual squared returns with the predicted variances. In general, the predicted standard deviations increase when the returns fluctuates rapidly. For some of the models, actual $r_t^2$ and the predicted variance are very close. In addition, to evaluate metrics in the recurrent network models, we calculate the mean absolute errors of absolute returns for the recurrent network models, and compare them with those for the GARCH(1,1) model with standardized Student’s t distribution. The results show that two of the recurrent network models outperform the GARCH(1,1) model with standardized Student’s t distribution. As a result that the models are complex and our aim was to understand the models better, we calculate the variable importance of each of the variables in the data set. The idea of variable importance in this paper is the difference in loglikelihood after removing the variable of interest. Our result shows that Net Debt/EBITA is important in all of the recurrent network models we have examined.

There are two main advantages of the proposed models. The first advantage is that they are easier to implement because the program runs are carried out in Tensorflow. As mentioned in Heaton et al. [19] and Chen et al. [20], it is convenient to set up ANN architectures in Tensorflow. The computational efficiency, which is demonstrated in our numerical experiments, is also an attractive feature of Tensorflow. In Nikolaev et al. [13] and Kristjanpoller and Minutolo [9], it was necessary to write down and compute the derivatives. In Tensorflow, with a standard layer like LSTM, the optimization is automatic and fast after defining the objective function, network structure and commonly used optimization algorithm. The second advantage is that the proposed models enable variable importance analysis. Nikolaev et al. [13], Nikolaev et al. [2], Kristjanpoller and Minutolo [9], and Kristjanpoller and Minutolo [14] did not provide variable importance analysis and therefore did not allow for a thorough understanding by readers of what variables are important in their models. Our paper provides a metric for accessing variable importance to demonstrate which variables are important.

Our paper is structured as follows. Section 2 presents our proposed network architecture and displays the objective function. Section 3 describes the data and its processing. Section 4 describes the training specifications. Section 5 presents the results after running the models.

2. GARCH–ANN Construction

2.1. Network Architecture

To forecast variance, the methodology in this study follows a hybrid model: LSTM model and GARCH framework. The model first defines that the return $r_t$ follows a standardized Student’s t distribution with conditional variance $h_t$. The conditional variance $h_t$ is then modeled by LSTM. The objective function for parameter estimation is the negative loglikelihood.

The GARCH model is commonly used for return modeling because it takes into account the dynamic nature of the conditional variance of the return. The GARCH(1,1) model can be expressed as

$$\begin{array}{cccc}\hfill & r_t={ϵ}_t\sqrt{h_t}\hfill & \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hfill & \hfill {ϵ}_t\sim N(0,1),\end{array}$$

(1)

$$\begin{array}{cc}\hfill & h_t={\alpha }_0+{\alpha }_1{h}_{t-1}+{\beta }_1{r}_{t-1}^2,\hfill \\ \hfill & {\alpha }_0>0{\alpha }_1,{\beta }_1\ge 0.\hfill \end{array}$$

(2)

The stationary condition of the GARCH(1,1) model is ${\alpha }_1+{\beta }_1<1$. In Formulas (1) and (2), the return at time t, $r_t$, follows a normal distribution with mean zero and variance $h_t$, given the information up to time $t-1$. The variance at time t is the linear combination of the conditional variance and the squared return at time $t-1$.

In the GARCH(1,1) model, the current conditional variance is a linear function of conditional variance and the squared return on the previous day. However, this assumption may not hold because the relationship may not be linear. Nikolaev et al. [13] showed that dynamic recurrent network yields results with improved statistical performance. Nikolaev et al. [2] showed an improved result after using a nonlinear recurrent network. The target variable return can be modeled as

$$\begin{array}{cc}\hfill r_t={ϵ}_t\sqrt{h_t},& {ϵ}_t\sim T(0,1,\upsilon ),\hfill \end{array}$$

where $T(0,1,\upsilon )$ is Student’s t distribution with mean 0, variance 1 and degrees of freedom $\upsilon$. Assume that $x_t$ is an information vector at time t.

The architecture of $h_t$ is shown in Figure 1. In the model, we use the data on the previous 20 days ${(x_{t-20}^{\prime },\dots ,x_{t-1}^{\prime })}^{\prime }$ to predict the conditional variance of the current returns, $h_t$. If the period used for predicting the variance of the current return is too long, the risk is that the prediction decreases owing to the fact that the earlier data may not reflect the future return structure. If the period used for predicting the variance of the current return is too short, the risk is that the available information may not be utilized to predict the variance of the returns. However, the numbers of days of data used for prediction of the returns can be changed according to users’ preferences.

Assume that input row vector in LSTM cell is $x_{\ell -1}$. The LSTM cell can be represented by

$$\begin{array}{cc}\hfill & y^{ou{t}_j}(\ell )=f_{ou{t}_j}(\sum _m{w}_{ou{t}_{j,m}}{x}_{\ell -1,m}),\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & y^{i{n}_j}(\ell )=f_{i{n}_j}(\sum _m{w}_{i{n}_{j,m}}{x}_{\ell -1,m}),\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill & s_{c_j^v}(\ell )=s_{c_j^v}(\ell -1)+y^{i{n}_j}\left(t\right)g_{c_j^v}(\sum _m{w}_{F_{j,m}}{x}_{\ell -1,m}),\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill & y^{c_j^v}(\ell )=y^{ou{t}_j}(\ell )u(s_{c_j^v}(\ell )),\hfill \end{array}$$

(6)

where $x_{\ell -1,m}$ is the m-th element of $x_{\ell -1}$ for $\ell =t-19,,,,t$. The terminology in LSTM models can be found in Hochreiter and Schmidhuber [21] and Gers et al. [22]. Here, $w_{ou{t}_{j,m}}$, $w_{i{n}_{j,m}}$, and $w_{F_{j,m}}$ are parameters to be estimated in the model; $f_{ou{t}_j}(.)$ and $f_{i{n}_j}(.)$ are activation functions of output gate and input gate respectively, and the default activation functions are sigmoid functions in Tensorflow; $s_{c_j}$ is internal state; $g_{c_j^v}\left(\right)$ is an activation function for cells and the default activation function is a hyperbolic tangent function in Tensorflow; $u(.)$ is a differentiable function that scales an memory cell from an internal state, and the default function, $u(.)$, is a hyperbolic tangent function in Tensorflow. Equations (3) and (4) compute the output and input gate activation, respectively. In Equation (5), the internal state $s_{c_j^v}(\ell )$ is calculated by adding squashed and gated input to the previous internal state $s_{c_j^v}(\ell -1)$. Then, in Equation (6), the cell output $y^{c_j^v}(\ell )$ is calculated by multiplying the output gate activation with internal states squashed by $u(.)$.

LSTM models can help understand the long-distance relationship between volatility and the explanatory variables. He et al. [23] compared an LSTM model with other neural network models when predicting students’ performance in a specific course. They discussed that LSTM can memorize longer sequence and achieve better results for tasks requiring understanding of long-distance relationship.

After running 20 LSTM cells as specified in Figure 1, we use a deep network to output $h_t$. Assume that $D(.)$ is a deep neural network as in Feng et al. [24]. Then, $h_t$ can be represented by

$$\begin{array}{c}\hfill h_t={\left(log(exp\left(D\left(y^{c_1}\left(t\right)\right)\right)+1)\right)}^2,\end{array}$$

(7)

where $D(.)$ is a deep learning architecture consisting of dense layers. In the deep learning architecture with function $D(.)$, the number of hidden units and number of layers are as specified in Feng et al. [24]. The activation functions may not be Rectified Linear Unit (RELU) functions because we experimented with different activation functions and found that the model performance depends on the activation functions.

In the neural network, the backpropagation and chain rule are used when computing the gradients of the objective function. The LSTM layer is considered to be the first layer in Tensorflow and, therefore, we can use the default functions without worrying about gradient issues. However, when considering the activation functions in $D(.)$, we prefer the activation functions that do not result in diminishing/exploding gradient issue. To assess this issue, we tried an RELU and identity function. In our experiment, the identity function works better for this network because RELU may introduce a zero gradient in some regions.

To model non-negative volatility, a mapping from neural network output to a non-negative number is required. Dugas et al. [25] introduced a softplus function as sigmoid’s primitive. They discussed that a softmax function is differentiable and has positive derivatives. Senior and Lei [26] explained that the reason for using a softplus function in neural network is that unlike an RELU, a softplus function does not have a zero gradient in regions, which enables easier training. They found that a neural network with a softplus function shows an improved performance and faster convergence. Consequently, we apply a softplus function to the output from LSTM cells in (7).

2.2. Objective Function

We follow the approach by Nikolaev et al. [2] who applied a maximum likelihood approach in their proposed models. Assume that $g(.;\mu ,{\sigma }^2,\upsilon )$ denotes Student’s t density function with a mean $\mu$, variance ${\sigma }^2$, and degree of freedom $\upsilon$. The objective function is

$$\begin{array}{c}\hfill \underset{f}{\arg \max }{\mathsf{\Pi }}_{t=1}^{N}g(r_t;0,h_t,\upsilon )=\underset{f}{\arg \min }-\sum _{t=1}^{N}logg(r_t;0,h_t,\upsilon ),\end{array}$$

where

$$\begin{array}{cc}\hfill logg(r_t;0,h_t,\upsilon )=& log\mathsf{\Gamma }\left(\frac{\upsilon +1}{2}\right)-log\mathsf{\Gamma }\left(\frac{\upsilon }{2}\right)-\frac{1}{2}(log\pi +log(\upsilon -2)+log{h}_t)\hfill \\ \hfill & -\frac{\upsilon +1}{2}log(1+\frac{r_t^2}{(\upsilon -2)h_t}).\hfill \end{array}$$

The default optimization in Tensorflow is to minimize the objective function and therefore the negative loglikelihood objective function is used.

3. Data Processing

The target stock is the S&P 500 Index. After the removal of data entries for holidays, the data consisted of 17 variables with 5032 observations between 29 February 2000 and 28 February 2020 from the Bloomberg Terminal. In the data, the target variable is the daily percentage change in prices. Assume that $P_t$ is the price at time t. According to the definition from Bloomberg, the daily percentage change in prices can be written as

$$r_t=\frac{P_t-P_{t-1}}{P_{t-1}}\times 100.$$

Denote an indicator function as $I(.)$. Variables in the data set and their summary statistics are as follows:

Table 1. Data summary, where Q1, Q2, Q3 refer to 25% quartile, median, and 75% quartile, respectively.

Variables	Mean	SD	Min	Q1	Q2	Q3	Max
Dividend Per Share	0.12	0.17	0.00	0.01	0.05	0.17	1.48
Dividend Payout Ratio	47.07	17.96	31.76	37.06	40.98	50.93	149.44
Price to Book Ratio	2.82	0.60	1.45	2.42	2.80	3.12	5.18
Book Value per Share	569.11	178.44	294.92	419.68	531.66	734.31	909.82
Total Assets	2963.16	671.13	1697.03	2490.23	2993.91	3439.50	4289.84
Total Current Assets	414.51	111.10	248.15	327.81	374.38	511.33	638.52
Return on Assets	2.48	0.79	0.47	2.23	2.79	3.06	3.46
Total Debt to Total Assets	31.08	6.05	23.33	25.02	29.78	37.53	39.40
Net Debt/EBITDA	2.87	1.38	1.04	1.52	2.42	4.30	4.93
Total Debt to EV	0.45	0.11	0.28	0.35	0.46	0.51	0.84
Net Debt Per Share	500.11	209.89	214.27	318.64	467.07	639.34	960.83
Enterprise Value	2074.31	587.25	1069.78	1635.50	1935.85	2356.26	3929.46
Current Ratio	1.29	0.11	1.04	1.18	1.32	1.38	1.45
$r_t$	0.02	1.19	−9.03	−0.47	0.06	0.57	11.58
$r_t^2$	1.41	4.65	0.00	0.05	0.28	1.10	134.10
$I(r_t>0)$	0.53	0.50	0.00	0.00	1.00	1.00	1.00
$r_t^{2}I(r_t>0)$	0.70	3.51	0.00	0.00	0.00	0.32	134.10

shows the S&P 500 Index variables collected from the Bloomberg terminal. In Table 1, Total Assets, Total Current Assets, Net Debt Per Share, Book Value per Share, and Enterprise Value have a substantially higher mean, standard deviation, and range than other variables, indicating that additional effort in normalization or standardization may be needed in order to simplify the training for the neural networks.

Then, we have done the following steps to create training and testing instances for program runs:

• Read the data by pandas in Python.
• Normalize the data set. Assume that x is a vector. We normalize it by

  $$norm\left(x\right)=\frac{x-\overline{x}}{max\left(x\right)-min\left(x\right)}.$$
• Create the instances by labeling the data on the previous 20 days as the input and labeling the current return as the target variable.
• Create the training instances (90%) and testing instances (10%).
• Shuffle the training instances.

Normalization is crucial for simplifying the tasks in a neural network. Shavit [27] concluded that proper normalization simplifies neural network’s task and improves its ability to control the process. Jayalakshmi and Santhakumaran [28] showed that the performance of diabetes classification is dependent on the normalization method. Yin et al. [29] applied min-max normalization images. As a result that normalization is widely used and can help to simplify the task in the neural network, we opted to apply normalization technique. The interrelationship of the data after normalization can be visualized by the correlation matrix graph shown in Figure 2.

Most of the variables do not have a strong linear relationship, justifying the use of a neural network. After normalization, the instances are created by labeling the data on the previous 20 days as the input and labeling the current return as the target variable. As shown in Figure 1, the inputs and outputs are ${(x_{t-20}^{\prime },x_{t-19}^{\prime },\dots ,x_{t-1}^{\prime })}^{\prime }$ and $r_t$ for $t=21,\dots ,4529$, respectively. The next step is to split the data set into training instances (90%) and testing instances (10%). The instances with the 90% earliest dates are training instances and the rest of instances are testing instances. The target variables for training are in between 28 March, 2000 and 1 March, 2018 while the target variables for testing are in between 2 March, 2018 and 28 February, 2020. The final step is to shuffle the training instances so as to avoid an effect from the ordering of training instances.

4. Training

We have the following specifications for program runs:

• The instances during training are split into batches with size 500 for optimization.
• The optimization algorithm is Adam [30].
• The number of epochs it can tolerate for no improvement is 10.
• The maximum number of epochs is 1000.
• Student’s t distribution in Python needs to be standardized as in GARCH-t (Bollerslev [1], So and Xu [7]).

During training, the instances are split into batches with a size of 500 for optimization. According to Goodfellow et al. [31], with the minibatch method, the gradient is computed on more than one occasion but fewer than full training examples. They listed a number of reasons for using minibatches in deep neural networks. Minibatches provide a more linear return of accuracy. They also noted that in order to maintain stability due to the high variance in estimation of the gradient, small learning rate needs to be set.

To address the problem of setting appropriate learning rates at each epoch, Adam [30], an adaptive learning rate algorithm, is used in neural network training.The core idea of Adam is that it incorporates a bias correction. In commonly used optimization algorithms such as RMSprop [32], they did not take into account the bias correction even though they accumulated the first order moments and second order moments in the (exponential) average equations. Goodfellow et al. [31] discussed that with early stopping, the algorithm can halt whenever overfitting begins to occur. The number of epochs it can tolerate for no improvement, which is referred as patience in keras, is set to be 10. The maximum number of epochs is 1000, which enables the program to continue running before early stopping. Special care is needed in defining the distribution within the Tensorflow package. Student’s t distribution in Python needs to be standardized by $\sqrt{(\upsilon -2)/\upsilon }$ in order to produce the desired results.

5. Empirical Results

We have produced network diagrams shown in Figure 3 for reference. The shape of each neural network and the names of the layers have been shown in order to give a more extensive view of the architecture. For example, in Figure 3a, the input layer is of dimension 20 × 17 because we use 20 days of data with 17 variables. Architectures such as LSTM(32)+dense(16) and LSTM(32)+dense(16-8) are shown in captions of Figure 4. LSTM(32)+dense(16) means an LSTM layer with 32 units, followed by a dense layer with 16 units. LSTM(32)+dense(16-8) means an LSTM layer with 32 units, followed by two dense layers with 16 units and 8 units, respectively. As LSTM(32)+dense(16-8) contains one more layer than LSTM(32)+dense(16), LSTM(32)+dense(16-8) contains more parameters than LSTM(32)+dense(16). The reason for an LSTM layer with different dense layers in the table is that we would like to see if the difference in dense layers can result in different out-of-the-sample mean absolute error.

The program runs last for less than two minutes on a CPU-compiled Tensorflow. All of the program runs converge and stop early. To ensure that the program always returns the same results, the random seed in neural networks is set to be zero. After training, we have also drawn training and testing negative loglikelihood over epochs. If the experimental designs are not good, the negative loglikelihoods do not decrease with time. Figure 4 shows the negative loglikelihood over training. As the negative loglikelihood decreases over training for all models except LSTM(32)+dense(64-32-16-8), the overall performance for the training set improved over training. We have also plotted actual $r_t$ and the predicted standard deviation for validation set. If the actual $r_t$ fluctuates rapidly and the predicted standard deviation does not increase, the model does not have high prediction power. Figure 5 shows that except for LSTM(32), the predicted standard deviations increase when the returns fluctuates rapidly. To review the performance of the neural networks, the squared returns and the predicted variances in the testing set are plotted. In Figure 6, except for LSTM(32), actual $r_t^2$ and the predicted variance are very close.

Forsberg and Ghysels [33] discussed that absolute returns predict volatility better. They showed that absolute returns is immune to jumps. From So and Xu [7] and Winkelbauer [34], we can prove the moments of absolute returns in GARCH model to be

$$\begin{array}{cc}\hfill E\left(\right|r_t{|}^c\left|{\Im }_{t-1}\right)& =h_t^{c/2}{(\upsilon -2)}^{c/2}\frac{\mathsf{\Gamma }\left(\frac{\upsilon -c}{2}\right)}{\mathsf{\Gamma }\left(\frac{\upsilon }{2}\right)}{2}^{-c/2}\times 2^{c/2}\frac{\mathsf{\Gamma }\left(\frac{c+1}{2}\right)}{\sqrt{\pi }}\hfill \\ \hfill & =h_t^{c/2}{(\upsilon -2)}^{c/2}\frac{\mathsf{\Gamma }\left(\frac{\upsilon -c}{2}\right)\mathsf{\Gamma }\left(\frac{c+1}{2}\right)}{\mathsf{\Gamma }\left(\frac{\upsilon }{2}\right)\sqrt{\pi }},\hfill \end{array}$$

(8)

where $\mathsf{\Gamma }(.)$ is a gamma function. So and Xu [7] discussed that the use of $|r_{t,i}{|}^{1.5}$ can make their evaluation more robust.

Out-of-the-sample mean absolute errors (MAE) in

Table 2. Out-of-sample mean absolute errors (MAE) evaluated between 2 March 2018 and 28 February 2020. To evaluate the performances of models, we computed MAE for $|r_t|$, $|r_t{|}^{1.5}$, and $r_t^2$. The smaller MAE are, the better the performance is.

Architecture	$|{\mathit{r}}_{\mathit{t}}|$	$|{\mathit{r}}_{\mathit{t}}{|}^{1.5}$	${\mathit{r}}_{\mathit{t}}^2$
LSTM(32)	0.0293	0.0102	0.0036
LSTM(32)+dense(16)	0.1163	0.0843	0.0702
LSTM(32)+dense(16-8)	0.0290	0.0107	0.0042
LSTM(32)+dense(16-8-4)	0.0204	0.0064	0.0020
LSTM(32)+dense(32-16-8)	0.0195	0.0060	0.0019
LSTM(32)+dense(64-32-16-8)	0.0209	0.0067	0.0022
GARCH-t(1,1)	0.0221	0.0073	0.0021

are evaluated between 2 March 2018 and 28 February 2020. The model does not change when evaluating the errors. The following steps are taken during MAE:

• Forecast one-step ahead $h_t$.
• Compute the expectations of the variables shown in Table 2 by Equation (8).
• Out-of-the-sample mean absolute error (MAE) is computed by the equation:

  $$\mathrm{MSE}=\frac{1}{n}\sum _t\left|\right|r_t{|}^c-E(|r_t{|}^c|{\Im }_{t-1})|.$$

To compare the model performances with a standard model, we used an ARCH package [35] to run a GARCH-t(1,1) model which is a standard GARCH(1,1) model, where the residual term at time t (${ϵ}_t$) follows standardized Student’s t distribution. The parameters ${\alpha }_0$, ${\alpha }_1$, and ${\beta }_1$ in the Equation (2) are estimated to be 0.2234, 0.1001, and 0.8978, respectively. Their respective p values by t tests are 2.012 × 10${}^{-3}$, 8.910 × 10${}^{-15}$, and 0, meaning that the parameters are significant. The package reports that the optimization is robust and the degree of freedom is determined to be 6.7057 by the package. The method used is maximum likelihood estimation, and the final loglikelihood is −13,192.

Compared to the GARCH-t(1,1) model, the two deep learning models performed better as they have smaller out-of-sample errors between 2 March 2018 and 28 February 2020. In terms of MAE for $|r_t|$ and $|r_t{|}^{1.5}$, LSTM(32)+dense(16-8) and LSTM(32)+dense(32-16-8) outperform GARCH-t(1,1). However, in term of MAE for $|r_t{|}^2$, the two LSTM models are very close to GARCH-t(1,1). As Forsberg and Ghysels [33] have discussed that realized absolute returns are immune to jumps and are more persistent, it is believed that the MAE for $|r_t|$ and $|r_t{|}^{1.5}$ are more reliable measures, indicating that the two LSTM models outperform GARCH-t(1,1). Out of all the models, in terms of $|r_t|$ and $|r_t{|}^{1.5}$, LSTM(32)+dense(32-16-8) performs the best. Compared to LSTM(32)+dense(32-16-8), LSTM(32)+dense(64-32-16-8) is more likely to suffer from overfitting because it has one more layer and more parameters.

Variable importance is essential for variable selection. However, there is no universal way to evaluate variable importance in different models. Breiman [36] discussed that for randomly generated input features with weak correlations, the convergence is very slow and the result is poor in the random forest model. He permuted the value of the feature of interest while calculating variable importance. Two Python packages were developed to help researchers calculate the variable importance. Korobov and Lopuhin [37] suggested replacing the feature with random noise when calculating variable importance. Pedregosa Gael Varoquaux Alexandre Gramfort Vincent Michel Bertrand Thirion et al. [38] suggested permuting the input columns of features.

In addition to the two Python packages, caret [39], an R package that has been used for Exam Predictive Analytics by the Society of Actuaries, has taken variable importance into consideration when developing machine learning models. Kuhn [39] defined different kinds of variable importance for different models. For example, in linear regression models, variable importance is defined to be the absolute value of the t-statistic for each model parameter that is used. For the recursive partitioning model, variable importance is defined to be the reduction in the loss function (e.g., mean squared error) attributed to each variable at each split that is tabulated.

To observe which variables are more important, we perform a variable importance analysis. The variable of interest in the input is replaced with zero. Then, the new negative loglikelihood, denoted as $L_{reduced}$, is computed. Assume that $L_{full}$ is the original loglikelihood. The variable importance for the variable is calculated as follows:

$$2(L_{full}-L_{reduced}).$$

The intuition for this likelihood ratio metric is that the models we are using are probability models and the metric for evaluation is loglikelihood. Note that the variable importance we are using can be negative because the variable may not predict the variances very well. Figure 7 shows the variable importance for the six models. In Figure 7, Net Debt/EBITA is among the six most important variables for all models. In Figure 7e, the importance of Net Debt/EBITA is extremely high compared to other variables.

6. Conclusions

A non-linear GARCH model is of great importance to researchers and academics because the relationship between the current variance and the variances in the previous period may not be linear. This paper proposes a GARCH(1,1) model with an LSTM network for conditional variance in order to examine how important neural networks are when modeling stock returns. The appropriateness of deploying a recurrent neural network is investigated by the visualization of the actual returns and predicted standard deviations, visualization of the actual squared returns, and predicted variance and tabulation of mean absolute errors. Our results show that for most of the neural network models, the predicted standard deviations increase when the returns fluctuate rapidly. The actual $r_t^2$ and the predicted variance are very close and two of the recurrent network models outperform the GARCH(1,1) model with the standardized Student’s t distribution in terms of MAE for $|r_t|$ and $|r_t{|}^{1.5}$. As some tested models do not outperform the GARCH(1,1) model, the proposed LSTM model should be assessed carefully before being applied for prediction of returns. Our variance importance analysis for the models shows that Net Debt/EBITA is among the six most important predictor variables in all of the neural network models that we have examined. For implementation of our ANN–GARCH approach, in practice, we can follow what we did to split a data sample into two parts: the training set and the testing set. Since the computation is fast, we can consider a large number of ANN architectures based on experience and perform the out-of-sample testing as shown in Table 2. Then, we can identify promising architectures for predicting future volatility.