Estimating the Value-at-Risk by Temporal VAE

Abstract

Estimation of the value-at-risk (VaR) of a large portfolio of assets is an important task for financial institutions. As the joint log-returns of asset prices can often be projected to a latent space of a much smaller dimension, the use of a variational autoencoder (VAE) for estimating the VaR is a natural suggestion. To ensure the bottleneck structure of autoencoders when learning sequential data, we use a temporal VAE (TempVAE) that avoids the use of an autoregressive structure for the observation variables. However, the low signal-to-noise ratio of financial data in combination with the auto-pruning property of a VAE typically makes use of a VAE prone to posterior collapse. Therefore, we use annealing of the regularization to mitigate this effect. As a result, the auto-pruning of the TempVAE works properly, which also leads to excellent estimation results for the VaR that beat classical GARCH-type, multivariate versions of GARCH and historical simulation approaches when applied to real data.

1. Introduction

The value-at-risk (VaR) is the most prominent risk measure in finance. Being defined as a typically high quantile (the $95\%$ or the $99\%$ level are often used) of the loss function of a portfolio of assets over a fixed time horizon, it is notoriously hard to estimate. Methods to determine it range from simple historical simulation to parametric approximation. This in particular includes variants of GARCH models combined with conditional normal distribution assumptions (see e.g., Engle 2012) and first applications of neural networks (see e.g., Chen et al. 2009).

In this work, we propose the temporal variational autoencoder (TempVAE), a variational recurrent neural network (VRNN) designed to handle the low signal-to-noise ratio in financial time series1, and apply it to VaR estimation. By not assuming an autoregressive structure for the observation variables, we explicitly force the model to use the bottleneck structure from autoencoders.

The key concept behind the TempVAE is the variational autoencoder (VAE; see Kingma et al. 2014; Rezende et al. 2014). As a variational distribution is used to approximate an otherwise usually intractable posterior, VAE can be interpreted as a stochastic version of standard autoencoders (AE). The use of a prior distribution results in a more meaningful latent space fragmentation and an increase in robustness against common training pitfalls such as over-fitting or model misspecification (see Sicks et al. 2021).

With the introduction of VRNNs, Chung et al. (2015) show promising results to model complex data sequences, such as speech data, as compared to standard RNN models. Based on this work, Luo et al. (2018) propose a neural stochastic volatility model (NSVM) and show that, using VRNN, it is possible to adequately model the volatility of financial data. However, the low signal-to-noise ratio makes variational models prone to posterior collapse, i.e., the training results in the encoder become independent of the input.

The reason for the posterior collapse is the auto-pruning. During training, the net sets nodes of the latent space as inactive, which can be troublesome. On one hand, the model focuses on useful representations by pruning away not needed nodes. On the other hand, it is a problem when too many nodes become inactive before learning a useful representation. This property of variational models has been analyzed by various authors (see, e.g., Burda et al. 2016; Sicks et al. 2021) who demonstrate that VAE are capable of identifying the correct data-generating model.

As we show here, the TempVAE is also prone to posterior collapse. We use annealing (see also Bowman et al. 2015) of the regularization to mitigate this effect. As we weaken the effect of the auto-pruning only at the initial steps of the training, we are able to find useful representations of sequence data.

In total, we are using an existing method/model which has been introduced in (Chung et al. 2015) and further developed in (Luo et al. 2018). However, as an innovative ingredient, we restrict the generative part of the model to solemnly rely on the latent variables. Therefore, in contrast to the models of the two aforementioned references, our model can only pass information through the bottleneck structure of the VAE. As a result, the signal identification via the bottleneck activities (see Equation (21)) becomes feasible, whereas for the other models, it is not clear which source influences the outcome. We have added the use of beta-annealing (as introduced in (Bowman et al. 2015)) to this setting to arrive at our TempVAE framework. Our further main innovations and contributions are

• The signal identification analysis of the model applied to financial time series where we in particular show that the TempVAE possesses the auto-pruning property;
• A test procedure to demonstrate that the TempVAE identifies the correct number of latent factors to adequately model the data;
• The demonstration that our newly developed TempVAE approach for the VaR estimation performs excellently and beats the benchmark models; and
• The detailed documentation of the hyperparameter choice in the appendix (the ablation study).

The remaining parts of our contribution are structured as follows. In Section 2, we present related work and compare it to our research. The definition of the TempVAE can be found in Section 3. In Section 4, we explain the data and provide our results for the VaR estimation with the TempVAE. We state our conclusions in Section 5. The appendix contains a detailed description of various analyses that lead to the actual form of the TempVAE that we have suggested.

2. Comparison to Related Work

VAEs are popular generative probabilistic models that allow us to model complex distributions. The models approximate a usually untractable posterior distibution by one that is easier to handle. Girin et al. (2020) provide an overview on VAEs and focus especially on dynamic VAEs. Stochastic units in RNNs, such as in dynamic VAEs, have been considered by various authors. Bayer and Osendorfer (2014), Chung et al. (2015) and Fraccaro et al. (2016) propose stochastic RNN with autoregressive structures for the observations, while Goyal et al. (2017) and Luo et al. (2018) extend the approach of (Chung et al. 2015) via auxiliary cost terms in the objective or to bidirectional RNNs, respectively. Moreover, Xu and Chen (2021) propose a similar model to (Luo et al. 2018) and apply it to financial return data.

Krishnan et al. (2017) consider a Gaussian state space model by assuming the Markov property for the latent variables and excluding autoregressive structures. Given their assumptions, they show that the true posterior for the latent variables given the observations only depends on future observations. Additionally, Fraccaro et al. (2017) consider a combination of VAE with a state space model. In particular, they assume that the observations of the state space model make up the bottleneck of the VAE. This way, they achieve a disentanglement of the latent representation and the dynamics.

Finally, Bowman et al. (2015) consider the encoding and decoding of single-layer RNN sequences via VAEs to model speech data. They argue that annealing of the regularization term is needed for learning meaningful information passing through the bottleneck.

Alternatively to the models by (Chung et al. 2015) and (Luo et al. 2018), we model a temporal dependency only in the prior distribution. Therefore, our model is similar to the deep Markov model by (Krishnan et al. 2017), but we do not use the Markov assumption.

By excluding the dependency from past realizations (the autoregressive part) for the observable returns R, the return distribution is independent from past returns but not from the past latent variables Z. Therefore, we force the encoder of the implemented model to encode as much information as possible in the latent variables Z. As a consequence, properties of conventional VAE such as the auto-pruning can be observed properly. This is not necessarily the case if we model the decoder distribution to be autoregressive on the returns R2, as assumed by (Luo et al. 2018) and (Chung et al. 2015). In this case, a substantial part of the information that can be used by the decoder would not have to pass the bottleneck. Even if the posterior $q(Z|R)$ collapses and becomes independent of the input, the decoder would be able to use past observed R to achieve a good fit.

Various authors have proposed to estimate the VaR of a portfolio with artificial neural networks (ANN). Liu (2005) uses the estimates of historical simulation and GARCH as input for ANNs. Therefore, his model can be interpreted as an ensemble model. Chen et al. (2009) and Arimond et al. (2020) estimate the VaR by modeling parameters of their respective distribution assumption. Chen et al. (2009) use a standard ANN whereas Arimond et al. (2020) compare ANNs with temporal convolutional neural nets and RNNs. Arian et al. (2020) use standard VAE for the task of estimating the VaR. To account for the time dependency in the data, they preprocess the data with rolling window statistics. Fatouros et al. (2022) estimate the VaR of the underlying univariate time series composing the portfolio via a recurrent neural net. They combine the univariate VaR estimates via an estimate of the covariance matrix of the log-returns of the elements of the portfolio.

In contrast to this, in our work, we model the data not univariately but in a multivariate way. In particular, we do not have to rely on a conventionally estimated covariance matrix or on a parameterized model for the log-returns. Our approach is based on the observation that the joint log-returns of asset prices can often be projected to a lower dimensional space. By using the TempVAE for modeling the portfolio return, we can generate many realizations of the actual return (we use 1000 realizations) and then use the resulting empirical quantile of the loss function for the estimation of the VaR. Thus, the TempVAE yieds a time-dependent and, due to the auto-pruning property, parsimonious model for the data sequences.

3. The Temporal Variational Autoencoder

We use the TempVAE to learn the multivariate distribution of asset returns. Therefore, we assume that the return series $R=R_{1:T}$ is generated via latent variables $Z=Z_{1:T}$, where $T\in \mathbb{N}$ and $T>1$. The distributions of the variables $Z_t$ are estimated via approximate inference, and we expect these to hold information such as the current market environment, peer group behavior or interactions. Without further assumptions, the identification of the underlying information within the variables is usually infeasible. For example a mere rotation results in a model with the same expressiveness. Nonetheless, as we will see, the auto-pruning forces the TempVAE to model the data by only active dimensions of the latent space.

Given the two discrete-time stochastic processes R and Z, we assume an autoregressive time dependency on the latent variables given by

$$Z_t|Z_{1:t-1}\sim \mathcal{N}\left({\mathit{\mu }}^z(Z_{1:t-1}),{\mathbf{\Sigma }}^z(Z_{1:t-1})\right),$$

(1)

$$R_t|Z_{1:t}\sim \mathcal{N}\left({\mathit{\mu }}^r(Z_{1:t}),{\mathbf{\Sigma }}^r(Z_{1:t})\right).$$

(2)

Here, ${\mathit{\mu }}^z$, ${\mathit{\mu }}^r$, ${\mathbf{\Sigma }}^z$ and ${\mathbf{\Sigma }}^r$ are functions mapping to ${\mathbb{R}}^{\kappa }$, ${\mathbb{R}}^d$, ${\mathbb{R}}^{\kappa \times \kappa }$ and ${\mathbb{R}}^{d\times d}$, respectively. The variables $R_1,\dots ,R_T$ are assumed to be i.i.d. Given this dependency structure, the joint distribution can be factorized as

$$\hspace{1em}\hspace{1em}{p}_{\mathit{\theta }}(Z)=\prod _{t=1}^T{p}_{\mathit{\theta }}(Z_t|Z_{1:t-1}),$$

(3)

$$p_{\mathit{\theta }}(R|Z)=\prod _{t=1}^T{p}_{\mathit{\theta }}(R_t|Z_{1:t}).$$

(4)

To account for the dependency of the past sequences, we use RNN layers. Furthermore, we use Multi-Layer Perceptrons (MLPs) to map from the output of the RNNs to the parameter space of the distributions. In total, we use the model architecture given by

$$\left\{{\mathit{\mu }}_t^z,{\mathbf{\Sigma }}_t^z\right\}={\mathrm{MLP}}_G^z\left(h_t^z\right),$$

(5)

$$\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{h}_t^z={\mathrm{RNN}}_G^z\left(h_{t-1}^z,Z_{t-1}\right),$$

(6)

$$\hspace{1em}\hspace{1em}\hspace{1em}{Z}_t\sim \mathcal{N}\left({\mathit{\mu }}_t^z,{\mathbf{\Sigma }}_t^z\right),$$

(7)

$$\left\{{\mathit{\mu }}_t^r,{\mathbf{\Sigma }}_t^r\right\}={\mathrm{MLP}}_G^r\left(h_t^r\right),$$

(8)

$$\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{h}_t^r={\mathrm{RNN}}_G^r\left(h_{t-1}^r,Z_t\right),$$

(9)

$$\hspace{1em}\hspace{1em}\hspace{1em}{R}_t\sim \mathcal{N}\left({\mathit{\mu }}_t^r,{\mathbf{\Sigma }}_t^r\right),$$

(10)

which is also depicted in Figure 1. Here, Equations (5), (6), (8) and (9) mean that the parameters on their left sides are the output of the neural networks on the right ones. These parameters then characterize the normal distributions in relations (7) and (10). $h_t^z$ and $h_t^r$ are defined as the hidden states of the RNNs, propagating past information through later time points. The initial states are set to zero. Combining the hidden states of the model is possible and yields a more general model formulation. However, during our experiments, it turned out to be favorable to use not trainable priors (see Appendix B.2). In order to achieve this, we separate the hidden states as shown in Figure 1.

We approximate the intractable $p_{\mathit{\theta }}(Z|R)$ by the inference distribution $q_{\mathit{\varphi }}(Z|R)$. Similar to (Krishnan et al. 2017), one can show that $Z_t⫫R_{1:t-1}|Z_{1:t-1}$. Therefore, we can factorize the distribution such that the variable $Z_t$ depends on the (future) observations $R_{t:T}$:

$$p_{\mathit{\theta }}(Z|R)=\prod _{t=1}^T{p}_{\mathit{\theta }}(Z_t|Z_{1:t-1},R_{t:T}).$$

(11)

Krishnan et al. (2017) argue that q should be factorized the same way. As q is only an approximation to p, we also considered providing the full sequence $R_{1:T}$ instead of $R_{t:T}$ which yields a better performance (see Appendix B.7). We therefore factorize q as

$$q_{\mathit{\varphi }}(Z|R)=\prod _{t=1}^T{q}_{\mathit{\varphi }}(Z_t|Z_{1:t-1},R_{1:T})$$

(12)

and assume $q_{\mathit{\varphi }}(Z_t|Z_{1:t-1},R_{1:T})\sim \mathcal{N}\left({\widehat{\mathit{\mu }}}^z(Z_{1:t-1},R_{1:T}),{\widehat{\mathbf{\Sigma }}}^z(Z_{1:t-1},R_{1:T})\right)$. ${\widehat{\mathit{\mu }}}^z$ and ${\widehat{\mathbf{\Sigma }}}^z$ are functions mapping to ${\mathbb{R}}^{\kappa }$ and ${\mathbb{R}}^{\kappa \times \kappa }$, respectively. To use the full sequence of returns $R_{1:T}$, we implement the inference distribution via a bidirectional RNN as follows

$$\left\{{\widehat{\mathit{\mu }}}_t^z,{\widehat{\mathbf{\Sigma }}}_t^z\right\}={\mathrm{MLP}}_I^z\left({\widehat{h}}_t^z\right)$$

(13)

$$\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\widehat{h}}_t^z={\mathrm{RNN}}_I^z\left({\widehat{h}}_{t-1}^z,Z_{t-1},\left[{\widehat{h}}_t^{\to },{\widehat{h}}_t^{\leftarrow }\right]\right)$$

(14)

$$\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\widehat{h}}_t^{\to }={\mathrm{RNN}}_I^z\left({\widehat{h}}_{t-1}^{\to },R_{t-1}\right)$$

(15)

$$\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}{\widehat{h}}_t^{\leftarrow }={\mathrm{RNN}}_I^z\left({\widehat{h}}_{t+1}^{\leftarrow },R_{t+1}\right)$$

(16)

$$\hspace{1em}\hspace{1em}\hspace{1em}{Z}_t\sim \mathcal{N}\left({\widehat{\mathit{\mu }}}_t^z,{\widehat{\mathbf{\Sigma }}}_t^z\right).$$

(17)

As we have a distribution assumption with an intractable $p(Z|X)$, we train the model using the evidence lower bound (ELBO; see (Bishop 2006) Section 9.4 for a formal derivation) given as

$$\begin{array}{cc}\hfill \mathcal{L}(\mathit{\theta },\mathit{\varphi })& ={\mathbb{E}}_{q_{\mathit{\varphi }}(Z|R)}\left[\log p_{\mathit{\theta }}(R|Z)\right]+{\mathbb{E}}_{q_{\mathit{\varphi }}(Z|R)}\left[\log \left({\displaystyle \frac{p_{\mathit{\theta }}(Z)}{q_{\mathit{\varphi }}(Z|R)}}\right)\right]\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill & \hspace{1em}={\mathbb{E}}_{q_{\mathit{\varphi }}(Z|R)}\left[\log p_{\mathit{\theta }}(R|Z)\right]-D_{KL}\left(q_{\mathit{\varphi }}(Z|R)||p_{\mathit{\theta }}(Z)\right).\hfill \end{array}$$

(19)

The components are given by Equations (3), (4) and (12). As we have $\log p_{\mathit{\theta }}(R)\ge \mathcal{L}(\mathit{\theta },\mathit{\varphi })$, maximization of the ELBO most likely leads to an increase in the likelihood of our distribution model. In a similar way to (Krishnan et al. 2017), we can rewrite the ELBO as

$$\begin{array}{c}\hfill \mathcal{L}(\mathit{\theta },\mathit{\varphi })={\mathbb{E}}_{q_{\mathit{\varphi }}(Z|R)}\left[\log p_{\mathit{\theta }}(R|Z)\right]-\sum _{t=1}^T{\mathbb{E}}_{q_{\mathit{\varphi }}(Z_{1:t-1}|R)}[D_{KL}\left(q_{\mathit{\varphi }}(Z_t|Z_{1:t-1},R)\left.∥p_{\mathit{\theta }}(Z_t|Z_{1:t-1})\right)\right].\end{array}$$

(20)

Since we assume a Gaussian distribution for q and p, we can analytically calculate the Kullback–Leibler divergence (KL-Divergence) inside the expectation in (20).

The Auto-Pruning Property of VAEs and the Posterior Collapse

The auto-pruning property of VAE originates from the KL-Divergence and has been analyzed by various authors (see Burda et al. 2016; Sicks et al. 2021). When nodes in the bottleneck are “pruned away”, it means that these nodes are not used in the neural net and therefore also not for the reconstruction. The auto-pruning capabilities of VAE are desirable, as the model focuses on useful latent representations. However, it is considered a problem when too many units become inactive before the actual learning of such a useful representation has performed some successful first steps. In the extreme case, the KL-Divergence between $q_{\mathit{\varphi }}(Z|R)$ and $p_{\mathit{\theta }}(Z)$ is 0, and we have $q_{\mathit{\varphi }}(Z|R)\equiv p_{\mathit{\theta }}(Z)$. Hence, q is independent of R, and we say that the posterior q collapses as the input does not matter.

As we consider multivariate Gaussian distributions with a diagonal covariance matrix, it becomes apparent that only a subset of nodes can be inactive. The posterior does not collapse in this case, as information can still be propagated to the decoder through active nodes. A diagonal covariance matrix implies independence of the latent variables Z. In order for a univariate part to be inactive, it has to be independent of the other dimensions. Nonetheless, the overall independence is not a necessity to monitor inactivity. For example, block-wise covariance matrices would result in batches of active or inactive latents.

When using the TempVAE on financial data, we observe an instantaneous collapse of the posterior at the beginning of the training. From our point of view, the signal within the financial returns data is not strong enough to be captured before the KL-Divergence term dominates the ELBO and causes a posterior collapse. Therefore, we consider $\beta$-annealing of the KL-Divergence similar to (Bowman et al. 2015) (see Appendix A).

To measure the amount of active units, Burda et al. (2016) propose an activity statistic that is calculated after training. For each node, they estimate the activity by

$$A_u={\mathrm{Cov}}_x\left({\mathbb{E}}_{q_{\varphi }\left(u|x\right)}[u]\right)$$

and call a node inactive if $A_u<0.01$. The intuition behind this statistic is as follows. If for given inputs x, the expected values in a latent dimension do not show significant variability, the resulting latent dimension has no useful information of the input and is seen as independent.

Since we assume a time-dependent model for the prior in (3), we adjust the activity statistic to calculate the activities of a sequence of $Z_{1:M}\in {\mathbb{R}}^{M\times \kappa }$. For $m=1,\dots ,M$ and $k=1,\dots ,\kappa$, we calculate

$$A_z^{(m,k)}:={\mathrm{Cov}}_{R,Z_{1:m-1}}\left({\mathbb{E}}_{q_{\varphi }\left(Z_{m,k}|Z_{1:m-1},R\right)}\left[Z_{m,k}\right]-{\mathbb{E}}_{p_{\theta }\left(Z_{m,k}|Z_{1:m-1}\right)}\left[Z_{m,k}\right]\right).$$

(21)

The statistic becomes equal to the one proposed by (Burda et al. 2016) if we assume a time-independent standard Gaussian prior3 in (3). With the statistic in (21), we quantify the effect of the input R. Similar to (Burda et al. 2016), we call a node inactive from the input if $A_z^{(m,k)}<0.01$.

4. Implementation, Experiments and VaR-Estimation

In this section, we present the results we achieved on four datasets (see Section 4.1) with our model regarding the identification of the underlying signal (see Section 4.2). Then, in Section 4.3, we present results for the fit to financial data as well as the VaR estimation. For further details on the model implementation and for an additional ablation study validating our design choices, see Appendix A and Appendix B.

4.1. Description of the Used Data Sets

We consider the four datasets “DAX”, “S&P500”, “Noise” and “Oscillating PCA” for our studies. Here, the last two datasets are simple artificial examples with the only purpose to check the correct performance of the TempVAE. Throughout the section, data are stored in d-dimensional vectors

$$R_t:={\left(R_{t,1},\dots ,R_{t,d}\right)}^T\in {\mathbb{R}}^d.$$

(22)

First, we consider (d = 22) stock price time series, provided by DAX (German stock index) enlisted companies, from 11.06.2001 to 09.06.2021. We calculate the daily logarithmic returns4 via the observed closing prices. Therefore, if $S_{t,i}$ denotes the price of stock $i=1,\dots ,d$ at time $t=1,\dots ,T$, the dataset “DAX” contains

$$R_{t,i}:=\ln \left(S_{t,i}\right)-\ln \left(S_{t-1,i}\right).$$

(23)

For “S&P500”, we consider 397 stocks of the S&P500 index that have a history from 02.01.2002 up to 09.06.2021. We calculate the log-returns analogously to the “DAX” data.

The dataset “Noise” is just white noise with dimension $d=22$. Hence, for all $t=1,\dots ,T$ and $i=1,\dots ,d$

$$R_{t,i}\sim \mathcal{N}\left(0,1\right).$$

The dataset “Oscillating PCA k” (with $k\in \{2,5,10\}$) is constructed based on k independent harmonic oscillator signals. Each univariate signal $Z_{1:T}$ is constructed by randomly choosing an intercept $i\sim \mathcal{U}\left(-1,1\right)$, an amplitude $a\sim \mathcal{U}\left(-1,1\right)$, a frequency $f\sim \mathcal{U}\left(0.5,24\right)$ and a noise series ${ϵ}_{1:T}$, with $ϵ\sim \mathcal{N}\left(0,{0.02}^2\right)$. Then, we set

$$Z_t=i+a·\cos \left({\displaystyle \frac{t}{100}}·f·\pi \right)+a·{ϵ}_t.$$

The case $k=2$ is shown in Figure 2. The oscillator sequences $Z_{1:T,1:k}\in {\mathbb{R}}^{T\times k}$ are then transformed to a 22-dimensional series by applying a randomly chosen rotation $U\in {\mathbb{R}}^{d\times k}$:

$$S_t=U·Z_{t,1:k}^T+5\mathrm{and}{R}_{t,i}:=\ln \left(S_{t,i}\right)-\ln \left(S_{t-1,i}\right)$$

for each $t=1,\dots ,T$ and $i=1,\dots ,d$.

Finally, a sliding window is applied to the d-dimensional series $R_{1:T}$ to generate $T-M$ consecutive series in ${\mathbb{R}}^{d\times M}$ of size $M=21$. For further details on preprocessing procedures, see Appendix A.

4.2. Signal Identification

We train the TempVAE on the four datasets from Section 4.1 and analyze calculated activities given by (21) as well as the fit to the data. As we can see in Figure 3 and Figure 4, the TempVAE correctly identifies the amount of hidden dimensions. For the “Noise” data, no activity is reported at all. The decoder model $p(R|Z)$ is left on its own to model the data as no meaningful information can be identified that can be propagated from input to output. For the data “Oscillating PCA 2”, the model correctly identifies the two active nodes per timestep. For the “Oscillating PCA 5” and “Oscillating PCA 10” data, the model also identifies two active dimensions through time (see Appendix D). Although this is not the amount given by the linear construction, identifying fewer dimensions is not necessarily problematic, since we have a non-linear model. We observe comparable fitting results for these datasets (see Appendix D).

When applied to the financial data, we can observe two and four active dimensions for the “DAX” and “S&P500” datasets, respectively (see Appendix E). The fact that only a few active nodes could be identified is not surprising. In their work, Laloux et al. (2000) show that only a small fraction of dimensions (roughly 11) are needed to linearly model 406 assets of the S&P500 during the years 1991–1996.

4.3. Fit to Financial Data and Application to Risk Management

To assess the performance of the model on the “DAX” data5, we compare the TempVAE to a GARCH model as well as to the multivariate GARCH versions DCC-GARCH-MVN and DCC-GARCH-MVt (see Appendix C for details). The capacities of these models are less compared to the used TempVAE neural net. Nonetheless, GARCH and its variants are market standard for VaR calculation. To underline the usefulness of a model, the results have to be compared to these benchmarks.

To compare the models, we calculate the negative log-likelihood (NLL), which is given by

$$\mathrm{NLL}(R_t^{\mathrm{nonlog}},{\mu }_t,{\mathrm{\Sigma }}_t):=\frac{1}{2}\left(d\log \left(2\pi \right)+\log \left|{\mathrm{\Sigma }}_t\right|+{\left(R_t^{\mathrm{nonlog}}-{\mu }_t\right)}^T{\mathrm{\Sigma }}_t^{-1}\left(R_t^{\mathrm{nonlog}}-{\mu }_t\right)\right)$$

and average across all timestamps in the test set. $R_t^{\mathrm{nonlog}}$ represents the non-logarithmic returns of the test data and ${\mu }_t$ as well as ${\mathrm{\Sigma }}_t$ are estimated by the models. To estimate these values, we first sample the log-returns from a model. For the TempVAE, we estimate these empirically by sampling

$$\begin{array}{c}\hfill z_{1:t}^{(i)}\sim q_{\varphi }(Z|r_{1:t-1})\mathrm{as}\mathrm{well}\mathrm{as}{r}_t^{(i)}\sim p_{\mathit{\theta }}(R|z_{1:t}^{(i)})\end{array}$$

for $i=1,\dots ,1000$. Then, we transform these samples to non-log-returns and calculate the empirical means and covariances to approximate ${\mu }_t$ and ${\mathrm{\Sigma }}_t$.

We also consider the diagonal NLL, where we use the diagonal matrix with entries $diag\left({\mathrm{\Sigma }}_t\right)$ instead of ${\mathrm{\Sigma }}_t$. Furthermore, we calculate a univariate version by considering the returns of an equally weighted portfolio containing the 22 assets. Given the log-returns in (23), we calculate these by

$$R_{t,i}^{P,\mathrm{nonlog}}:={\displaystyle \frac{1}{d}}\sum _{i=1}^d\left(\exp \left(R_{t,i}\right)-1\right).$$

(24)

Table 1. Fit scores for models TempVAE, GARCH and two multivariate GARCH. In all cases, lowest is best.

Model	Diagonal NLL	NLL	Portfolio NLL
GARCH	22.76	22.84	−0.51
TempVAE	25.79	20.35	−3.04
DCC-GARCH-MVN	25.03	18.69	−3.07
DCC-GARCH-MVt	25.44	19.17	−3.05

displays the different fit scores for the three models. Considering the NLL scores, the GARCH and normal DCC-GARCH models achieve the best performance since these models are directly minimizing the respective scores. The TempVAE as a regularized model has to find a trade-off between the NLL and the regularization terms. Nonetheless, if we consider the scores for the portfolio returns, we see all multivariate models performing similarly and outperforming the GARCH model. Due to the strong dependency between the different assets in the portfolio, estimating the underlying covariance structure plays an essential role in estimating the next day’s portfolio performance. Furthermore, we could observe the models capturing the correlations between the assets (see Appendix F for an excerpt of the first two dimensions).

To compare the performance on the VaR estimation, as a further method, we consider historical simulation (HS).6 Then, for each method, we calculate the VaR estimates via 1000 Monte Carlo samples from the respective distributions.7 VaR95 then denotes the empirical estimator for the $95\%$-Value-at-Risk which is the 50th value of the ordered samples of returns (arranged from lowest to highest). VaR99 denotes the corresponding estimator given as the 10th value of the ordered sample. For this, also note that we are looking at the actual returns, i.e., the negative of the losses.

We compare the models by their average number of exceedances, i.e., when the VaR estimate was higher then the observed return, within the test set. We below denote these values by AE95 and AE99, and they help us to validate the VaR estimates by their 5% and 1% quantile assumption. A good performance is a necessary condition for filtering out those candidates that we compare with respect to a further criterion.

As this second performance check, we use the ‘Regulatory Loss Function’ (RLF) proposed by (Sarma et al. 2003). This score penalizes exceedances of observed returns below the VaR and is given by

$$\mathrm{RLF}\left({\mathrm{VaR}}_t,r_t\right)=\left\{\begin{array}{cc}{\left({\mathrm{VaR}}_t-R_t\right)}^2,\hfill & \mathrm{if}{R}_t\le {\mathrm{VaR}}_t\hfill \\ 0,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

Here, $R_t$ denotes the value of the (non-logarithmic) portfolio return at time t. We calculate the average of these values over the test set.

In

Table 2. The table displays the values (all of these have to be multiplied by ${10}^{-2}$) of the RLF scores as well as the average number of exceedances for the VaR95 and the VaR99 within the test set. For the RLF values, lowest is best. For the average exceedances, the respective quantile level assumption is the optimal value. Hence, for VaR95, the best value is 5, and for VaR99, the best value is 1.

Model	RLF95	RLF99	AE95	AE99
GARCH	44.47	32.35	24.3	17.8
TempVAE	12.64	5.96	4.8	1.3
DCC-GARCH-MVN	13.15	7.10	5.4	2.1
DCC-GARCH-MVt	10.23	4.14	3.9	0.6
HS	14.57	7.43	5.1	1.3

, the losses for the models as well as the average number of exceedances are displayed. The amount of average exceedances helps us to validate the VaR estimates by their 5% and 1% quantile assumption, and a good performance is a necessary condition. As the number of exceedances of a predicted ${\mathrm{VaR}}_t$ does not look at the size of the exceedances8, we have introduced RLF as an additional criterion to judge among those models that are good in estimating the average number of exceedances. As small numbers of RLF show that the ${\mathrm{VaR}}_t$ requires a high reserve when it is needed, a small RLF value is good. Therefore, although DCC-GARCH-MVt has the lowest values for the RLF95 and RLF99, the TempVAE produces the best estimates.

In Figure 5, the VaR95 values for the DCC-GARCH-MVN, HS and TempVAE are displayed. The blue line shows the original returns of the DAX portfolio. The full path of the VaR95 estimates as well as the VaR99 estimates can be found in Appendix F. We can see the VaR95 estimates for the DCC-GARCH-MVN and TempVAE are quickly responding to the changes in the volatility of the data. Comparing TempVAE directly to DCC-GARCH-MVN, we see the VaR estimates to be more volatile but also responsive to the evolution of the returns. The HS on the other side is showing the well-known time delay (see Liu 2005). Even though in terms of average exceedances, the HS is best followed by TempVAE, in our point of view, the TempVAE provides the most reasonable estimates among the models. This is particularly supported by the RLF values.

5. Conclusions

In this paper, we present a novel variational recurrent neural net with the well-known bottleneck structure of autoencoders to handle sequential data. As we model the data on the basis of a time-dependent prior, we call the model temporal variational autoencoder (TempVAE). By ensuring that the information has to flow through the bottleneck, we leverage the auto-pruning property of VAE. This way, we are able to find parsimonious model representations of the data without using methods such as grid search.

We successfully apply the TempVAE to risk management. More precisely, we estimate the risk measure VaR and show that our model performs competitively to the considered benchmark models. We provide a roughly unbiased estimation of VaR exceedances, showing the model is performing best in terms of the considered RLF scores.

Future research directions can incorporate applying the TempVAE architecture to other types of data sequences. Furthermore, the active dimensions in the bottleneck are not necessarily yielding disentangled representations. Research in this direction can foster the understanding of the auto-pruning and hence variational models further.