Estimating Value-at-Risk in the EURUSD Currency Cross from Implied Volatilities Using Machine Learning Methods and Quantile Regression

Round 1

Reviewer 1 Report

Referee Report for the Paper “Estimating Value at Risk from Implied Volatilities Using Machine Learning Methods”

Summary: The paper evaluates the value-at-risk models for the for the EURUSD exchange rate between 2009 and 2020 using ensemble models and neural networks. The main contribution of the paper is the finding that implied volatilities are helpful at explaining value-at-risk for the EURUSD exchange rate. My comments and suggestions are presented below in no particular order.

Major Comments:

1.      The abstract contains a sentence that sounds very vague: “Explanatory variables are implied volatilities with plausible economic interpretations.” Much more could be said about the economic interpretation of explicitly using measures of risk to study VaR both in the abstract and in the introduction. More generally, my point is that it would be helpful to the audience to provide a solid background for the importance of studying the value at risk in the context of current literature in finance and expand on the economic interpretation of the models with implied volatilities as explanatory variables. This should be done early in the paper to capture the attention of the reader. With the exception of three opening sentences in the introduction, both the Introduction and Literature Review sections are heavily focused on comparing various econometric models to estimate VaR. I’m concerned that we can lose sight of the forest for the trees. At the end of the day, the models are only supposed to serve us in answering an important theoretical and/or empirical question. From my reading of the Introduction, the importance of this question from the point of view of economic intuition and its treatment in the previous literature is not clearly explained.

 

2.      Along the same lines, value-at-risk is first defined on page 8 in Section 4.2. Since this is the main topic of the paper, its intuition and importance should be introduced on the onset of the paper in the Introduction. Moreover, the paper focuses on specific value-at-risk measure that pertains to the FOREX market, while the title of the paper fails to indicate that. I would consider changing the title of the paper to reflect the scope of the paper.

 

3.      At-the-money volatilities are first described in the data section. It would be helpful to add a basic definition of the variable and what it is intended to capture in the model.  

 

4.      After reading the paper, I was confused about the main contribution of the paper. The paper compares the two types of models for the EURUSD exchange rate between 2009 and 2020. However, as it is stated in the introduction and conclusions, the performance of the neural network models are “quite unstable” and improving them can lead to them becoming “more of a black box”. Instead of trying to evaluate a broad array of models in one paper, I would consider focusing on the models that work relatively better and add a few more currency pairs for major currencies. This would help to generalize the main result that ensemble models can be effective in predicting VaR in the FOREX market, which is less convincing with just one currency pair and one sample period. Neural network models may be a topic for a separate paper.

 

5.      The contribution of the paper that implied volatilities are helpful at explaining value-at-risk for the EURUSD exchange rate is very interesting. However, the result needs to be compared with other benchmark right-hand side variables that are conventional in the literature and tested on other currency pairs to be well established.

6.      The sample period includes highly volatile periods in 2009 and 2020. How does the performance of the models change if the financial crisis and the COVID-19 period is dropped from the sample?

 

7.      In addition, it would be helpful to add notes to each table.   

Comments for author File: Comments.pdf

Author Response

List of changes made to the manuscript: “Estimating Value at Risk in the EURUSD currency cross from implied volatilities using machine learning methods and quantile regression”.

We want to thank three anonymous reviewers for constructive comments to our manuscript. We have now carefully considered all issues raised by the reviewers. Our responses to the different issues are included in the revised manuscript and copied below. We do not repeat the full text provided by the reviewer, but extract what we think is the main criticism contained in each point.

Comments from Reviewer 1

1 and 2. The reviewer wants us to better explain the concept of VaR and our using measures of risk to study VaR in the introduction. The Reviewer suggests that we change the title of the paper to reflect the fact that we study FOREX markets.

Our response

On page 1 and 2 in the introduction, we write:

VaR is an estimate of the loss that will be exceeded with a small probability during a fixed holding period. It measures the worst attainable expected loss over a given time horizon at a given confidence level. The (parametric) VaR measure typically relies on the assumption that the associated portfolio (or investment position) is normally distributed, implicitly assuming normal market conditions. The normal assumption has been criticized in the literature, but need not be “too” wrong when portfolios are well diversified.  Another limitation with VaR is that if it is used as the objective in an optimization problem, the problem becomes non-convex, and one must use complicated numerical methods to find VaR efficient portfolios.

In addition to occupying a prominent role in regulatory frameworks, VaR will continue to be important for financial institutions as a measure of market risk. For instance, VaR could be used by banks to compute the amount of assets needed to cover losses.

We have changed the title of the paper to:

Estimating Value at Risk in the EURUSD currency cross from implied volatilities using machine learning methods and quantile regression

3. At-the-money volatilities are first described in the data section. It would be helpful to add a basic definition of the variable and what it is intended to capture in the model.

Our response

On page 2 in the introduction, we write:

This study utilizes market prices for option contracts on the EURUSD exchange rate quoted in the over-the-counter (OTC) foreign exchange (FX) interbank market. The models we employ use implied volatility metrics; ATM implied volatilities and Risk reversals, to forecast VaR. The ATM implied volatility is the risk-neutral expectation of spot rate volatility over the remaining life of the option. The risk reversal reflects the difference in the demand for out-of-the money options at high strikes compared to low strikes. Thus, it can be interpreted as a market-based measure of skewness, the most likely direction of the spot movement over the expiry period.

4. …, the performance of the neural network models are “quite unstable” and improving them can lead to them becoming “more of a black box”. Instead of trying to evaluate a broad array of models in one paper, I would consider focusing on the models that work relatively better and add a few more currency pairs for major currencies. This would help to generalize the main result that ensemble models can be effective in predicting VaR in the FOREX market, which is less convincing with just one currency pair and one sample period. Neural network models may be a topic for a separate paper.

Our response

We would like to note that in order to discover that the performance of the neural network models is quite unstable and improving them can lead to them becoming more of a black box, we needed to employ these models and run them on our data sets. The fact that these models do not perform satisfactorily is a result of our study.

Further, on page 2 in the introduction, we write:

From our literature study on machine learning methods presented in section 2 below, we discovered that both neural networks and ensemble methods had been successfully used for VaR predictions. We therefore elected to employ both neural networks and ensemble methods for VaR predictions.

We certainly agree with the reviewer in that similar analysis for other currency pairs beyond EURUSD is interesting in the context of the research question addressed in the paper. However, we do not currently have access to the required data for other currency pairs. We hope to obtain data in the future, which will enable us to investigate other currencies in future research.

5. The contribution of the paper that implied volatilities are helpful at explaining value-at-risk for the EURUSD exchange rate is very interesting. However, the result needs to be compared with other benchmark right-hand side variables that are conventional in the literature and tested on other currency pairs to be well established.

Our response

On page 26, we write:

In Table 7, all the models are summarized with their Christoffersen test p-values and DQ-test p-values with four lags. For further comparison, we report results for a set of benchmark models frequently employed in the literature. GARCH models can accommodate a wide range of assumptions with regards to the distribution of residuals. Filtered historical simulation (FHS) was introduced by Barone-Adesi et al (2008). Here, the conditional distribution of residuals is derived from the empirical distribution of standardized returns. McNeil and Frey (2000) suggest fitting a GARCH model to the time series or returns and then applying EVT to the standardized residuals. We refer to these approaches as FHS-GARCH and EVT-GARCH, respectively. The CAViaR methodology from Engle and Manganelli (2004) models VaR as an autoregressive process. We apply the symmetric absolute value specification (CAViaR-SAV), in which the estimated VaR responds symmetrically to the absolute value of realized returns. Furtermore, we compute VaR estimates from the GJR-GARCH model from Glosten et al (1993), which allows for the asymmetric response of conditional volatility to negative and positive returns through the leverage parameter.

We also refer the reviewer to table 7.

6. The sample period includes highly volatile periods in 2009 and 2020. How does the performance of the models change if the financial crisis and the COVID-19 period is dropped from the sample?

 

Our response:

We have deliberately included data covering extreme events such as the global financial crises (2007-2008), including the sovereign debt crises in the EMU following the global financial crises, the slump in oil prices in 2014 and the 2019-2020 COVID 19 pandemic. It is of vital importance to examine how forecasting models can handle such events.

7. In addition, it would be helpful to add notes to each table.

Our response:

We have now added notes to selected tables.

Author Response File: Author Response.pdf

Reviewer 2 Report

1. The author needs to give more detail and explain why they prefer to use ensemble methods and neural networks to predict the VaR of currency.

 

2. The author needs to confirm that all time series data are stationary by proceeding with the unit root test.

 

3. The VaR may not be good enough to quantify the risk of a portfolio; the author needs to calculate the ES (Expected Shortfall) as well. The ES is to provide more information for investors to better understand the risk of their investment.

 

4. The authors need to find out more citations about VaR vs. ES.

 

 

 

 

Forexample,

 

- https://www.mdpi.com/2227-7099/9/1/13

 

- https://www.sciencedirect.com/science/article/abs/pii/S0378426604001499

 

- https://www.scopus.com/record/display.uri?eid=2-s2.0-85037852427&origin=inward&txGid=b0a127f69bce89709c951aaab3c7fd0d

 

Minor editing of English language required

Author Response

List of changes made to the manuscript: “Estimating Value at Risk in the EURUSD currency cross from implied volatilities using machine learning methods and quantile regression”.

We want to thank three anonymous reviewers for constructive comments to our manuscript. We have now carefully considered all issues raised by the reviewers. Our responses to the different issues are included in the revised manuscript and copied below. We do not repeat the full text provided by the reviewer, but extract what we think is the main criticism contained in each point.

Comments from reviewer 2

1. The reviewer remarks: The author needs to give more detail and explain why they prefer to use ensemble methods and neural networks to predict the VaR of currency.

 

Our response

On page 2 in the introduction, we write:

From our literature study on machine learning methods presented in section 2 below, we discovered that both neural networks and ensemble methods had been successfully used for VaR predictions. We therefore elected to employ both neural networks and ensemble methods for VaR predictions.

2. The reviewer remarks: The author needs to confirm that all time-series data are stationary by proceeding with the unit root test.

Our response

On page 6, we write:

In order to test the data for stationarity we performed an augmented Dickey-Fuller unit root test and conclude that our data is stationary. Further to rule out multicollinearity, we ran a variance inflation factor test, or VIF in short. No sign of multicollinearity was discovered. We also tested our data for heteroskedasticity using the Breusch-Pagan test and serial correlation between the explanatory variables using a Breusch-Godfrey test. The Breusch-Pagan test indicated some sign of heteroskedasticity, whereas no sign of serial correlation was found.

3. The reviewer remarks: The VaR may not be good enough to quantify the risk of a portfolio; the author needs to calculate the ES (Expected Shortfall) as well. The ES is to provide more information for investors to better understand the risk of their investment.

Our response

On page 2, we write:

Expected Shortfall (ES) is a risk-measure closely related to the VaR. ES is often labeled conditional VaR or tail risk. The VaR metric assignes a 100% weighting to the Qth quantile and zero to other quantiles. Expected shortfall, however, gives equal weight to all quantiles greater than the Qth quantile and zero weight to all quantiles below the Qth quantile. In certain situations, ES gives traders in financial markets better incentives to control risk than does the VaR measure. We have however elected to employ the VaR measure, since typically traders are constrained from constructing portfolios with excessive tail risk by detailed mandates of which the VaR metric typically is the important component.

4. The reviewer remarks: The authors need to find out more citations about VaR vs. ES.

Our response:

We have added three references:

Chaiboonsri, C & Satawat W. (2021). Applying Quantum Mechanics for Extreme Value Prediction of VaR and ES in the ASEAN Stock Exchange. Economies 9: 13. https://doi.org/10.3390/ economies9010013

Yamai, Y & Yoshiba, T. (2005) Value at risk versus Expected Shortfall: A practical perspective. Journal of Banking and Finance, Vol 29, Issue 4, pages 997-1015.

Also, McNeil et al employs both VaR and conditional VaR, i.e., Expected Shortfall (ES), when fitting a GARCH model to time series or returns:

McNeil, A. J., and Frey, R. (2000). Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach. Journal of Empirical Finance 7(3–4), 271–300 (https://doi.org/10.1016/S0927-5398(00)00012-8).

Author Response File: Author Response.pdf

Reviewer 3 Report

Please see the attached

Comments for author File: Comments.pdf

Only minor editing of English language required

Author Response

List of changes made to the manuscript: “Estimating Value at Risk in the EURUSD currency cross from implied volatilities using machine learning methods and quantile regression”.

We want to thank three anonymous reviewers for constructive comments to our manuscript. We have now carefully considered all issues raised by the reviewers. Our responses to the different issues are included in the revised manuscript and copied below. We do not repeat the full text provided by the reviewer, but extract what we think is the main criticism contained in each point.

Introductory comments from reviewer 3:

My general comment is that although the presented empirical study may potentially provide useful insights into the forecasting capabilities of selected machine learning models, the contribution made by the Authors to the problem of proper selection of such models and techniques for estimating VaR is in my opinion rather limited.

Our response:

We have not applied a quantitative search method to select models from the on-line literature. We have selected models based on a significant number of relevant papers we have read, some of which are reported in our literature study. On page 2 in the introduction, we write:

“From our literature study on machine learning methods presented in section 2 below, we discovered that both neural networks and ensemble methods had been successfully used for VaR predictions. We therefore elected to employ both neural networks and ensemble methods for our VaR predictions.”

Main comments from reviewer 3

1. The reviewer remarks:

 

Although it is of interest to understand whether modern machine learning techniques can be successfully applied for predicting VaR in the context of particular time series, it is important that the Authors explain merits of using machine learning approaches when compared with at least some of the existing nonlinear quantile regression models.

 

Our response

In order to better explain merits of using machine learning approaches, we have included additional benchmark models frequently employed in the ML literature.

On page 26, we write:

“In Table 7, all the models are summarized with their Christoffersen test p-values and DQ-test p-values with four lags. For further comparison, we report results for a set of benchmark models frequently employed in the literature. GARCH models can accommodate a wide range of assumptions with regards to the distribution of residuals. Filtered historical simulation (FHS) was introduced by Barone-Adesi et al (2008). Here, the conditional distribution of residuals is derived from the empirical distribution of standardized returns. McNeil and Frey (2000) suggest fitting a GARCH model to the time series or returns and then applying EVT to the standardized residuals. We refer to these approaches as FHS-GARCH and EVT-GARCH, respectively. The CAViaR methodology from Engle and Manganelli (2004) models VaR as an autoregressive process. We apply the symmetric absolute value specification (CAViaR-SAV), in which the estimated VaR responds symmetrically to the absolute value of realized returns. Furtermore, we compute VaR estimates from the GJR-GARCH model from Glosten et al (1993), which allows for the asymmetric response of conditional volatility to negative and positive returns through the leverage parameter.”

We also refer the reviewer to table 7.            

2. The reviewer suggests including two additional references.

Our response

We have studied and included the references suggested by the reviewer.

3. The reviewer claims that since we generate data using the QR-IM model, … “then

the best we can achieve when using a machine learning model is to recover the linear mapping as represented by (4)” i.e., the QR-IM model.

Our response

We respectfully disagree. On page 8, section 4.1. we write:

“The purpose of this study is to investigate the ability of a set of machine learning models to provide accurate VaR-estimates out-of sample. This requires a training dataset from which the ML models can learn the relationship between the conditional return distribution for the  variables and the explanatory variables, the latter being at-the-money volatility and the risk reversal in this study. We apply two different approaches to generate the conditional return distribution. First, we employ the methods proposed by de Lange et al. (2022). Here we use the QR-IM model to estimate the return quantiles, conditional on at-the-money volatility and risk reversal values in the training sample. This ensures that the ML methods are trained on a basis directly comparable to the linear QR-IM model. By virtue of this particular approach, we ensure that any relative performance advantage of the ML methods is due to their ability to capture non-linearities. We refer to this as the QR-IM generated data set. Second, we rely on gradient boosting; referred to as the Gradient boost and LGBM datasets, respectively.”

Additional comments from reviewer 3

1. In several places, the presented definitions and concepts are not properly explained.

 

Our response:

 

We have carefully reviewed all mathematical equations and revised these according to the reviewer’s remarks.

 

2. It would be helpful to have an explicit formulation of the criterion (or criteria) that the Authors have used to train the machine learning models.

 

Our response

 

On page 12, we write:

The process of adjusting the layer weights until the network satisfactorily approximates the target function is known as training the neural network. The function that the neural network is at- tempting to approximate is (arbitrarily) denoted by F. An objective function, frequently referred to as the loss function, is used to measure how closely the network approximates F. When the neural network accurately approximates F, the loss function has a global minimum. It calculates the distance between the output yˆ to the target output y.

One of the most commonly used loss functions, which is employed in this study, is the mean squared error (MSE). It is defined as: (see attached file for equation)

3. Some subsections in Section 4 are not presented in the order suggested by their numbers. More generally, the organization of this whole section should be improved.

 

Our response

 

We apologize for this inconvenience. All sections and subsections should now be correctly numbered.

 

4. Updating references

 

Our response

 

We have now checked all references and updated some links.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

n/a

Author Response

Thank you very much!

Reviewer 3 Report

No further comments

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 3

Reviewer 3 Report

No further comments