Calibrating FBSDEs Driven Models in Finance via NNs

Abstract

The curse of dimensionality problem refers to a set of troubles arising when dealing with huge amount of data as happens, e.g., applying standard numerical methods to solve partial differential equations related to financial modeling. To overcome the latter issue, we propose a Deep Learning approach to efficiently approximate nonlinear functions characterizing financial models in a high dimension. In particular, we consider solving the Black–Scholes–Barenblatt non-linear stochastic differential equation via a forward-backward neural network, also calibrating the related stochastic volatility model when dealing with European options. The obtained results exhibit accurate approximations of the implied volatility surface. Specifically, our method seems to significantly reduce the neural network’s training time and the approximation error on the test set.

1. Introduction

The curse of dimensionality problem refers to a set of troubles that arise when dealing with big data. Classical methods for solving partial differential equations suffer from this problem, especially in the financial field (Bayer and Stemper 2018; Wang 2006; Weinan et al. 2019). For example, we can think of the non-linear Black–Scholes equation for pricing financial derivatives in which the number of considered underlying assets gives the dimensionality of the Partial Differential Equation (PDE). The latter scenario, from a purely mathematical point of view, typically translates in the need of approximating nonlinear functions in a high dimension. An effective solution can be obtained exploiting a Deep Learning approach, namely using a particular type of Machine Learning (set of) solutions. For the sake of completeness, let us recall that Machine Learning (ML) is the branch of computer science in which mathematical and statistical techniques are used to give computer systems the ability to learn through data. There are two main branches of ML algorithms: Supervised Learning and Unsupervised Learning. The former use labeled datasets while the latter use unlabeled data. The increase of available data in recent years and the need to analyze them, finds in Machine Learning the ideal tool. Indeed, the peculiar features and the change of programming paradigm of the latter, make it the best candidate to solve problems with huge number of data in different fields. For example, computer vision, natural language processing, time series analysis, differential equation. Many papers have been published with the aim of improving theoretical and empirical knowledge; see, among others, Germain et al. (2021); Han (2016) and (Carleo and Troyer 2017). This fame inspires speculations that Deep Learning might hold the key to solving the curse of dimensionality problem.

Neural networks (NNs) are the backbone of deep learning algorithms and their use to solve real-world problems. Indeed, even if their conceptual development can be dated back to several years ago, only during last few years, particularly since powerful hardware became economical affordable, they proved to be concretely and efficiently applied to fruitfully solve complex tasks dealing with huge amount of data. The latter is the case, e.g., of NNs-steered solution to forecast financial scenarios, aggregate energy consumption, social events, etc.

In this paper, we aim at showing how to use NNs toward two financial connected goals: find a solution of a particular non-linear stochastic differential equation (the Black–Scholes–Barenblatt one) via a forward-backward neural network and to calibrate a stochastic volatility model (the Heston model) using a NN-structure for regression. In both cases, we deal with European option-based pricing models. The two goals are strictly connected from a financial point of view, because the first one provides a solution to the option pricing perspective, while the second covers the model calibration task. Even if they seem different from a mathematical point of view, we underline that the specific features of the proposed methodology allow us to solve them with similar Neural Network specification.

The rest of the paper is divided as follows: Section 2 and Section 3 provide an overview of Backward Stochastic Differential Equations (BSDE) and Neural Networks, respectively, while Section 4 is devoted to depicting the link between BSDEs and NNs, to analyze the type of NN that we choose and why we have considered this specific option. In Section 5, we exploit the obtained findings to calibrate the above mentioned rough stochastic volatility model with NN and in Section 6 we show the empirical results. Finally, Section 7 states our conclusions with possible outlooks. Finally, Appendixes Appendix A and Appendix B collect the main notation and definitions respectively.

2. Backward Stochastic Differential Equations

BSDEs were first introduced in Bismut (1973), where the author described them in connection with stochastic control problems. Then, stochastic control cases have been widely investigated during the 1970s by Bismut himself, as well as by, e.g., Bensoussan; Haussmann; and Kushner, particularly under no control on the diffusion coefficient, while Pardoux and Peng gave a full presentation in Mao (1995). Nowadays, the theory of the hedging and pricing of a contingent claim is typically expressed in terms of a linear Backward Stochastic Differential Equation (BSDE) of Ito’s type. Along the latter lines, let us start defining a filtered probability space, endowed with the usual conditions $(\Omega ,\left({\mathcal{F}}_t\right),\mathbb{F},\mathbb{P})$, where the filtration $\left({\mathcal{F}}_t\right)$ is the one generated by a standard, real-valued, $d-$ dimensional, $d\ge 1$, Brownian motion $W=\{W_t:t\ge 0\}$, defined on $\Omega$. Accordingly, we define a linear BSDE as follows:

$$\begin{array}{c}\hfill \left\{\begin{array}{c}-d{Y}_t=+f(t,Y_t,Z_t)dt-Z_{t}d{W}_t,t\in [0,T]\hfill \\ Y_T=\zeta \in L^p(\Omega ,{\mathcal{F}}_T,\mathbb{P}),\hfill \end{array}\right.\end{array}$$

(1)

or, equivalently,

$$Y_t=\zeta +{\int }_t^{T}f(s,Y_s,Z_s)ds-{\int }_t^T{Z}_{s}d{W}_s.$$

The solution of (1) is a pair of processes $(Y,Z)$ such that Y is continuous and adapted, and Z is predictable and satisfies ${\int }_0^T{\left|Z_s\right|}^{2}ds<\infty$ a.s.

The parameters f and $\xi$ are called standard parameters if:

• $\zeta$$\in L_T^2\left({\mathbb{R}}^d\right)$, where $L_T^2\left({\mathbb{R}}^d\right)$ is the space of ${\mathbb{F}}_t$-measurable random variables $\xi$ s.t. $\mathbb{E}\left(\right|\xi {|}^2)<\infty$
• $f(t,0,0)\in H_T^2\left({\mathbb{R}}^d\right)\forall t\in [0,T]$, where $H_T^2\left(R^d\right)$ is the space of predictable process Y s.t. ${\left|\right|Y\left|\right|}^2=\mathbb{E}({\int }_0^T|Y_t{|}^{2}dt)<\infty )$
• f is uniformly Lipschitz: there exists L s.t.

  $$|f(t,y_1,z_1)-f(t,y_2,z_2)|\le L(|y_1-y_2|+|z_1-z_2\left|\right)$$

  for any $y_1,y_2,z_1,z_2$.

Theorem 1. 

Given the standard parameters, the BSDE (1) has a unique solution $(Y,Z)$ in $H_T^2\left({\mathbb{R}}^d\right)\times H_T^2\left({\mathbb{R}}^{d\times n}\right)$.

See El Karoui et al. (1997) for the proof.

Forward–Backward Stochastic Differential Equation

As in the previous case, let us start considering a standard probability space $(\Omega ,\mathcal{F},\mathbb{P})$, and a d-dimensional Brownian motion $W=\{W_t:t\ge 0\}$, where $\left\{{\mathcal{F}}_t\right\}=\sigma \{W_S:0\le s\le t\}$. Then, we define a Forward–Backward Stochastic Differential Equation (FBSDE), as follows:

$$\left\{\begin{array}{c}{X}_t=x+{\int }_0^{t}b(s,X_s,Y_s,Z_s)ds+{\int }_0^t\sigma (s,X_s,Y_s)d{W}_s\hfill \\ Y_t=g\left(X_T\right)+{\int }_t^T\widehat{b}(s,X_s,Y_s,Z_s)ds+{\int }_t^T\widehat{\sigma }(s,X_s,Y_s,Z_s)d{W}_s,\hfill \end{array}\right.$$

(2)

where the processes X, Y and Z are in ${\mathbb{R}}^n,{\mathbb{R}}^d$ and ${\mathbb{R}}^{d\times n}$.

Definition 1. 

A triple of stochastic processes $(X,Y,Z):[0,T]\times \Omega \to {\mathbb{R}}^n\times {\mathbb{R}}^d\times {\mathbb{R}}^{d\times n}$ is called an ordinary adapted solution of FBSDE (2) if it is $\left\{{\mathcal{F}}_t\right\}$-adapted, square integrable and satisfies (2), $\mathbb{P}$-a.s.

We can also rewrite the system of Equation (2) in a pure forward differential form as follows:

$$\left\{\begin{array}{c}d{X}_t=b(t,X_t,Y_t,Z_t)dt+\sigma (t,X_t,Y_t)d{W}_t\hfill \\ d{Y}_t=-\widehat{b}(t,X_t,Y_t,Z_t)dt-\widehat{\sigma }(t,X_t,Y_t)d{W}_t\hfill \\ Y_T=g\left(X_T\right),X_0=x.\hfill \end{array}\right.$$

(3)

also called a stochastic two-point boundary value problem.

In Pardoux and Tang (1999), it is shown how a couple of FBSDEs of (3)-type are connected to:

$$u_t=\tilde{f}(t,x,u,Du,D^{2}u),$$

(4)

a quasi-linear partial differential equation with terminal condition given by $u(T,x)=g\left(x\right)$ and where $u(t,x)$ is the unknown solution. The function $f(t,x,u,Du,D^{2}u)$ is:

$$\tilde{f}(t,x,u,Du,D^{2}u)=\phi (t,x,u,Du)-\mu {(t,x,u,Du)}^{\prime }Du-\frac{1}{2}Tr\left[\sigma (t,x,u)\sigma {(t,x,u)}^{\prime }{D}^{2}u\right],$$

where $Du$, resp. $D^{2}u$, is the gradient, resp. the Hessian matrix, of u. Then, by the Ito’s lemma, solutions to (3) and (4), are related by:

$$Y_t=u(t,X_t)Z_t=Du(t,X_t).$$

(5)

3. Neural Networks

An Artificial Neural Network (ANN) is a mathematical computing system inspired to the biological neural network of mammals brain. It is composed by nodes and neurons that learns from the input data, through a training process, and use information to set the parameters; the latters are called weights. Mathematically speaking, a Neural Network can be formalized as a directed graph in which the neurons are nodes and every node receives signals from the external environment or from other neurons. An activation function is contained in every neuron and aims to regulate and propagate signals throughout the network; see Sharma et al. (2020) for further details. The connections topology plays a crucial role in neural networks: we can distinguish the connection structure between Feedforward Neural Network and Recurrent Neural Network. Indeed, the former can be represented by a directed acyclic graph, while the latter has feedback connections and cycles.

In the late 1980s, the first multilayer feedforward neural network was built (Svozil et al. 1997) and presents a sequence of neuron layers connected in a cascade. This type of neural network belongs to the larger family of Deep Neural Networks (DNN). One of the main goal of DNN, is to build a general tool to approximate continuous functions, and this took shape by the Universal Approximation Theorem (Cybenko 1989):

Theorem 2. 

Let $N\left(\sigma \right)$ denote the space of functions that a fully connected neural network with activation function $\sigma :\mathbb{R}\to \mathbb{R}$, a single hidden layer with a finite number of neurons $l\in \mathcal{N}$ and a linear output layer can represent, that is,

$$\begin{array}{c}\hfill N\left(\sigma \right)=\left\{f:\mathbb{R}\to \mathbb{R}|f\left(x\right)=\sum _{i=1}^l{w}_i\sigma \left(\sum _{j=1}^d{\overline{w}}_j^{\left(i\right)}{x}_j+{\overline{b}}^{\left(i\right)}\right)+b_i\right\}\\ \hfill \mathrm{for}\mathrm{some}w,b\in {\mathbb{R}}^l\mathrm{and}{\overline{w}}^{\left(i\right)},{\overline{b}}^{\left(i\right)}\in \mathbb{R},1\le i\le l\end{array}$$

Assuming that the activation function $\sigma :\mathbb{R}\to \mathbb{R}$ is non-constant, unbounded and continuous, $N\left(\sigma \right)$ is dense in $C\left(X\right)$ for compact $X\subseteq \mathbb{R}$ in the uniform topology.

See Cybenko (1989) for the proof of Theorem 2.

The main components of Deep Neural Networks are the following:

• input layer composed by n units, where n is the number of network entrances (n is the dimension of the features’ vector);
• hidden layer with $L-1,L\ge 2$ neurons and outputs connected with the inputs of the following layer;
• output layer composed by $K\ge 1$ neurons that describe the network outputs;
• A set of directed and weighted edges (called w) that represent all possible connections among layers.

Figure 1 describes an example of a deep neural network.

Every node, no matter about the layer it belongs to, is considered as an operator performing a weighted sum of its inputs (possibly of multi-dimensional type), also adding possible sources of noise (usually described by a white noise or a standard Brownian motion), to then obtain a non-linear composition by mean of a scalar function $\sigma :\mathbb{R}\to \mathbb{R}$. If we consider an input $x\in {\mathbb{R}}^d,d\in \mathcal{N}$, the output of every node is given by:

$$y=\sigma (w^{T}X+b),$$

where the function $\sigma$ is the so-called activation function, usually assumed to be non-decreasing, continuous and bounded, see, e.g., Sharma et al. (2020) for details.

Deep neural network techniques require an initial training procedure to find the optimal weights $\omega \in {\mathbb{R}}^n$. This is usually conducted by combining the backpropagation algorithm (Rumelhart et al. 1986) for the computation of loss function’s gradient and the gradient descent algorithm proposed by Augustin-Louis Cauchy in 1847, to update the network weights. The gradient descent algorithm requires the computation of the derivatives of the loss function with respect to all the parameters: this could be a problem if we are working with a large number of features because all the computations take too long. To avoid this, Robbins and Monro (1951) proposed the Stochastic Gradient Descent (SGD) that uses only a single sample to perform each iteration. However, the path of the algorithm is usually noisier than the standard Gradient Descent algorithm, but (Kingma and Ba 2014) proposed Adam optimizer that is able to solve this issue combining two extension of classic SGD: Momentum and Root Mean Square Propagation.

4. Solution of FBSDEs via NNs

Let us consider the following FBSDE:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill d{X}_t& =\mu (t,X_t,Y_t,Z_t)dt+\sigma (t,X_t,Y_t)d{W}_t,t\in [0,T]\hfill \\ \hfill X_0& =\xi \hfill \\ \hfill d{Y}_t& =\phi (t,X_t,Y_t,Z_t)dt+Z_t^{\prime }\sigma (t,X_t,Y_t)d{W}_t,t\in [0,T]\hfill \\ \hfill Y_T& =g\left(X_T\right),\hfill \end{array}\end{array}$$

(6)

where $W_t$ is a vector-valued standard Brownian motion. By (5), the solution to (6) can be expressed as a deterministic function of time under state process X, in particular, via Itô’s formula, we have:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \frac{\partial u_j}{\partial t}=\phi (t,x,u,Du)-\mu {(t,x,u,Du)}^{\prime }Du-\frac{1}{2}Tr\left[\sigma (t,x,u)\sigma {(t,x,u)}^{\prime }{D}^{2}u\right]\hfill \\ \hfill & u(T,x)=g\left(x\right),\hfill \end{array}\end{array}$$

(7)

where Du is the gradient vector and $D^{2}u$ denotes the Hessian matrix. Our aim is to approximate the solution u of the latter equation applying the automatic differentiation method, see (Baydin et al. 2018), as to approximate the gradient vector Du(t,x). We start discretizing Equation (6) by the standard Euler-Maruyama scheme:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill X^{n+1}& \approx X^n+\mu (t^n,X^n,Y^n,Z^n)dt+\sigma (t^n,X^n,Y^n)d{W}^n\hfill \\ \hfill Y^{n+1}& \approx Y^n+\mu (t^n,X^n,Y^n,Z^n)dt+{\left(Z^n\right)}^{\prime }\sigma (t^n,X^n,Y^n)d{W}^n,\hfill \end{array}\end{array}$$

(8)

where $n=0,1,\dots ,N-1$, $\Delta t^n:=t^{n+1}-t^n=T/N$ and $\Delta W^n\sim \mathbb{N}(0,\Delta t^n)$ is a random variable with mean 0 and standard deviation given by $\sqrt{\Delta t^n}$.

We consider the following loss function:

$$\sum _{m=1}^M\sum _{n=0}^{N-1}|Y_m^{n+1}-Y_m^n-{\Phi }_m^n\Delta t^n-{\left(Z_m^n\right)}^{\prime }{\Sigma }_m^n\Delta W_m^n{|}^2+\sum _{m=1}^M{|Y_m^N-g\left(X_m^N\right)|}^2.$$

(9)

Here, M describes the number of different realizations of the underlying Brownian motion. Furthermore we consider:

$${\Phi }_m^n:=\phi (t^n,X^n,Y^n,Z^n),{\Sigma }_m^n:=\sigma (t^n,X_m^n,Y_m^n),$$

in which m represent the m-th realization of the underlying Brownian motion and n the time $t^n$. From Equation (5) we have:

$$Y_m^n=u(t^n,X_m^n),Z_m^n=Du(t^n,X_n^m),$$

from which we derive that the loss Function (9) is a function of the parameters of the neural network $u(t,x)$.

5. Model Calibration via NN

A fundamental step in developing concrete forecasting solutions, mainly because of lack of explicit solutions, is the so called model calibration (set of) procedure(s). The latter can be seen as the search of the parameters’ values optimizing the adherence of the model output to real data. Within such scenario, in what follows we provide an efficient neural network based model aiming at calibrating the Heston model, see (Heston 1993), parameters based on specific time series.

The Heston model is a stochastic volatility model that describes both the dynamics of the asset price $S={\left(S_t\right)}_{t\in [0,T]}$ and the instantaneous variance process $\nu ={\left({\nu }_t\right)}_{t\in [0,T]}$. Specifically:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill d{S}_t& =r{S}_{t}dt+\sqrt{v_t}{S}_{t}d{\tilde{W}}_t^{\left(S\right)},t\in [0,T]\hfill \\ \hfill d{\nu }_t& =\lambda (\overline{\nu }-{\nu }_t)dt+\eta \sqrt{{\nu }_t}d{\tilde{W}}_t^{\left(V\right)},t\in [0,T].\hfill \end{array}\end{array}$$

(10)

${\tilde{W}}^{\left(S\right)}$ and ${\tilde{W}}^{\left(V\right)}$ are two correlated Brownian Motion with correlation parameter $\rho$, $\lambda$ is the variance mean reversion speed, $\overline{\nu }$ is the long term variance, ${\nu }_0$ is the initial value of variance, and $\eta$ represent the volatility of variance.

Since we do not have an analytical solution for the proposed problem, we might solve the weighted non-linear least squares problem using Levenberg–Marquardt algorithm (Moré 1978). The trouble of this algorithm is the time consuming and the labour intensive that make efficient calibration prohibitively expensive. We avoid this problem exploiting a Neural Networks based approach. In particular, we first build a multi-layered artificial neural network, in the spirit of Bayer and Stemper (2018), to then improve accurancy exploiting a Long Short Term Memory (LSTM) solution. All NNs are used to closely approximate the function $\phi$ through input-output pairs ${\left\{(x_i,\phi \left(x_i\right))\right\}}_{i=1}^N$ called labeled data.

5.1. Calibration Objective

We consider the Heston model in (10) with model parameters $\mu \in \mathcal{M}\subseteq {\mathbb{R}}^m$ and market information $\xi \in \epsilon \subseteq {\mathbb{R}}^k$ (A10). Let the market implied volatility (IV) quotes $Q:={\left(Q(M^{\left(1\right)},T^{\left(1\right)}),\dots ,Q(M^{\left(N\right)},T^{\left(N\right)})\right)}^T\in {\mathbb{R}}^N$ of N European options with moneyness $M^{\left(i\right)}$ and time to maturity $T^{\left(i\right)}$. The corresponding model IV quotes are:

$$\phi (\mu ,\xi ):={\left(\phi \left(\mu ,\xi ,M^{\left(1\right)},T^{\left(1\right)}\right),\dots ,\phi \left(\mu ,\xi ,M^{\left(1\right)},T^{\left(1\right)}\right)\right)}^T\in {\mathbb{R}}^N.$$

We define the residual function $\mathbb{R}\left(\mu \right):\mathcal{M}\mapsto {\mathbb{R}}^N$ as

$$R\left(\mu \right):=\phi (\mu ,\xi )-Q.$$

(11)

The calibration objective is achieved by minimizing the following quantity:

$${\mu }^{*}=\underset{\mu \in \mathcal{M}}{argmin}{\left|\left|W^{\frac{1}{2}}R\left(\mu \right)\right|\right|}_2^2=\underset{\mu \in \mathcal{M}}{argmin}{\left|\left|W^{\frac{1}{2}}[\phi (\mu ,\xi )-Q]\right|\right|}_2^2:=\Psi (W,\xi ,Q),$$

(12)

where $W=diag[{\omega }_1,\dots ,{\omega }_N]\in {\mathbb{R}}^{N\times N}$ is the weight matrix.

This is a non-linear optimization problem in the parameters $\mu \in \mathcal{M}\subseteq {\mathbb{R}}^m$ and it might be solved by Levenberg–Marquardt (LM) algorithm.

Proposition 1 

(LM calibration). Suppose $R:O\to {\mathbb{R}}^N$ be a twice differentiable function on an open set $O\subseteq {\mathbb{R}}^m$ and $N>m$. Denote by $J:O\to {\mathbb{R}}^{N\times m}$ the Jacobian of R w.r.t. $\mu \in {\mathbb{R}}^m$:

$${\left[J_{ij}\right]}_{\stackrel{1\le i\le N}{1\le j\le m}}={\left[\frac{\partial R_i\left(\mu \right)}{\partial {\mu }_j}\right]}_{\stackrel{1\le i\le N}{1\le j\le m}}={\left[\frac{\partial {\phi }_i(\mu ,\xi )}{\partial {\mu }_j}\right]}_{\stackrel{1\le i\le N}{1\le j\le m}}.$$

The algorithm starts with a value ${\mu }_0\in {\mathbb{R}}^m$. At each iteration with the current estimation ${\mu }_k\in {\mathbb{R}}^m$ the update ${\Delta }_{\mu }\in {\mathbb{R}}^m$ solves:

$$[J{\left({\mu }_k\right)}^{T}WJ\left({\mu }_k\right)+\lambda I_m]{\Delta }_{\mu }=J{\left({\mu }_k\right)}^{T}WR\left({\mu }_k\right),$$

(13)

where $I_m\in {\mathbb{R}}^{m\times m}$ denotes the identity and $\lambda \in \mathbb{R}$.

As mentioned above, finding the solution of (13) in a stochastic volatility setting is not an easy task: both the IV map $\varphi :\mathcal{I}\to {\mathbb{R}}_{+}$ and its Jacobian $J:O\to {\mathbb{R}}^{N\times M}$ are unknown in analytical form. For this reason, we propose a deep calibration procedure.

5.2. Deep Calibration Procedure

The IV map $\tilde{\phi }:\mathcal{I}\to {\mathbb{R}}^{+}$, see Definition A11, is approximated with the purpose of generating a set of labeled data $\mathcal{D}:={\left\{\left(x^{\left(i\right)},\tilde{\phi }\left(x^{\left(i\right)}\right)\right)|x\in \mathcal{I}\right\}}_{i=1}^n\in {(\mathcal{I}\times {\mathbb{R}}_{+})}^n,n\in \mathbb{N}$. We denote this function ${\phi }_{NN}:\mathcal{I}\to {\mathbb{R}}^{+}$. In this case, the neural network is used as a regression tool to learn the implied volatility map $\phi$: the NN’s degree of accuracy must be as greater as possible, and it is related to the data set $\mathcal{D}={\left\{\left({\mu }^{\left(i\right)},{\xi }^{\left(i\right)},M^{\left(i\right)},T^{\left(i\right)},\tilde{\phi }\left({\mu }^{\left(i\right)},{\xi }^{\left(i\right)},M^{\left(i\right)},T^{\left(i\right)}\right)\right)\right\}}_{i=1}^n$$\in {(\mathcal{I}\times {\mathbb{R}}_{+})}^n,$$n\in \mathbb{N}$. We aim to proxy the liquidity by inverse bid-ask spread of traded European Call Options on S&P 500 and then run a KDE (Kernel Density Estimation) on samples ${\{M^{\left(i\right)},T^{\left(i\right)}\}}_{i=1}^n$ using a smoothing bandwidth and weights ${\left\{L_i\right\}}_{i=1}^n$. We assume to do not have prior knowledge on the model parameters $\mu$, and we sample the from the uniformly continuous marginal distribution.

As a second step, we proceed with backpropagation. Let $L\in \mathbb{N}$ the number of hidden layers, $n_j,1\le j\le L$ the number of nodes for each hidden layers and ${\mathcal{S}}_{h_{model}}$ the function space spanned by the NN with model parameters given by $h_{model}=(L,n_1,\dots ,n_L)$. We denote by $X:\Omega \to \mathcal{I}$ a random input and consider $h_{model}$ fixed; we have to learn the function that minimizes the generalization error:

$$f_{h_{model}}^{*}=\underset{f_{h_{model}}\in {\mathcal{S}}_{h_{model}}}{argmin}=\left|\right|f_{h_{model}}\left(X\right)-\tilde{\phi }\left(X\right){\left|\right|}_{L^2(\Omega )}^2,X\sim \mathcal{G},$$

(14)

where $\mathcal{G}\in \left\{{\mathcal{G}}_{Heston}\right\}$. In practical training, we can rewrite (14) as follows:

$$f_{h_{model}}^{*}=\underset{f_{h_{model}}\in {\mathcal{S}}_{h_{model}}}{argmin}\frac{1}{n_{valid}}\sum _{i=1}^{n_{valid}}{\left[f_{h_{model}}\left(x^{\left(i\right)}\right)-\tilde{\phi }\left(x^{\left(i\right)}\right)\right]}^2.$$

(15)

We adopt the Adam optimizer in order to prevent the so called zigzag phenomenon of Gradient Descent in long and sharp valleys of the error surface. With this optimization procedure we end up with the hyper parameters $h_{opt}=\{\gamma ,\beta \}$. Here, $\gamma$ is the global warning rate and $\beta$ is the mini-batch size used. From now on, ${\mathcal{A}}_{h_{opt}}:{\mathcal{I}}^n\to {\mathcal{S}}_{h_{model}}$ denotes the Adam optimizer that maps ${\mathcal{D}}_{train}$ to a local minimizer $f_{h_{model}}^{*}$.

Let us define a hyperparameter response function $\mathcal{H}$ as:

$$\mathcal{H}(h_{opt},h_{model})=\frac{1}{n_{valid}}\sum _{i=1}^{n_{valid}}{\left[{\left[{\mathcal{A}}_{h_{opt}}\left({\mathcal{D}}_{train}\right)\right]}_{h_{model}}\left(x^{\left(i\right)}\right)-\tilde{\phi }\left(x^{\left(i\right)}\right)\right]}^2.$$

(16)

In order to gain an accurate neural network, we need to find the hyper parameters which minimize (16). The problem becomes:

$$(h_{opt}^{*},h_{model}^{*})=\underset{(h_{opt},h_{model})}{argmin}\mathcal{H}(h_{opt},h_{model}).$$

(17)

These two parameters play an important role: $h_{model}$ determines the capacity of ${\mathcal{S}}_{h_{model}}$, while $h_{h_{opt}}$ oversees the local minimum at which the optimization procedure converges and how fast this happens. Algorithm 1 (Bayer and Stemper 2018) summarizes the main steps of the method.

Algorithm 1: Deep calibration algorithm

6. Empirical Results

In this section, we apply our findings to the solution of the Black–Scholes–Barenblatt equation as well as to calibrate the Heston model. In both cases, we start adopting the neural network structure proposed in Raissi (2018), using Tensorflow 1.1 (Abadi et al. 2016) on a HP PRO BOOK 450 G8 with an Intel Core i7-1165G7 CPU 2.80 GHz × 8 with 16 Gigabyte (GB) RAM.

6.1. Black–Scholes–Barenblatt Equation

The Black–Scholes–Barenblatt (BSB) equation (Li et al. 2019) arises from the Black–Scholes equation and generalizes it considering a nonlinear source term.

The price $V(S,t)$ of an European option with underlying S is given by the famous Black–Scholes equation:

$$\frac{\partial V}{\partial t}+\frac{1}{2}{\sigma }^2{S}^2\frac{{\partial }^{2}V}{\partial S^2}+rS\frac{\partial V}{\partial S}-rV=0.$$

It is important to note, that that latter equation depends on two parameters (could be functions), namely, the volatility $\sigma$ and the risk-free interest rate r.

The exact choice of r and $\sigma$ is not easy, but upper and lower bounds on them can be imposed with reasonable certainty. With this bounds we can compute the range and all admissible option prices.

For example, if we consider vanilla puts and calls, we know that the option prices have a monotone and convex behavior and in this case envelopes are described by a linear Black–Scholes equation. However, if we have options which prices have a different behavior (non-monotone and non-convex), then the envelopes must be found from a fully non linear equation as follow:

$$u_t=-\frac{1}{2}Tr\left[{\sigma }^{2}diag\left(X_t^2\right)D^{2}u\right]+r(u-{\left(Du\right)}^{\prime }x),$$

(18)

with terminal condition $u(T,x)=g\left(x\right)$. The latter being related to the following system of Forward–Backward SDEs:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill d{X}_t& =\sigma diag\left(X_t\right)d{W}_t,t\in [0,T]\hfill \\ \hfill X_0& =\xi \hfill \\ \hfill d{Y}_t& =r(Y_t-Z_t^{\prime }{X}_t)dt+\sigma Z_t^{\prime }diag\left(X_t\right)d{W}_t,t\in [0,T)\hfill \\ \hfill Y_T& =g\left(X_T\right).\hfill \end{array}\end{array}$$

(19)

To verify the effectiveness and accuracy of the algorithm, we need the exact value of the solution of the BSB equation, which is given by the following lemma (Beck et al. 2019):

Lemma 1. 

Let $c,{\sigma }_{max},r,T\in (0,\infty )$, ${\sigma }_{min}\in (0,{\sigma }_{max})$, $d\in \mathcal{N}$, let $\overline{\sigma }:\mathbb{R}\to \mathbb{R}$ be the function that satisfies for all $x\in \mathbb{R}$ the following:

$$\overline{\sigma }\left(x\right)=\left\{\begin{array}{c}{\sigma }_{max}:x\ge 0\hfill \\ {\sigma }_{min}:x<0,\hfill \end{array}\right.$$

(20)

and let $g:{\mathbb{R}}^d\to \mathbb{R}$ and $u:[0,t]\times {\mathbb{R}}^d\to \mathbb{R}$ be the function that satisfies for all $t\in [0,T]$, $x=(x_1,\dots ,x_d)\in {\mathbb{R}}^d$ that $g\left(x\right)={c\left|\right|x\left|\right|}_{{\mathbb{R}}^d}^2=c{\sum }_{i=1}^d{\left|x_i\right|}^2$ and

$$u(t,x)=exp\left(\right[r+|{\sigma }_{max}{|}^2\left](T-t)\right)g\left(x\right).$$

(21)

Then, it holds for all $t\in [0,t],x=(x_1,\dots ,x_d)\in {\mathbb{R}}^d$ that $u\in C^{\infty }([0,T]\times {\mathbb{R}}^d,\mathbb{R})$, $u(T,x)=g\left(x\right)$, and

$$\frac{\partial u}{\partial t}(t,x)+\frac{1}{2}\sum _{i=1}^d{\left|x_i\right|}^2{\left|\overline{\sigma }\left(\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)\right)\right|}^2\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)=r{(u(t,x)-\langle x,\left({\nabla }_{x}u\right)(t,x))\rangle }_{{\mathbb{R}}^d}.$$

(22)

Proof. 

Since u is infinitely differentiable and (21) ensures that $\forall t\in [0,T]$, $x=(x_1,\dots ,x_d)\in {\mathbb{R}}^d$, then we have:

$$u(t,x)=exp(-t[r+|{\sigma }_{max}{|}^2]+T[r+|{\sigma }_{max}{|}^2\left]\right)g\left(x\right).$$

(23)

Hence, we obtain that for all $t\in [0,T],x=(x_1,\dots ,x_d)\in {\mathbb{R}}^d$, $i\in \{1,2,\dots ,d\}$ it holds that

$$\frac{\partial u}{\partial t}(t,x)=-[r+|{\sigma }_{max}{|}^2]u(t,x)$$

(24)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \langle x,\left({\nabla }_{x}u\right)(t,x)){\rangle }_{{\mathbb{R}}^d}& =exp(-t[r+|{\sigma }_{max}{|}^2]+T[r+|{\sigma }_{max}{|}^2\left]\right){\langle x,(\nabla g)\left(x\right)\rangle }_{{\mathbb{R}}^d}\hfill \\ \hfill & =exp(-t[r+|{\sigma }_{max}{|}^2]+T[r+|{\sigma }_{max}{|}^2\left]\right){\langle x,2cx\rangle }_{{\mathbb{R}}^d}\hfill \\ \hfill & =2cexp(-t[r+|{\sigma }_{max}{|}^2]+T[r+|{\sigma }_{max}{|}^2{\left]\right)\left|\right|x\left|\right|}_{{\mathbb{R}}^d}^2=2u(t,x)\hfill \end{array}\end{array}$$

(25)

and

$$\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)=2cexp(-t[r+|{\sigma }_{max}{|}^2]+T[r+|{\sigma }_{max}{|}^2\left]\right)>0,$$

(26)

hence, exploiting (20), $\forall t\in [0,T],x=(x_1,\dots ,x_d)\in {\mathbb{R}}^d$, $i\in \{1,2,\dots ,d\}$, we have:

$$\overline{\sigma }\left(\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)\right)={\sigma }_{max}.$$

(27)

Equations (24)–(27) ensure that for all $t\in [0,T]$, $x=(x_1,\dots ,x_d)\in {\mathbb{R}}^d$ it holds that

$$\begin{array}{cc}\hfill & \frac{\partial u}{\partial t}(t,x)+\frac{1}{2}\sum _{i=1}^d{\left|x_i\right|}^2{\left|\overline{\sigma }\left(\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)\right)\right|}^2\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)-r{(u(t,x)-\langle x,\left({\nabla }_{x}u\right)(t,x))\rangle }_{{\mathbb{R}}^d}\hfill \\ \hfill & =-[r+|{\sigma }_{max}{|}^2]u(t,x)+\frac{1}{2}\sum _{i=1}^d{\left|x_i\right|}^2\left|\overline{\sigma }\left(\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)\right)\right|\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)-r(u(t,x)-2u(t,x))\hfill \\ \hfill & =-[r+|{\sigma }_{max}{|}^2]u(t,x)+\frac{1}{2}\sum _{i=1}^d|x_i{|}^2{\left|{\sigma }_{max}\right|}^2\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)+ru(t,x)\hfill \\ \hfill & =\frac{1}{2}\sum _{i=1}^d|x_i{|}^2|{\sigma }_{max}{|}^2\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)-{\left|{\sigma }_{max}\right|}^{2}u(t,x)\hfill \\ \hfill & =|{\sigma }_{max}{|}^2\left[\frac{1}{2}\sum _{i=1}^d{\left|x_i\right|}^2\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)-u(t,x)\right]\hfill \\ \hfill & =|{\sigma }_{max}{|}^2\left[\frac{1}{2}\left|\right|x_i{\left|\right|}_{{\mathbb{R}}^d}^2\frac{{\partial }^{2}u}{\partial x_i^2}(t,x)-u(t,x)\right]\hfill \\ \hfill & =|{\sigma }_{max}{|}^2\left[c\left|\right|x_i{\left|\right|}_{{\mathbb{R}}^d}^{2}exp(-t[r+|{\sigma }_{max}{|}^2]+T[r+|{\sigma }_{max}{|}^2\left]\right)-u(t,x)\right]=0,\hfill \end{array}$$

so the proof is completed.   □

For simulations, we used the following parameters: $T=1$, $\sigma =0.4$, $r=0.05$, $\xi =(1,0.5,1,0.5,\dots ,1,0.5)\in {\mathbb{R}}^{100}$, and $g\left(x\right)={\left|\right|x\left|\right|}^2$, so that (18) becomes:

$$\begin{array}{cc}\hfill & u_t=-\frac{1}{2}Tr\left[0.16diag\left(X_t^2\right)D^{2}u\right]+0.05(u-{\left(Du\right)}^{\prime }x)\hfill \\ \hfill & u(T,x)={\left|\right|x\left|\right|}^2,\hfill \end{array}$$

and the corresponding FBSDE:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill d{X}_t& =0.4diag\left(X_t\right)d{W}_t,t\in [0,1]\hfill \\ \hfill X_0& =(1,0.5,\dots ,1,0.5)\hfill \\ \hfill d{Y}_t& =0.05(Y_t-Z_t^{\prime }{X}_t)dt+0.4Z_t^{\prime }diag\left(X_t\right)d{W}_t,t\in [0,T)\hfill \\ \hfill Y_1& =\left|\right|X_1{\left|\right|}^2.\hfill \end{array}\end{array}$$

(28)

As we said before, we take the following as explicit solution of (18):

$$u(t,x)={exp\left(0.21(1-t)\right)\left|\right|x\left|\right|}^2,$$

having fixed $\sigma$. We employed a 5-layer deep neural network with 256 neurons for each hidden layer to approximate the unknown solution $u(t,x)$. Time domain has been subdivided into $N=10$ equally spaced interval to then apply the Adam optimizer minimizing the loss function (9) with 100 different realizations of the the underlying Brownian motion.. In Figure 2 we plot the learned solution $u(t,X_t)$ at a representative realizations of the underlying high-dimensional process $X_t$. The latter shows how the whole solution, rather than only the terminal points, has been approximated. Moreover, Figure 3 shows how the error decreases in the number of Adam’s iterations, witnessing the goodness of the implemented NN-architecture in accurately approximate the exact solution $u(t,X_t)$. Both pictures are obtained after, resp., $2\times {10}^3$, $3\times {10}^3$, $3\times {10}^3$, $2\times {10}^3$ consecutive iterations of Adam with decreasing learning rates (${10}^{-3}$,${10}^{-4}$, ${10}^{-4}$, ${10}^{-5}$).

6.2. Heston Calibration via Neural Network

In this empirical example, we use the SPX option price data from 15 January 2018 to 9 April 2019, as in Bayer and Stemper (2018). Specifically, we focus on Time to Maturity and Strike Price to estimate the joint distribution $(M,T)$ denoted with ${\mathcal{K}}_{(M,T)}$. We filter the dataset by inverse bid-ask spread, where the inverse bid-ask is the difference between the quoted prices (either by a single market maker or in a limit order book) for an immediate sale (ask) and an immediate purchase (bid) for stocks, futures contracts, options, or currency pairs in some auction scenario. The size of the bid–ask spread in a security is one measure of the liquidity of the market and of the size of the transaction cost, to consider options with a good liquidity level. In Figure 4 we can see that the higher liquidity level is concentrated in the region bounded by $-0.1\le m\le 0.29$ and $\frac{1}{365}\le T\le 0.2$.

For this reason, we bound the domain of IV map to this region. Now, the aim is to estimate the density distribution ${\mathcal{K}}_{(M,T)}$. We apply Kernel Density Estimate (KDE), a non-parametric way to estimate the probability density function of a random variable, using the function GrindSearchCV in scikit-learn with a Gaussian kernel to select the optimal value for the bandwidth. Then, we generate a sample of one million elements (two columns with time to maturity and moneyness) from the distribution. Finally, we add the parameters $\mu ,\rho ,\lambda ,\overline{\nu },{\nu }_0$ of Heston model extracted from a Uniform distribution $\mathcal{U}$, see

Table 1. Marginal priors of the model parameters $\mu$ for synthetically generating $\mathcal{D}$. The interest rate r is fixed.

Parameter	Marginal
$\eta$	$\mathcal{U}[0,5]$
$\rho$	$\mathcal{U}[-1,0]$
$\lambda$	$\mathcal{U}[0,10]$
$\overline{\upsilon }$	$\mathcal{U}[0,1]$
${\upsilon }_0$	$\mathcal{U}[0,1]$

, and the value of IV.

Last column’s values have been computed exploiting the Heston prices function based on the Levenberg–Marquardt algorithm, see Proposition 1. The latter allows to compute European Call prices under the Heston dynamic via associated closed form solution and by using the QuantLib library (a free/open-source library for quantitative finance) providing many financially oriented solvers 1. We end up with IV values for every simulation, then used to train the neural network. In particular, we consider 900,000 samples for the training set, 45,000 for the test set and 55,000 for the validation set.

The first NN that we apply, a ReLU Fully Connected Neural Network (FCNN), is the same as the one proposed in Bayer and Stemper (2018). It is based on a 4-hidden layer NN with ReLU as activation function and 512 neurons for each layer the output layer being linear (no activation function), and with tensors denoted by rectangles.

The second NN-approach we propose belongs to the family of Recurrent Neural Network (RNN), a class of artificial NNs where connections between nodes form a directed or undirected graph along a temporal sequence. The latters, may present some problems when working with time series forecasting, as in our case. Accordingly, we choose to use a Long Short Term Memory Neural Network (LSTM) (Hochreiter and Schmidhuber 1997), which is capable of learning long-term dependencies. The chain structure is the same as RNN but, instead of a single layer, there are four layers interacting with each other in a special way.

The key ingredient of LSTMs is the cell state, the horizontal line running through the top of the diagram. The cell state is a kind of conveyor belt; it runs through the entire chain, with only a few minor linear interactions. This type of neural network has the ability to remove or add information to the cell state, carefully regulated by structures called gates. Each of this gates is composed by a sigmoid neural layer and a point-wise multiplication operation. In our case, we consider two hidden layers with 64 neurons for each one.

Once the neural networks are assembled, we train them in ${\mathcal{D}}_{train}$ and use ${\mathcal{D}}_{val}$ as a validation set. The performance are evaluated considering the loss curve generated during the training, the training time, the loss computed on the test set and the Relative Error (RE) between the IV obtained with NN and the IV computed by classic Fourier-based method:

$$RE(\mu ,m,T):=\frac{|{\phi }_{NN}(\mu ,{\nu }_0,e^m,T)-\tilde{\phi }(\mu ,{\nu }_0,e^m,T)|}{\tilde{\phi }(\mu ,{\nu }_0,e^m,T)}.$$

(29)

Both neural networks are stopped using the callback $EarlyStopping$: this allows us to decide what performance measure use in order to end training. In our case, we would seek a minimum for both validation loss and mean squared error and a maximum for validation accuracy. We impose patients (number of epoch with no improvement) equal to 3.

The loss curves in Figure 5 show that both neural networks have a good but different behavior. Indeed, both of them decrease quickly, but the FCNN train and the validation loss are very close to each other, with some exception epoch where the model shows a large gap between the validation and the train. This does not happen in the case of LSTM, since both curves decrease and reach a point of stability with a smaller gap.

Looking at the computational time, we end up with a very import result: FCNN spend about 59 min while LSTM only 21. This huge difference suggests that LSTM might better explain the link between model parameters and the IV surface in a shorter time. This evidence is also confirmed by the errors computed on the test set and on a random dataset generated by setting the parameters of the model and varying only time to maturity and log-moneyness. The results are $9.7\times {10}^{-6}$ and $3.6\times {10}^{-6}$, respectively. From Equation (29) we obtain the heatmaps with different time to maturity and log moneyness; we compute this error in a new dataset with random log moneyness and time to maturity and fixed model parameters. As the heatmap of interpolated relative errors shows, in both models there are small errors throughout most of the IV surface, with increased relative errors only for times on the short and long end, which may be attributed to less training due to less liquidity; see Figure 6 and Figure 7. The mean of DNN and LSTM are 0.09388463628 and 0.035987278, respectively.

To empirically check the goodness of the LSTM model error, we also analyzed it in a different way: we computed the difference between the value given by the NN on the test set and the true value in the dataset over the last value. The results are in Figure 8 where the normalized histogram of the relative errors in the test set is shown with quantiles. From this distribution we obtain a good approximation of $\phi$ with respect to ${\phi }_{NN}$.

Finally, the implied volatility surface generated by the two NN are shown in Figure 9.

Once the NN are trained, we are able to proceed with the calibration step. We apply the algorithm described in Algorithm 1 to initialize the parameters for the Heston model following Gatheral (2011);

Table 2. Comparison between the theoretical parameters and the neural networks parameters.

Parameter	Theoretical	FCNN	LSTM
$\eta$	$0.3877$	$0.3889$	$0.4124$
$\rho$	$-0.7165$	$-0.6871$	$-0.7345$
$\lambda$	$1.3253$	$1.3421$	$1.105$
$\overline{\nu }$	$0.0354$	$0.0312$	$0.0401$
${\nu }_0$	$0.0174$	$0.0169$	$0.0171$

reports the initialization parameters.

Using the above steps, we compute the market quotes. At this point, the deep calibration starts. First, we define a function that returns the implied volatility value and the Jacobian matrix of the NN with respect to a specific input. It will be used in the iteration step of the LM algorithm. Finally, deep calibration runs the algorithm, and after many iterations, we will get the calibrated parameters for the Heston model. A comparison between the obtained values is shown in Table 2. From the fitted values there is no clear evidence that LSTM specification outperforms the FCNN or vice-versa. Indeed, the absolute value of difference between calibrated and theoretical parameters is not better overall in neither case. However, considering the training time and the relative error in (29), we may conclude that the LSTM specification performs better.

7. Conclusions

PDEs play a fundamental role in solving many practical problems, from physics to financial mathematics. They can describe almost all time evolution models, but their solution could be problematic due to the “curse of dimensionality”. It happens because real world problems have a huge number of variables, making classic deterministic algorithms (Finite Element of Finite Difference) unfeasible. In addition, the presence of non-linear factors takes off any possibility to use probabilistic algorithms.

The growth of technology has given us powerful tools to overcome these problems. With deep learning and, in particular, with deep neural networks, we are now able to approximate the solution or part of them. In both experiments described in Raissi (2018) and Bayer and Stemper (2018), neural networks solve the course of the dimensionality problem.

The approach initially suggested by Raissi (2018) and further developed in this paper can be seen as an upgrade of the classical backward stochastic differential equation approximation provided in El Karoui et al. (1997). Furthermore, the solution of the algorithm is a function which can be evaluated not only in the initial point, but also in the whole space-time domain, see Figure 2, due to the independence between the parameters number of NN and the number of points needed to discretize the time domain. Because of its flexibility, we plan to extend the proposed NNs-based framework to consider second-order backward stochastic differential system of equations.

Concerning calibration, we have shown that the stochastic volatility model can be effectively approximated via NNs architectures, and also limited computational efforts, hence reducing execution times if compared with those related to standard numerical approaches. It is worth mentioning that, although our NN-scheme has been implemented as a regression tool, it approximates the implied volatility map with high accuracy. Moreover, to better explain the relationship between data, we showed that our LSTM-inspired solution (implemented via Keras) requires less training computation time, also with smaller loss error.

The same procedure might also be used to calibrate the optimal interest rate r. This could be conducted by using a priori on it by training the neural network on the r value.

In order to improve our performance, we also implemented a different approach, exploiting TensorFlow and building our neural network piece by piece. Nevertheless, the obtained numbers do not significantly improve that which we already obtained with our methods, despite the increased quantity of software/hardware resources needed to implement this new attempt.

Further research will be devoted to using more complex models, such as the Fractional Heston Model, to see if we are able to improve the results using more complex neural network.