1. Introduction
Imagine a rational agent that observes some information set (a filtration ) and is interested in forecasting values of some random variable. As new information becomes available (in the form of a random variable L, such that we have a new, enlarged filtration where , one would expect correction of systematic errors and/or some reduction of prediction variance. Through the information accumulation process useless information is discarded, and only information that yields true prediction gains is incorporated—one thus expects forecasts based on the enlarged information set to be at least as good as forecasts based on the smaller information set . In this work we show an example where a larger set of information leads to a bias in the forecasting process, which seems in contradiction with the mechanism described above.
We compare survey-based forecasts from the Market Expectations System of the Central Bank of Brazil (henceforth Focus) with future markets (B3 futures exchange markets, former BMF and BOVESPA) 30-days forward rates; and find that future markets rates are unbiased predictor of future spot rates, but Focus survey forecasts are biased predictors through a substantial portion the observed sample. This is counter-intuitive because forward rates are obviously within the information set of keen market participants. No arbitrage theory asserts that exchange rates should be well characterized by a martingale process under the equivalent risk neutral measure
P—i.e., the best possible predictor of future exchange rates is the present value of forward rates. Results in
Section 3 suggest it is indeed the case in Brazilian markets; it is not possible to reject the martingale hypothesis for either the spot exchange rate, or future market forward rates. Yet Focus survey forecasts seem inconsistent with a martingale process. One could say that markets participants are employing a prediction scheme systematically incompatible with the observed stochastic process of interest. How to explain such phenomenon?
We argue this puzzling phenomenon can be interpreted through strict local martingale theory—specifically through the mechanism detailed by Dandapani and Protter [
1]. Dandapani and Protter [
1] explain how enlarging filtration
F to
G changes the underlying measure (say from
P to
Q), which can induce a stochastic drift in the volatility of
S, turning
S from an
martingale to a
strict local martingale.
The idea of smaller and larger information sets is natural to problems in finance. The filtration enlargements problem is studied in Jeulin and Yor [
2], Jeulin [
3] and Jeulin and Yor [
4], and more recent developments in Aksamit and Jeanblanc [
5]. As an example of application in finance Corcuera and Vadivia [
6] show how filtration enlargements can be applied to model default risk and insider trading problems. Strict local martingales are at the core of a blooming real-time bubble detection literature. For example, Jarrow et al. [
7] apply a nonparametric methodology (which we borrow in
Section 4) to show that Linked-In stock exhibited price bubble behavior immediately after the firms IPO, and Chaim and Laurini [
8] identify Bitcoin bubbles. Recent developments include, for example, Dandapani and Protter [
9] who provide sufficient conditions for the multidimensional case, and Piiroinen et al. [
10] who introduce a bubble detection indicator based on option prices.
Using two different identification techniques we explore the strict local martingale hypothesis. In
Section 4 we employ a nonparametric kernel approach based on Jarrow et al. [
11] to study how the volatility of our series of interest behave as price varies, and access the convergence of condition (
3). We then turn to an alternative, parametric strict local martingale test in
Section 5.
To our knowledge this is the first work to relate systematic prediction errors and the filtration enlargement mechanism. While the nonparametric identification procedure in
Section 4 draws heavily from Jarrow et al. [
11], formal univariate estimation of the stochastic volatility model in
Section 5 is itself novel. Standard practice in financial industry is to calibrate parameter values from option price information (see for example Mikhailov and Nögel [
12]), and classical estimates (like by Ait-Sahalia and Kimmel [
13]) employ volatility proxies to circumvent the latent variable problem—we tackle the latent variable issue by employing Bayesian simulation techniques to estimate model
13, so that it is possible to make inference regarding model parameters from a single time series.
The remainder of this paper is structured as follows.
Section 2 presents some key definitions and a discussion about the filtration enlargement mechanism. Empirical evidence of our main points are found in
Section 3. Nonparametric strict local martingale identification is presented in
Section 4.
Section 5 details our parametric estimation. We discuss results in
Section 6, and
Section 7 concludes.
3. Empirical Evidence
Our sample consists of (30-days ahead) exchange rate predictions of Brazilian market participants (the Focus survey); future markets closing forward rates traded at B3 (former Bovespa BMF); and the official Brazilian exchange rate (PTAX). Sample range and frequency are limited by Focus data such that 179 monthly observations, from November 2001 to September 2016, are available. The data used in this work are available on the websites of the Central Bank do Brazil and B3.
Focus forecasts regard the last week day of the corresponding month, while future markets forward rates are for the first week day of the same month. We thus obtained the “predicted forward rate at the last day of the month” by using the Brazilian interbank deposit certificate (CDI) rate. We also considered two alternative formulations—first using the forward rate the day before Focus release, and then using the forward two months maturity. Results were similar to the baseline formulation and therefore not reported. Trajectories are plotted in
Figure 1, and descriptive statistics are displayed in
Table 1.
The Focus Bulletin is a survey released every Monday by the Central Bank of Brazil. The report, a daily survey of forecasts of about 120 banks, funds and other institutions summarizes the market expectations for the Brazilian economy. The market expectations system, which collects information reported in the Focus Bulletin, consists of a web interface where agents report their expectations for the macroeconomic variables of interest. The system considers the last information reported in the past 30 days by market participants. If an agent does not revise his or her expectations in the last thirty days, this information is not considered in statistic calculations. An important incentive of this system is the monthly release of best performing institutions, indicated in the top five predictors for each macroeconomic variable monitored in the system (see Mariani and Laurini [
18] for a thorough description of the Focus survey).
We now verify whether processes describing forward rates, Focus forecasts, and the spot exchange rate are consistent with a martingale process—as expected by the first theorem of asset pricing. For this we employed test-based on Cramer–von Mises (CvM) and Kolmogorov–Smirnov (KS) statistics proposed by Domínguez and Lobato [
19], augmented with bootstrap finite sample correction of Charles et al. [
20]. Test statistics and equivalent
p-values are reported in
Table 2. Both tests failed to reject the martingale null hypothesis for the spot exchange rate and future markets forward rates, but rejected it when considering Focus forecasts. The same tests were applied to prediction errors (future-spot and Focus-spot). We find that the difference martingale hypothesis was rejected for Focus prediction errors, and not rejected for future market prediction errors. This result indicated that there was a systematic bias component in the forecasts reported in the Focus system, which was inconsistent with the behavior of spot and future exchange rates, and indicated that Focus forecasts were sub-optimal.
Figure 2 plots p-values for a rolling-sample forecast bias test. We regress 30-days ahead exchange rates against Focus forecasts and a constant, i.e.,
, and jointly test if
and
. From November 2001 to July 2008 it was possible to reject the null
and
hypothesis at a
significance level, again supporting the existence of a relevant bias in the forecasts of market analysts reported in the Focus system.
Table 3 displays an illustrative prediction performance comparison between Focus survey forecasts and future market rates. We repeat the procedure for the “biased” and “unbiased” Focus subsamples (30 November 2001–31 July 2008 and 29 September 2008–30 September 2016). Forward rates were generally a better predictor the 30-days ahead spot interest rate. When considering the whole sample or any subsample forward rates fared better when considering RMSE or MAE, while Focus mean prediction errors were smaller in absolute terms.
For each subsample we tested the null hypothesis of no difference in forecast accuracy by using the Diebold and Mariano [
21] and Clark and West [
22] tests. From
p-values in
Table 4 we can see the null of equal forecast performance was rejected at 5% by both tests for all subsamples. Indeed, Clark and West [
22] tests rejected the null at the 1% level for all subsamples; and
p-values of Diebold and Mariano [
21] were always smaller than 0.03. These results reinforced that forward rates were indeed better predictors of 30-days ahead spot rates than Focus survey forecasts.
In the following sections we show that this bias component can be explained by the presence of a strict local martingale, using the framework proposed in Dandapani and Protter [
1]. For this we used non-parametric and parametric tests for the presence of an SLM, verifying the existence conditions in the volatility structure of the price process.
4. Nonparametric Estimation
Recall from our discussion in
Section 2 that a stochastic process is a strict local martingale if its volatility increases fast enough with higher level, such that
for all
. We tried to empirically characterize the volatility term
in order to study the convergence properties of the integral above. In order to estimate
and relation (
3), we employed the kernel-based estimator of Florens-Zmirou [
23] and Jiang and Knight [
24] for the diffusion component:
where
is a kernel function with bandwidth
. Is this work we used a Gaussian kernel and generalized cross-validation to estimate the optimal bandwidth parameter.
Figure 3 and
Figure 4 plots estimated volatility functions (top panel) and relation
(bottom panel), for our three processes of interest. One readily notices that Focus forecasts were less volatile than spot exchange rates or future market predictions for every observed price realization, and that volatility functions of both spot rates and future markets stopped increasing somewhere past the
mark. The relation
was stable (one might even say increasing) at higher price ranges for spot and future processes, which alludes to a violation of the condition in Equation (
3) (i.e., the integral is infinite)—indicating that spot rates and future market predictions are true martingales.
The curvature of the volatility function of Focus forecasts decreased slower, such that the function was always increasing for observed prices. This reduces the term
and indicates convergence of the integral in Equation (
3).
We argue systematic Focus prediction errors are associated with incompatibility between the stochastic process that drives forecasts and exchange rates.
Figure 5 plots
and
for the first subsample period. Note the absence of inflection at the end of Focus’ variance function, and how sharply
decreases. This can be taken as evidence that Focus forecasts behaved like strict local martingales from November 2001 to August 2008. Conversely, when considered the second subsample, as in
Figure 6, it becomes hard to distinguish between Focus and spot processes—both are consistent with what is to expect from true martingales.
This nonparametric analysis has some important limitations. First, the analysis is visual and presents no formal test for the SLM hypothesis. The second point is that the SLM behavior refers to extrapolation for the whole set of unobserved prices, whereas we only analyzed the variance function for observed sample values. To circumvent this limitation, we present a formal parametric test, detailed in the next section.
5. Parametric Estimation
Consider the stochastic volatility model studied by Andersen and Piterbarg [
25]
where
and
is a standard two dimensional Brownian motion with correlation coefficient
. Note that if
the model in Equation (
13) reduces to the well-known Heston [
26] model. This model is equivalent to the one discussed in
Section 2, but also allows for testable implication of the Martingale hypothesis. Results of Andersen and Piterbarg [
25] and Protter [
27] describe how the parameter space of model (
13) can be divided in two disjoint sets, such that a specific parameter range characterizes stochastic process
S as a strict local martingale.
Theorem 2. Andersen and Piterbarg [25]. For the model (13): if , S is a true martingale; if and or , S is a true martingale; if and , then S is a strict local martingale. For the case S is a true martingale if , and S is a strict local martingale if . Proof: see Protter [27]. Equipped with this theorem we employed Bayesian techniques to estimate the stochastic volatility model of Andersen and Piterbarg [
25] to investigate whether Focus forecasts are compatible with a true martingale or a strict local martingale process.
Estimation was carried out in the Stan software (with help of the R-package “rstan"). Stan employs a Hamiltonian Monte Carlo (HMC) algorithm rather than the traditional Metropolis Hastings (MH). HMC augments the usual random walk posterior sampling scheme with Hamiltonian dynamics, such that new proposed values can be further apart than they would be under a random walk exploration scheme, but have a high acceptance probability nevertheless. HMC is then more efficient than MH in the sense that it takes fewer algorithm iterations in order to achieve chain convergence. This is especially relevant in our stochastic volatility context because Bayesian statistics treats latent variables as it treats unknown parameters, what can be computationally expensive.
We elected prior distributions given restrictions on parameter values in model (
13). A Uniform distribution between 0 and
was specified for
and
;
was assumed Normal, with mean
and standard deviation
;
was assumed Normal, with mean
and standard deviation
;
p was also assumed Normal, with mean
and standard deviation
; correlation
was specified as
,
.
Table 5 reports estimation results, posterior densities are displayed in
Figure 7. Notice correlation coefficient
was estimated positive inside the
credible interval—this is desirable since if
,
is a true martingale for all values of
p, and further analysis would be unnecessary.
The posterior mean of parameter p is estimated around , which according to Theorem 2 characterizes Focus forecasts as a true martingale process. However, the posterior distribution of p is quite disperse and imply substantial uncertainty regarding parameter value. Also, prior and posterior distributions of parameters and are very similar, thus it is questionable whether those parameters are identifiable from data.
We recognize that available Focus data is of monthly frequency and sample size is limited, which makes estimating model (
13) an ambitious enterprise. In face of this identification difficulty we decided to fix
and focus attention on estimates of
p and
, which were at the core of our parametric approach. This restriction was also used in Baldeaux et al. [
28] and Chaim and Laurini [
8], which also attempted to verify the presence of strict local martingales.
One can see from
Table 6 that the restricted model indeed yields more precise estimates. Comparing the second column of
Table 5 and
Table 6 one can see that posterior standard errors of
,
, and
p were substantially smaller; while
estimates were essentially unchanged.
Figure 8 displays bell-shaped, less disperse, posterior distributions of
,
, and
p.
Parameter
p is now estimated between
and
with
probability, a much narrower credibility interval. Still, because substantial probability was attributed to the
range, it is not possible to convincingly assert that
is a strict local martingale process from
Table 6 alone. A more formal test is appropriate. We ask: does imposing that
is a strict local martingale worsen fit performance? for such we further restrict model (
13) such that
. Then compare this formulation to the one in
Table 6 and
Figure 8, for which
. We evaluated model performance through the leave-one-out cross-validation (LOO) and the widely applicable information criterion (WAIC) of Vehtari et al. [
29]—those are pointwise out-of-sample prediction accuracy tests of a fitted Bayesian model, and are based on the log-likelihood evaluated at the posterior simulation of the parameter values.
Table 7 reports WAIC and LOO information criteria for the unrestricted,
, and restricted,
models. Imposing SLM does not worsen out of sample forecast behavior, as both WAIC and LOO attribute a smaller information criterion to the restricted model.
Figure 9 plots fitted versus realized Focus volatility for both models in
Table 7.
6. Discussion
Although one must recognize methodological choices and statistical uncertainty, we view the results of
Section 4 and
Section 5 as evidence that Focus survey forecasts are better characterized by a strict local martingale than a driftless true martingale process, specially through the first subsample from 2000/08 to 2008/07.
The nonparametric approach of empirically evaluating condition (
3) is fundamentally limited by the range of observed values
in a given time interval. We are trying to extrapolate from a compact set of
properties of
for all of
, and, as discussed in Jarrow et al. [
7] results will invariably depend on the extension method employed.
Theorem 2 gives us empirically testable objective criteria that characterize a price process as either a true martingale or a strict local martingale. But formally estimating Heston-type models is not standard practice for understandable reasons. Classical likelihood inference has major difficulties when dealing with latent variables, what makes Bayesian simulation methods appealing.
When we compromise and fix , our estimates are still inconclusive with respect to Theorem 2—because although one can say with reasonable certainty that is positive, the posterior of p attributes significant probability to the true martingale range, what makes a more formal test of the strict local martingale hypothesis warranted.
We draw from developments in Bayesian model selection (Vehtari et al. [
29]) and devise an experiment in which one imposes that
is a strict local martingale (i.e.,
and
), and evaluate whether this restriction hinders out-of-sample forecast performance. Imposing SLM behavior does not worsen the out-of-sample forecast, but it is hard to distinguish between models. In hindsight this result is not surprising because even in the
p-unrestricted model more, probability is attributed to the SLM range of
than otherwise.
7. Concluding Remarks
We investigate the puzzling poor performance of Brazilian market participants in forecasting the BRL-USD exchange rate. Extrapolations of forward rates are a better predictor of the 30-days ahead spot exchange rates than Focus survey forecasts—which are also biased through a substantial portion of our sample, from August 2002 to July 2007. This result is counter-intuitive since Focus participants have access to the information on the future foreign exchange market, and yet they make biased forecasts.
Dandapani and Protter [
1] describe a mechanism through which the incorporation of new information induces a drift in an otherwise driftless local martingale, turning it into a strict local martingale. We argue this filtration enlargement mechanism could help explain our forecast performance conundrum. Spot and future rates are consistent with martingales, while Focus forecasts have a bias term, equivalent to the presence of a drift. Another important feature is the presence of stochastic volatility in this process, important in the characterization of a strict local martingale used by Dandapani and Protter [
1] to represent the drift term generated by the process of information acquisition (filtration enlargement).
In order to investigate the strict local martingale hypothesis, we employ two different identification methods. The first one is a nonparametric estimation of variance function
and rely on condition (
3)—that asserts a local martingale is a strict local martingale if its volatility increases faster than linearly with level. Focus forecasts are less volatile, in absolute terms, than either spot or future rates, but Focus volatility increases faster for higher price observations. The second draws on results of Andersen and Piterbarg [
25]: depending on parameter values of the stochastic volatility model (
13), the underlying price process is either a true martingale or a strict local martingale. Employing Bayesian simulation techniques we estimate Focus 30-days ahead exchange rate forecasts follow a strict local martingale with higher probability than otherwise.
In this paper we present a new way of interpreting the presence of biased forecasts in a market where there are alternative forecasting methods that are informationally efficient. Using the filtering process proposed by Dandapani and Protter [
1], we show that in the presence of stochastic volatility it is possible that more informed forecasts are worse than predictions based on a set of more restricted information. This result is important for the analysis of learning processes and information acquisition in financial markets.