Next Article in Journal
How Domestic Firms Benefit from the Presence of Multinational Enterprises: Evidence from Indonesia and Philippines
Previous Article in Journal
Value Premium and Technical Analysis: Evidence from the China Stock Market
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Nonparametric Evaluation of the Optimality of German Export and Import Growth Forecasts under Flexible Loss

by
Christoph Behrens
Department of Economics, Helmut Schmidt University, Holstenhofweg 85, 22043 Hamburg, Germany
Economies 2019, 7(3), 93; https://doi.org/10.3390/economies7030093
Submission received: 28 June 2019 / Revised: 4 September 2019 / Accepted: 5 September 2019 / Published: 9 September 2019

Abstract

:
This study contributes to research on the nonparametric evaluation of German trade forecasts. To this end, I compute random classification and regression forests to analyze the optimality of annual German export and import growth forecasts from 1970 to 2017. A forecast is considered as optimal if a set of predictors, which models the information set of a forecaster at the time of forecast formation, has no explanatory power for the corresponding (sign of the) forecast error. I analyze trade forecasts of four major German economic research institutes, a collaboration of German economic research institutes, and one international forecaster. For trade forecasts with a horizon of half-a-year, I cannot reject forecast optimality for all but one forecaster. In the case of a forecast horizon of one year, forecast optimality is rejected in more cases if the underlying loss function is assumed to be quadratic. Allowing for a flexible loss function results in more favorable assessment of forecast optimality.

1. Introduction

As one of the world’s main exporters, Germany’s trade policy has received much attention in recent years. Professional forecasts play a crucial role in this context, as economic agents rely on such forecasts, when shaping economic expectations (Carroll 2003). In order to make accurate policy and investment decisions, it is, therefore, necessary to correctly predict trade developments, as these dynamics greatly influence output growth and price levels through inflationary pressures from import prices (D’Agostino et al. 2017). Additionally, optimal trade forecasts are an essential part of GDP forecasts as macroeconomic forecasters tend to implement a so-called disaggregated approach when forming such forecast. In that case, economic research institutes individually predict all components of the GDP and combine these predictions to a forecast of total output (see, among others, (Angelini et al. 2010; Heinisch and Scheufele 2018) for a comparison of direct and disaggregated forecasting approaches). Research on predictions of single GDP components mostly focuses on forecasts of private consumption (see, for instance, Vosen and Schmidt 2011). In the case of trade developments, a study by Ito (1990) reports behavioral biases of importers and exporters when forming expectations about exchange rate changes, hinting that also macroeconomic trade forecasts might not be optimal.
Even though trade developments play a prominent role in the formation of macroeconomic forecasts, their evaluation has not been extensively studied. The focus of the research on trade forecasts lies on their formation. Here, studies either implement structural models of economic environments and simulate trade dynamics (Hervé et al. 2011; Riad et al. 2012) or other authors aim at optimizing the forecasting performance of time series models (Frale et al. 2010; Jakaitiene and Dées 2012; Keck et al. 2009; Yu et al. 2008). Studying German trade is especially interesting because of the country’s dependence on exports and its large trade surplus, which has been at the center of public attention in recent years. However, studies analyzing German trade forecasts are scarce. The existing studies concentrate on forecasting a single trade aggregate, i.e., German exports (Elstner et al. 2013; Grossmann and Scheufele 2019; Jannsen and Richter 2012), or imports (Grimme et al. 2019; Hetemäki and Mikkola 2005). However, despite the fact that suboptimal trade forecasts can potentially be costly if, for instance, a protectionist trade policy is pursued due to such forecasts, the evaluation of trade forecasts has not received much attention.
The research on macroeconomic forecasts published by German economic research institutes has a long tradition in the scientific community with early analyses by Neumann and Buscher (1980) and Kirchgässner (1984). Today, the research topics in this field are manifold, including studies on forecast revisions (Kirchgässner and Müller 2006), forecast accuracy (Heilemann and Stekler 2013), external assumptions of forecasts (Engelke et al. 2019), forecaster rankings (Kirchgässner 1993; Sinclair et al. 2016), or the economic value of forecasts (Döpke et al. 2018). Most of these studies focus on the analysis of GDP and inflation forecasts by means of panel-based models (Döpke and Fritsche 2006; Müller et al. 2019) or time series models (Kirchgässner and Savioz 2001). A fairly novel approach introduced by Behrens et al. (2018b, 2018a) utilizes nonparametric tree-based models to analyze German GDP growth and inflation forecasts. Behrens (2019) has recently adapted this nonparametric approach to an analysis of the joint efficiency of German trade forecasts in a multivariate setting under quadratic loss. I extend this branch of research by analyzing the optimality of German export growth and import growth forecasts under both quadratic and flexible loss.
To this end, I build on research by Elliott et al. (2005, 2008), who study optimal forecasts under asymmetric loss. The seminal work in this field by Granger (1969), Varian (1974), and Zellner (1986) as well as early applications (Christoffersen and Diebold 1996, 1997) have recently been applied in several fields such as financial forecasting (Aretz et al. 2011; Fritsche et al. 2015), fiscal forecasting (Artis and Marcellino 2001; Elliott et al. 2005), central banking (e.g., Capistrán 2008; Pierdzioch et al. 2016a), as well as GDP and inflation forecasting (Christodoulakis and Mamatzakis 2008; Pierdzioch et al. 2016b; Sun et al. 2018). The basic assumption in this field of research is an asymmetric loss function. In other words, researchers assume that a forecaster’s loss function is not symmetric, which allows for the possibility that forecasters prefer, e.g., underestimating economic growth over overestimating. A possible scenario could be that forecasters fear unfavorable media coverage in the case of an unforeseen recession, whereas an upward correction of the forecast in economically good times might not have such ramifications. In case of trade forecasts, especially in the case of Germany, a motivation for forecasters to prefer underestimating export growth over overestimating, might be the recently increasing protectionist tendencies in international trade (see, among others, Durusoy et al. 2015; Park 2018; Pelagidis 2018). An inflated export growth forecast for Germany could add into the spirit of protectionism of some of its trading partners. Patton and Timmermann (2007), for instance, find for GDP forecasts that, in times of low economic growth, overpredictions of output growth are costlier to the Federal Reserve Bank than underpredictions. The authors introduce the concept of a flexible loss function. The advantage of this concept is that no restrictions need to be imposed on the forecasters loss function; however, as a trade-off, restrictions need to be imposed on the data-generating process (for details on the concept of flexible loss and its implementation, see Section 2). When analyzing the optimality of forecasts, it can be crucial to consider a flexible loss function as results can be misleading if a researcher assumes a certain type of loss function, which, in fact, is misspecified Pierdzioch and Rülke (2013). I, therefore, extend the Behrens et al. (2018c) approach to testing forecast optimality by estimating random regression forests with a quadratic, i.e., symmetric, loss function as well as random classification forests with a flexible loss function to analyze whether or not German trade forecasts are best described by a symmetric loss function. Tree-based models are powerful nonparametric modeling instruments, for which no model specifications regarding the linkage of predictor and response variables need to be imposed a priori and that can deal with datasets with a small number of observations relative to the number of predictor variables. I study export and import growth forecasts for Germany from 1970 to 2017 formed by four major German economic research institutes, a collaboration of German economic research institutes, and an international forecaster.
I describe random forests and my adaption of the Behrens et al. (2018c) approach to testing forecast optimality in Section 2. In Section 3, I present my data and I summarize my empirical results in Section 4. In Section 5, I conclude.

2. The Model

In their study of the Federal Reserve’s economic growth forecasts, Patton and Timmermann (2007) introduce a method to analyze the optimality of a forecast when the type of a forecaster’s loss function is unknown. This concept of a flexible loss function requires a trade-off regarding the underlying data generating process. That is, if (a) the loss function depends only on the forecast error and the data generating process only has dynamics in the conditional mean, or (b) the loss function is homogeneous in the forecast error and the data generating process has dynamics in the conditional mean and variance, a forecast is considered as optimal if the sequence, l F E t + 1 0 , is independent of the forecaster’s information set at the time of forecast formation (see proposition 3, Patton and Timmermann 2007, pp. 1175f). Here, l is an indicator function and F E denotes a forecast error. Given the data at hand (see Section 3 for details), there are no issues regarding dynamics of higher-order moments, as the analyzed forecasts are only available midyear and at the turn of a given year. Patton and Timmermann (2007) further show that forecast optimality can, hence, be tested by estimating a logit or probit model of the form:
l F E t + 1 0 = β Z t + u t + 1 ,
where Z t is a matrix containing a forecaster’s information set in period of time t, β denotes a vector of coefficients, and u t + 1 is an error term. A forecast is considered as optimal if the null hypothesis, which states that the set of predictors does not have predictive value for the sign of the forecast error, i.e., β = 0 , cannot be rejected.
However, such a linear approach to testing forecast optimality runs into problems if, for instance, the number of observed forecast errors is relatively small, such that a large set of predictors causes a problem regarding the degrees of freedom of the model. In this case, one could reduce the number of predictors, which are contained in Z t . However, choosing the omitted predictors is arbitrary at least to some degree. Another problem arises when modeling the link between the sign of the forecast error and the predictor variables. A priori, the researcher needs to define the nature of said link, i.e., if it is, for instance, linear, quadratic, or cubic. Furthermore, the predictor variables can be interdependent, such that this relationship also needs to enter the predefined estimation equation. When using a linear approach to testing forecast optimality, there are several sources of possible model misspecifications, which in turn result in biased results. I follow Behrens et al. (2018c), who propose a nonparametric tree-based model, implementing the Patton and Timmermann (2007) test of forecast optimality, to overcome these drawbacks. Tree-based models have first been introduced by Breiman et al. (1984) and have an advantage in, loosely speaking, letting the data decide on an appropriate form of the estimated model. No predefined linkages between predictor variables and the response variable or among the predictor variables need to be imposed by the researcher. Behrens et al. (2018c) implement Equation (1) by means of random classification forests, which pool a large number of classification trees, such that forecast optimality under flexible loss can be studied by the following model (for an introduction to random forests, see (Breiman 2001)):
l F E t + 1 0 = T c T c ( Z t ) + u t + 1 ,
where T c is a single classification tree. I extend their model by reestimating Equation (2) by means of random regression forests, T r , in order to analyze forecast optimality of German export and import growth forecasts under quadratic and flexible loss.
Generally speaking, regression trees analyze the effects of predictors on a continuous response variable and classification trees do so on a discrete response variable. Both types of trees work along the same process, as they aim at structuring the feature space, i.e., the space set up by the analyzed n predictors, Z = ( z 1 , z 2 , , z n ) , into several non-overlapping regions, R j , in which a simple model is fit (for a comprehensive introduction to tree-based methods, see (James et al. 2015), whose notation I roughly follow, dropping the time index, t, for simplification). These rectangles, or partitions, are generated by recursive, binary splits at the so-called interior nodes of each tree. Figure 1 depicts this tree-building process. An exemplary tree is depicted in the left panel and the respectively divided feature space is shown on the right. The dataset is split into two subsequent regions at each node. The first split occurs at the root of the tree and observations with partitioning predictors z 1 less than or equal to cutpoint c 1 are sent down the left branch of the tree, where they reach a terminal node and are assigned to region R 1 . For observations with partitioning predictors z 1 greater than cutpoint c 1 , another split occurs at the following node. Here, observations which satisfy c 1 < z 1 c 2 , are sent down this node’s left branch and observations with partitioning predictors z 1 > c 2 are sent down the right branch. At each following node, another split occurs with regard to partitioning predictor z 2 , such that, at the first subsequent node, observations with partitioning predictors c 1 < z 1 c 2 and z 2 c 3 are assigned to region R 2 and those observations with c 1 < z 1 c 2 and z 2 > c 3 are assigned to region R 3 . At the second subsequent node, i.e., for z 1 > c 2 , observations which satisfy z 2 c 4 are assigned to region R 4 , whereas observations with partitioning predictors z 2 greater than cutpoint c 4 are assigned to region R 5 . The right panel of Figure 1 depicts how a single tree divides the thusly obtained two-dimensional feature space into the respective regions. The same logic of recursive binary splitting can be analogously applied to a multi-dimensional feature space consisting of various predictor variables. Once the whole feature space is divided into regions, the same prediction is made for all observations, which fall into one region. In the case of a regression tree, the prediction is the mean response of all observations, i.e., forecast errors, in said region. For a classification tree, the response is the sign of the forecast error, which is determined by a majority vote of all observations in the respective region.
Regression and classification trees differ in the way of searching for optimal nodes. For both types of trees, a partitioning predictor and a cutpoint are chosen at each node in order to minimize a so-called node impurity measure. For regression trees, the residual sum of squares, R S S , measures node impurity under quadratic loss. For every pair of predictor variables, z, and cutpoints, c, the following equation is minimized:
R S S j , c = i : z i R l ( j , c ) ( F E i F E ¯ R l ) 2 + i : z i R r ( j , c ) ( F E i F E ¯ R r ) 2 ,
where F E ¯ R l denotes the mean forecast error response for observations falling into the left subsequent region of a given node and F E ¯ R r denotes the mean forecast error response for observations falling into the right subsequent region of said node. In other words, for every possible split, the emerging regions are chosen such that they minimize the RSS of observations which they contain vis-à-vis the mean response of that region.
Analogous to the linear logit/probit Patton and Timmermann (2007) approach to testing forecast optimality, classification trees are a nonparametric way to implement a flexible loss function. Here, node impurity is measured by means of a Gini index, G:
G = s m , R l / r ( 1 s m , R l / r ) ,
where s m , R l / r is the share of positive/negative ( m = 0 , 1 ) forecast errors in the subsequent left/right region of a given node. A small Gini index indicates nude purity, i.e., that a region mostly contains forecast errors with the same sign.
The advantage of this recursive binary splitting process is that no restrictions regarding the linkages of the response variable and the predictor variables or among the predictor variables need to be imposed beforehand. These possibly complex linkages can easily be modeled by regression and classification trees. However, the prediction of a single tree suffers from high variance. Therefore, it is reasonable to combine a large number of trees to a so-called random forest (for an introduction, see Breiman 2001) in order to model the response variable. To this end, it is necessary to use bootstrap resampling to estimate a random regression or classification tree on every bootstrapped dataset (for an introduction to bootstrap resampling, see James et al. 2015). Random trees have the distinct specification that they only use a subset of the set of predictors, Z . This decorrelates the predictions of the single classification or regression trees and minimizes computation time. For every bootstrapped dataset, two thirds of the data are used to model a classification or regression tree and for the remaining third of the data out of bag predictions are made (for an introduction to out of bag error estimation using tree based models, see Hastie et al. 2009). By means of these out of bag predictions, the predictive performance of a tree can be measured. In the case of a classification problem, predictive performance is measured by the out of bag misclassification rate, whereas, for regression trees, the share of the explained out of bag variance of the forecast errors, which is a pseudo R 2 -statistic, serves as a performance metric.
Following Behrens et al. (2018c) and Murphy et al. (2010), I implement permutation tests to measure the statistical significance of the predictive power of a random classification or regression forest. By drawing hundreds of samples without replacement of the forecast errors and estimating random forests on every purely random sample, a benchmark model, which should have no predictive value for the (signs of the) forecast errors, is established. The implied pseudo R 2 -statistics and misclassification rates vary by chance. Hence, the respective metrics obtained by means of the original model should, under the null hypothesis of forecast optimality, not perform significantly better than the formerly described metrics obtained by permuting the data. In other words, if trade forecasts are optimal, the pseudo R 2 -statistic computed for the original random forest must not exceed, for instance, the 95th percentile of the permuted sampling distribution for random regression forests. On the other hand, for random classification forests, the misclassification error rate of the original model must not be smaller than the 5th percentile of the permuted sampling distribution. In summary, the permutation test is carried out along the following steps:
  • Fit a random classification/regression forest on the original data by means of bootstrap resampling and compute the respective out of bag performance metric (misclassification error rate/pseudo R 2 -statistic), P O .
  • Fit a random classification/regression forest on the permuted, i.e., randomized, data by drawing a sample without replacement and compute the respective out of bag performance metric, P P .
  • Repeat Step 2 1000 times and average over the respective performance metrics.
  • Compute p-value = l P P < P O , where l is an indicator function.
The underlying concept is similar to the one of a standard F-test. If the set of predictors has joint predictive power with respect to the sign of the forecast error (classification problem under flexible loss) or the forecast error (regression problem under quadratic loss), the null hypothesis of forecast optimality can be rejected.

3. The Data

I collected data on annual export and import growth forecasts for Germany from 1970 to 2017 by six economic research institutes. Among the forecasters are four German economic research institutes (alphabetical order): Deutsches Institut für Wirtschaftsforschung Berlin (DIW), Hamburgisches Weltwirtschaftsarchiv/-institut (HWWI), ifo Institut für Wirtschaftsforschung Munich (ifo), Institut für Weltwirtschaft Kiel (IfW); a collaboration of German economic research institutes: Gemeinschaftsdiagnose (GD); and the Organisation for Economic Co-operation and Development (OECD). The latter is included as an international forecaster to see if German forecast institutes have a possible advantage because of geographical proximity (see, e.g., Bae et al. 2008; Berger et al. 2009; Malloy 2005). Due to data availability, which varies across forecasters and decades, I focus on annual forecasts with a forecast horizon of one year and half-a-year, which are published at the turn of the year and mid-year, respectively. In order to minimize data revision effects in computing forecast errors, I use initial release data from the German statistical office to measure realized values of export and import growth (see (Statistisches Bundesamt 2018), data taken from “Wirtschaft und Statistik” publications). These realizations for the previous year are published by the German statistical office within the first months of a given year. Forecast errors are computed by subtracting these realized values for export or import growth from the forecast for the respective year (see also, Behrens et al. 2018a):
F E t ( h ) , i , j = y ^ t ( h ) , i , j y t + h , j ,
where y ^ denotes the annual forecast formed by forecaster i in year t (1970–2017), h is the forecast horizon (half-a-year, one year), j denotes the forecasted trade aggregate (export growth, import growth), and y denotes the realization of the respective trade aggregate. Shortly after the German reunification (i.e., between 1992 and 1993), the economic research institutes form forecasts for reunified Germany instead of only West-Germany. I, therefore, adapt the corresponding time series for realized export or import growth for each economic research institute.
Table 1 summarizes the descriptive statistics for the forecast errors of all forecasters in the sample. Most observations (N = 48) are available for the longer-term export and import growth forecasts of DIW and GD, whereas HWWI contributes the fewest observations (N = 31) for their shorter-term forecasts. For the majority of forecasters, the mean forecast errors are close to zero. It is, however, striking that the reported mean forecast errors are generally larger than the ones usually obtained in the standard analyses of German inflation and economic growth forecasts (see, among others, Behrens et al. 2018c; Döpke et al. 2017). The relatively high mean trade forecast errors are in line with recent research (Behrens 2019) and can be explained by the fact that exports and imports are among the most volatile GDP components (Döhrn and Schmidt 2011). Regarding the standard deviations, the results are, as one could have anticipated, generally larger for the longer-term forecasts. It is noteworthy that GD’s standard deviation is the highest for all but one forecasting scenario, in which it is second highest. This might be explained by the fact that GD is the only collaboration of forecasters in the sample and hence possible controversies among the participating forecasters regarding economic theory or estimation techniques might arise. However, it also needs to be taken into account that the GD forecasts are published on average two months before the other forecasts, resulting in a longer forecast horizon. In addition, I report coefficients of first-order autocorrelation, which hint at weak efficiency of the trade forecasts, as the coefficients are relatively small (see Section 4 for an in-depth analysis. Finally, the share of negative forecast errors is shown as an indicator of the symmetry of the forecasters’ loss functions. In case of symmetric loss, the F E < 0 -statistic should equal 0.5. For most forecasters, the share of negative forecast errors is close to the 0.5-benchmark.
The predictor variables I use to model a forecaster’s information set at the time of forecast formation are shown in Table 2. Recent studies on trade forecasts (D’Agostino et al. 2017) and their evaluation (Behrens 2019) show that typical trade variables as well as other macroeconomic variables are essential to accurately and efficiently predict future trade developments. My set of predictors, therefore, comprises, in addition to trade variables, several macroeconomic variables.
  • As trade variables I include (following Behrens 2019; D’Agostino et al. 2017): Export and import prices for Germany as well as German export and import volumes lagged one year and the German real effective exchange rate as a measure of international price competitiveness (Grimme et al. 2019; Lehmann 2015). I also include lags for the last four periods of the real effective exchange rate for Germany to account for a possible J-curve effect (see e.g., Hacker and Hatemi-J 2004).
  • As other macroeconomic variables, I include (following (Behrens et al. 2018b, 2018c, 2019; Döpke et al. 2017), who study German GDP and inflation forecasts): German industrial orders, German consumer and producer price indices, industrial production for Germany, the United States, France, the United Kingdom, Italy, and the Netherlands as leading indicators for Germany’s main trading partners (Guichard and Rusticelli 2011), as well as the oil price, German business tendency surveys for manufacturing (current and future tendency; on macroeconomic survey data as predictors, see (Frale et al. 2010; Lehmann 2015)), finally, I include the OECD leading indicator for Germany as a composite indicator.
Following Behrens et al. (2018a), predictor variables are lagged by one period to account for a publication lag of the forecasts. That is, I assume that a forecast which was published, for instance, in July is formed by means of information available in June. I furthermore account for different publication lags of the predictors based on research by Drechsel and Scheufele (2012). As real time data are not available at a monthly frequency for the examined time period, I minimize data revision effects of the affected variables by means of a backward looking moving-average of order 12 for the CPI, PPI, the real effective exchange rate, industrial productions, orders, and trade variables (following Behrens et al. 2018b; Pesaran and Timmermann 1995), in order to proxy the information set which was available to a forecaster when a given forecast was formed.

4. Empirical Results

The following empirical results were obtained by means of the R programming environment for statistical computing (R Core Team 2019) as well as the add-on packages “randomForest” (Liaw andWiener 2002) for estimating random trees and “rfUtilities” (Murphy et al. 2010; Evans and Murphy 2017) for computing permutation tests. Regarding the hyper-parameters of the random forests, I follow the common approach in the literature in setting the maximum number of terminal nodes of a given tree to five and setting the number of predictors used for growing a random forest to the square root of the total number of predictors (see, e.g., Behrens 2019; Hastie et al. 2009). For robustness checks, I report results for random forests, consisting of 250, 500, and 750 trees. Numbers in bold indicate a significantly (at the 5%-level) smaller out of bag misclassification error (classification problem) or a significantly larger pseudo R 2 (regression problem) of the original model compared to the model estimated by means of the permuted data. In other words, cases in which I reject forecast optimality are printed in bold.
The results for the basic specification of the model are reported in Table 3. The null hypothesis of forecast optimality cannot be rejected for DIW for all but one specification (longer-term import growth forecasts under quadratic loss, 750 trees). Regarding HWWI and ifo, the results are very interesting. Assuming a quadratic loss function for the forecasters results in the rejection of forecast optimality for both longer-term export and import growth forecasts, whereas, under flexible loss, forecast optimality cannot be rejected for these forecasts. Here, when evaluating the forecasting performance, the underlying loss function plays a crucial role. For IfW, I reject forecast optimality for their shorter-term export forecasts (500 and 750 trees) for both types of loss function. Forecast optimality for IfW’s longer-term export forecasts and all import forecasts cannot be rejected. The same applies to all trade forecasts of GD under quadratic as well as flexible loss. For OECD, the results are mixed. Under quadratic loss, I reject forecast optimality only for OECD’s longer-term import growth forecasts, whereas, under flexible loss, forecast optimality is rejected only for two cases (250, 750 trees) of their longer-term export growth forecasts. Overall, forecast optimality is rejected in more cases if the underlying loss function is assumed to be quadratic. Allowing for a flexible loss function results in a more favorable assessment of forecast optimality with respect to the forecasters. Additionally, for the forecasters’ shorter-term forecasts, optimality cannot be rejected even under quadratic loss, except for IfW’s export growth forecasts.
It is interesting to find out which predictors have explanatory power for the (sign of the) forecast error. I, therefore, report variable importance plots for the export growth forecast evaluation in Figure 2 and for the import growth forecast evaluation in Figure 3. Variable importance is computed by counting the times a predictor is chosen as splitting variable in the construction of a regression/classification tree. These measures are summed up over all trees and the share of each predictor as a splitting variable is computed by dividing by the number of trees in a forest, here 750. Hence, a large share indicates that this predictor has explanatory power for the (sign of the) forecast error. It is, therefore, not optimally incorporated into the respective forecast. On the other hand, if the share of a predictor variable is small, the predictor mentioned does not have explanatory power for the (sign of the) forecast error. It is, therefore, either optimally incorporated in the respective forecast or it does not have any explanatory power for the trade forecast. It is striking that, under quadratic loss, macroeconomic predictors other than typical trade aggregates (depicted in dark grey) are among the top splitting variables. In particular, macroeconomic survey indicators, such as the ifo business climate survey, are top splitting variables. Both findings are in line with recent research (Behrens 2019; D’Agostino et al. 2017). For most scenarios, lags of the German real effective exchange rate are among the top trade predictors (depicted in light grey), possibly hinting at a J-curve effect, which is not optimally incorporated in the analyzed trade forecasts. Another interesting observation is that, under flexible loss, the shares of the predictor variables as splitting variables are more evenly distributed. This is possibly due to the fact that the node impurity measure for regression trees (Equation (3)) punishes outliers more severely. Regression trees, therefore, split the feature space more often than classification trees, in order to minimize the effects of outliers on the RSS. Thus, top predictors are chosen as splitting variables multiple times when growing a regression tree and their overall share as splitting variables increases.
As further robustness checks, I report results estimated by means of random forest models with different hyperparameters in Table 4, Table 5 and Table 6. First, the maximum number of terminal nodes per tree is set to ten (Table 4). Second, I halve the number of predictors that are randomly chosen at each node in building a tree and report the results in Table 5. Finally, Table 6 shows results for a scenario in which the minimum number of observations at each terminal node is set to five. There are minor changes in the statistical significance of the permutation tests for most institutes, as can be expected when changing the model’s hyperparameters. For HWWI and ifo, I still observe that forecast optimality is rejected under quadratic loss for their longer-term export and import growth forecasts, whereas, when assuming flexible loss, I cannot reject forecast optimality in these cases. There are two exceptions for HWWI’s forecasts in scenario two—Table 5. For their longer-term import forecast computed with 500 trees and a quadratic loss function, the permutation test yields insignificant results, whereas the p-value for their shorter-term export forecast computed by means of 500 trees under flexible loss becomes significant compared to the basic specification. In scenario one (Table 4), there remains no evidence against the optimality of DIW’s longer-term import forecasts. For GD, the p-value for one of their shorter-term import forecasts under flexible loss (250 trees) becomes significant. The same is true for GD in scenario two (Table 5), in which also the p-value for one of their longer-term import growth forecasts under quadratic loss (750 trees) becomes significant. Overall, however, the main findings from the basic specification do not change qualitatively for all robustness scenarios. An underlying flexible loss function continues to result in more favorable assessments of the trade forecasts with respect to forecast optimality.
Recently, the effects of uncertainty and trade policy uncertainty in particular on the economy and on international trade have been analyzed by several studies (see, among others, Baker et al. 2016; Fernández-Villaverde et al. 2015; Hassan et al. 2017). Caldara et al. (2019) have developed a trade policy uncertainty index, which measures monthly media attention to news related to trade policy uncertainty. Electronic archives of seven leading U.S. newspapers are analyzed by means of automated text-search in order to construct the index (The list of newspapers comprises Boston Globe, Chicago Tribune, Guardian, Los Angeles Times, New York Times, Wall Street Journal, and Washington Post; for the data and further details on the construction of the index, see Caldara et al. (2019)). As U.S. trade policy uncertainty likely affects international as well as German trade, I add the trade policy uncertainty measure by Caldara et al. (2019) as well as a tariff volatility measure by the same authors to the list of predictors to proxy uncertainty regarding international trade and report the results in Table 7 (quarterly tariff volatility is estimated by a stochastic volatility model by Caldara et al. (2019)). The results mostly do not change qualitatively. An exception is IfW, for which shorter-term export growth forecasts under flexible loss are no longer significant. For ifo, the shorter-term import growth forecast under flexible loss computed by means of 250 trees is now significant compared to the basic specification. For the most part, the trade uncertainty measures do not seem to add further explanatory power for the forecast errors.
Next, I check if the forecasters form weakly efficient forecasts, where weak efficiency holds if a forecaster’s current forecast error cannot be explained by its preceding forecast error (see, among others, Behrens et al. 2018c; Öller and Barot 2000; Timmermann 2007). In other words, adding the lagged forecast error to the list of predictors should not lead to more efficient forecasts under the assumption of weak forecast efficiency. The results of the permutation tests reported in Table 8, hence, should not be more significant than in the basic specification. Here, I observe only minor changes for single specifications of DIW, HWWI, and OECD, suggesting that, for all forecasters, the lagged forecast errors do not seem to have further predictive value, such that the forecasts are in line with the concept of weak efficiency.
As one of the world’s largest exporters and due to interwoven transnational production chains, Germany’s exports and imports are interdependent. I, therefore, add each forecaster’s import growth forecast to the list of predictors for their export growth forecast and vice versa, in order to study if the respective trade forecast has predictive value for the other trade forecast and report the results in Table 9. Again, the results do not change qualitatively with respect to the basic specification. There are, however, three exceptions, where insignificant p-values in the basic specification become significant when adding the respective other trade forecast to the forecasters’ information set, namely IfW’s and GD’s import forecasts as well as OECD’s export forecasts all under flexible loss.
Based on work by Andrade and Le Bihan (2013) who find evidence on rational inattention of professional forecasters, I extend the forecasters’ information set by adding the lagged realizations of the predictor variables (see also (Behrens et al. 2018b), for an application to German GDP growth and inflation forecasts). The results presented in Table 10 show only minor changes in the magnitude of the permutation tests’ p-values. For two forecasters, results of one permutation test scenario become significant compared to the basic specification, namely DIW’s longer-term export forecasts (750 trees) and OECD’s shorter-term import forecasts (250 trees) both under quadratic loss. Forecast optimality continues to be rejected in more cases if the underlying loss function is assumed to be quadratic.
Recent research in the forecasting area has focused on possible effects of the 2007/2008 financial crisis on macroeconomic forecasting (see, for instance, Drechsel and Scheufele 2012; Frenkel et al. 2011; Müller et al. 2019). Table 11 reports results for a scenario, in which forecasts from the years 2007 and 2008 are deleted from the sample (I follow Behrens et al. 2018c, in this approach). In doing so, I check whether severe forecast errors made in these years affect the results of the entire sample. However, there are no qualitative changes in the results of the permutation tests. Only for OECD, one specification (250 trees) of their longer-term import growth forecasts under flexible loss, for which I cannot reject forecast optimality in the basic specification, now becomes significant.
In the same vein, I delete forecasts for the years 1992 and 1993 to check for possible biases due to large forecast errors in the time of German reunification (see also (Behrens 2019), the German statistical office changed national accounts statistics from west Germany to reunified Germany in 1992). The results documented in Table 12 are especially interesting for ifo, as forecast optimality for both longer-term trade forecasts under quadratic loss cannot be rejected anymore. This result documents the high volatility of results obtained from a scenario under the assumption of a quadratic loss function.
Finally, I report results for a scenario, in which I pool the data across forecast institutes, in order to enlarge the sample size. However, a disadvantage of this approach is that it is no longer possible to study the heterogeneity of the forecast institutes. Table 13 shows results for a pooled version of the basic specification computed by means of random forests consisting of 500 trees. In this scenario I add dummy variables for each institute to the list of predictors. In case of a classification problem, I report results for an alternative dependent variable, l F E t + 1 < 0 , since it is possible that some forecast errors are exactly equal to zero, as the institutes’ forecasts and the realizations of export and import growth are published as rounded two-digit numbers. I strongly reject forecast optimality for both trade aggregates, forecast horizons, types of loss function, and dependent variables.

5. Conclusions

I build on research by Behrens (2019) on the nonparametric evaluation of German trade forecasts. I extend the Behrens et al. (2018c) nonparametric adaption of the Patton and Timmermann (2007) approach to testing forecast optimality when the forecaster’s loss function is unknown. By means of random forests, I evaluate the optimality of German export growth and import growth forecasts, both under flexible and quadratic loss. To this end, I study annual trade forecasts from 1970 to 2017 published by four major German economic research institutes, one collaboration of German economic research institutes, and one international forecaster. A forecast is considered as optimal if a set of predictors, which models the information set of a forecaster at the time of forecast formation, has no explanatory power of the corresponding forecast error, in case of a regression problem under quadratic loss. For a classification problem, under flexible loss, the set of predictors should not have explanatory power for the sign of the forecast error, if the forecast is optimal.
For all but one forecaster, I cannot reject the optimality of export or import growth forecasts, neither under quadratic nor under flexible loss, at a forecast horizon of half-a-year. For longer-term forecasts, at a forecast horizon of one year, the results are more heterogeneous. For two forecasters, optimality of their longer-term trade forecasts is rejected under quadratic loss, whereas under flexible loss, forecast optimality cannot be rejected for these forecasters. Hence, the loss function plays a crucial role in the evaluation of these forecasters. Overall, an underlying flexible loss function results in a more favorable assessment of the trade forecasts with respect to forecast optimality. This analysis lends support to the study of Behrens (2019), who finds that macroeconomic predictors, other than typical trade aggregates, are not optimally incorporated in multivariate trade forecasts under quadratic loss. Variable importance plots show that, under quadratic loss, top predictors for the (sign of the) forecast error are not typical trade aggregates; however, under flexible loss, this finding cannot be confirmed. For these plots, the shares of predictors as splitting variables are more evenly distributed. Possibly, this is because, under quadratic loss, outliers are punished more severely, which in turn causes regression trees to split the feature space more often. Thus, top predictors are chosen more often as splitting variables than under flexible loss.
It needs to be mentioned that the results computed under flexible loss only hold if the assumptions underlying the Patton and Timmermann (2007) test for forecast optimality are fulfilled. It is, for instance, possible that the forecasters’ loss functions depend on other variables than the forecast error or that the loss functions are not homogeneous in the forecast error. It is, therefore, interesting for future research to see if these results hold for a multivariate analysis of both export and import growth forecasts under flexible loss. A multivariate analysis combining export and import growth forecasts would also be interesting when analyzing forecasts of a country’s trade balance, since this aggregate is often the subject of political debates. Furthermore, the nonparametric application of the Patton and Timmermann (2007) test of forecast optimality, brought forward by Behrens et al. (2018c), could be applied to other macroeconomic forecasts and/or countries.

Funding

This research was funded by the German Research Foundation (Exploring the experience expectation nexus in macroeconomic forecasting using computational text analysis and machine learning; Grant No.: 275693836).

Acknowledgments

I thank Christian Pierdzioch for very valuable and supportive feedback. I thank three anonymous reviewers for helpful and constructive comments. Furthermore, I thank the German Research Foundation for financial support. The usual disclaimer applies.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Andrade, Philippe, and Hervé Le Bihan. 2013. Inattentive professional forecasters. Journal of Monetary Economics 60: 967–82. [Google Scholar] [CrossRef]
  2. Angelini, Elena, Marta Bańbura, and Gerhard Rünstler. 2010. Estimating and forecasting the Euro Area monthly national accounts from a dynamic factor model. OECD Journal: Journal of Business Cycle Management and Analysis 1: 5–26. [Google Scholar] [CrossRef]
  3. Aretz, Kevin, Söhnke M. Bartram, and Peter F. Pope. 2011. Asymmetric loss functions and the rationality of expected stock returns. International Journal of Forecasting 27: 413–37. [Google Scholar] [CrossRef]
  4. Artis, Michael, and Massimiliano Marcellino. 2001. Fiscal forecasting: The track record of the IMF and OECD and EC. Econometrics Journal 4: 20–36. [Google Scholar] [CrossRef]
  5. Bae, Kee-Hong, René M. Stulz, and Hongping Tan. 2008. Do local analysts know more? A cross-country study of the performance of local analysts and foreign analysts. Journal of Financial Economics 88: 581–606. [Google Scholar] [CrossRef] [Green Version]
  6. Baker, Scott R., Nicholas Bloom, and Steven J. Davis. 2016. Measuring economic policy uncertainty. Quarterly Journal of Economics 131: 1593–636. [Google Scholar] [CrossRef]
  7. Behrens, Christoph. 2019. Evaluating the Joint Efficiency of German Trade Forecasts—A Nonparametric Multivariate Approach. Working Papers of the Priority Programme 1859, Experience and Expectation. Historical Foundations of Economic Behaviour. No. 9 (March). Berlin: Documentation and Publication Service of the Humboldt University. [Google Scholar]
  8. Behrens, Christoph, Christian Pierdzioch, and Marian Risse. 2018a. Do German economic research institutes publish efficient growth and inflation forecasts? A Bayesian analysis. Journal of Applied Statistics. forthcoming. [Google Scholar] [CrossRef]
  9. Behrens, Christoph, Christian Pierdzioch, and Marian Risse. 2018b. A test of the joint efficiency of macroeconomic forecasts using multivariate random forests. Journal of Forecasting 37: 560–72. [Google Scholar] [CrossRef]
  10. Behrens, Christoph, Christian Pierdzioch, and Marian Risse. 2018c. Testing the optimality of inflation forecasts under flexible loss with random forests. Economic Modelling 72: 270–77. [Google Scholar] [CrossRef]
  11. Berger, Helge, Michael Ehrmann, and Marcel Fratzscher. 2009. Forecasting ECB monetary policy: Accuracy is a matter of geography. European Economic Review 53: 1028–41. [Google Scholar] [CrossRef]
  12. Breiman, Leo. 2001. Random forests. Machine Learning 45: 5–32. [Google Scholar] [CrossRef]
  13. Breiman, Leo, Jerome Friedman, Charles J. Stone, and Richard A. Olshen. 1984. Classification and Regression Trees. New York: CRC Press. [Google Scholar]
  14. Caldara, Dario, Matteo Iacoviello, Patrick Molligo, Andrea Prestipino, and Andrea Raffo. 2019. The economic effects of trade policy uncertainty. Manuscript presented at the 91st meeting of the Carnegie-Rochester-NYU Conference on Public Policy, held at New York University, April 12–13. [Google Scholar]
  15. Capistrán, Carlos. 2008. Bias in Federal Reserve inflation forecasts: Is the Federal Reserve irrational or just cautious? Journal of Monetary Economics 55: 1415–27. [Google Scholar] [CrossRef]
  16. Carroll, Christopher D. 2003. Macroeconomic expectations of households and professional forecasters. The Quarterly Journal of Economics 118: 269–98. [Google Scholar] [CrossRef]
  17. Christodoulakis, George A., and Emmanuel C. Mamatzakis. 2008. An assessment of the EU growth forecasts under asymmetric preferences. Journal of Forecasting 27: 483–92. [Google Scholar] [CrossRef]
  18. Christoffersen, Peter F., and Francis X. Diebold. 1996. Further results on forecasting and model selection under asymmetric loss. Journal of Applied Econometrics 11: 561–71. [Google Scholar] [CrossRef]
  19. Christoffersen, Peter F., and Francis X. Diebold. 1997. Optimal prediction under asymmetric loss. Econometric Theory 13: 808–17. [Google Scholar] [CrossRef]
  20. D’Agostino, Antonello, Michele Modugno, and Chiara Osbat. 2017. A global trade model for the Euro Area. International Journal of Central Banking 13: 1–34. [Google Scholar]
  21. Drechsel, Katja, and Rolf Scheufele. 2012. The performance of short-term forecasts of the German economy before and during the 2008/2009 recession. International Journal of Forecasting 28: 428–45. [Google Scholar] [CrossRef]
  22. Döhrn, Roland, and Christoph M. Schmidt. 2011. Information or institution? On the determinants of forecast accuracy. Journal of Economics and Statistics 231: 9–27. [Google Scholar]
  23. Döpke, Jörg, and Ulrich Fritsche. 2006. Growth and inflation forecasts for Germany—A panel-based assessment of accuracy and efficiency. Empirical Economics 31: 777–98. [Google Scholar] [CrossRef]
  24. Döpke, Jörg, Ulrich Fritsche, and Christian Pierdzioch. 2017. Predicting recession with boosted regression trees. International Journal of Forecasting 33: 745–59. [Google Scholar] [CrossRef]
  25. Döpke, Jörg, Karsten Müller, and Lars Tegtmeier. 2018. The economic value of business cycle forecasts for potential investors—Evidence from Germany. Research in International Business and Finance 46: 445–61. [Google Scholar] [CrossRef]
  26. Durusoy, Serap, Edgardo Sica, and Zeynep Beyhan. 2015. Economic crisis and protectionism policies: The case of the EU countries. International Journal of Humanities and Social Science 5: 57–68. [Google Scholar]
  27. Elliott, Graham, Ivana Komunjer, and Allan Timmermann. 2005. Estimation and testing of forecast rationality under flexible loss. Review of Economic Studies 72: 1107–25. [Google Scholar] [CrossRef]
  28. Elliott, Graham, Ivana Komunjer, and Allan Timmermann. 2008. Biases in macroeconomic forecasts: Irrationality or asymmetric loss? Journal of the European Economic Association 6: 122–57. [Google Scholar] [CrossRef]
  29. Elstner, Steffen, Christian Grimme, and Ulrich Haskamp. 2013. Das ifo Exportklima - ein Frühindikator für die deutsche Exportprognose. ifo Schnelldienst 66: 36–43. [Google Scholar]
  30. Engelke, Carola, Katja Heinisch, and Christoph Schult. 2019. Effects of External Assumptions on Forecast Errors. IWH Discussion Paper. Halle: Halle Institute for Economic Research, forthcoming. [Google Scholar]
  31. Evans, Jeffrey S., and Melanie A. Murphy. 2017. rfutilities. R Package Version 2.1-0. Available online: https://cran.r-project.org/package=rfUtilities (accessed on 28 March 2018).
  32. Fernández-Villaverde, Jesús, Pablo Guerrón-Quintana, Keith Kuester, and Juan Rubio-Ramírez. 2015. Fiscal volatility shocks and economic activity. American Economic Review 105: 3352–84. [Google Scholar] [CrossRef]
  33. Frale, Cecilia, Massimiliano Marcellino, Gian Luigi Mazzi, and Tommaso Proietti. 2010. Survey data as coincident or leading indicators. Journal of Forecasting 29: 109–31. [Google Scholar] [CrossRef]
  34. Frenkel, Michael, Eliza M. Lis, and Jan-Christoph Rülke. 2011. Has the economic crisis of 2007–2009 changed the expectation formation process in the euro area? Economic Modelling 28: 1808–14. [Google Scholar] [CrossRef]
  35. Fritsche, Ulrich, Christian Pierdzioch, Jan-Christoph Rülke, and Georg Stadtmann. 2015. Forecasting the Brazilian real and the Mexican peso: Asymmetric loss, forecast rationality, and forecaster herding. International Journal of Forecasting 31: 130–39. [Google Scholar] [CrossRef]
  36. Granger, Clive W. J. 1969. Prediction with a generalized cost of error function. Operational Research Society 20: 199–207. [Google Scholar] [CrossRef]
  37. Grimme, Christian, Robert Lehmann, and Marvin Noeller. 2019. Forecasting Imports with Information from Abroad. Ifo Working Paper No. 7079. Munich: Ifo Institute - Leibniz Institute for Economic Research at the University of Munich. [Google Scholar]
  38. Grossmann, Sandra Hanslin, and Rolf Scheufele. 2019. PMIs: Reliable indicators for exports? Review of International Economics. forthcoming. [Google Scholar] [CrossRef]
  39. Guichard, Steéphanie, and Elena Rusticelli. 2011. A dynamic factor model for world trade growth. OECD Economics Department Working Papers No. 874. [Google Scholar]
  40. Hacker, R. Scott, and Abdulnasser Hatemi-J. 2004. The effect of exchange rate changes on trade balances in the short and long run—Evidence from German trade with transitional Central European economies. Economics of Transition 12: 777–79. [Google Scholar] [CrossRef]
  41. Hassan, Tarek A., Stephan Hollander, Laurence van Lent, and Ahmed Tahoun. 2017. Firm-Level Political Risk: Measurement and Effects. National Bureau of Economic Research Working Paper 24029. Cambridge: National Bureau of Economic Research. [Google Scholar]
  42. Hastie, Trevor, Robert Tibshirani, and Jerome Friedman. 2009. The Elements of Statistical Learning. Data Mining, Inference, and Prediction, 2nd ed. New York: Springer. [Google Scholar]
  43. Heilemann, Ulrich, and Herman O. Stekler. 2013. Has the accuracy of German macroeconomic forecasts improved? German Economic Review 14: 235–53. [Google Scholar] [CrossRef]
  44. Heinisch, Katja, and Rolf Scheufele. 2018. Bottom-up or direct? Forecasting German GDP in a data-rich environment. Empirical Economics 54: 705–45. [Google Scholar] [CrossRef]
  45. Hervé, Karine, Nigel Pain, Pete Richardson, Franck Sédillot, and Pierre-Olivier Beffy. 2011. The OECD’s new global model. Economic Modelling 28: 289–601. [Google Scholar] [CrossRef]
  46. Hetemäki, Lauri, and Jarmo Mikkola. 2005. Forecasting Germany’s printing and writing paper imports. Forest Science 51: 483–97. [Google Scholar]
  47. Ito, Takatoshi. 1990. Foreign exchange rate expectations: Micro survey data. The American Economic Review 80: 434–49. [Google Scholar]
  48. Jakaitiene, Audrone, and Stéphane Dées. 2012. Forecasting the world economy in the short term. The World Economy 35: 331–50. [Google Scholar] [CrossRef]
  49. James, Gareth, Daniela Witten, Trevor Hastie, and Robert Tibshirani. 2015. An Introduction to Statistical Learning. New York and Heidelberg: Springer. [Google Scholar]
  50. Jannsen, Nils, and Julia Richter. 2012. Foreign capacity utilisation and German exports of capital goods. Wirtschaftsdienst 92: 833–37. [Google Scholar] [CrossRef]
  51. Keck, Alexander, Alexander Raubold, and Alessandro Truppia. 2009. Forecasting international trade: A time series approach. OECD Journal. Journal of Business Cycle Measurement and Analysis 2: 157–76. [Google Scholar] [CrossRef]
  52. Kirchgässner, Gebhard. 1984. Wie gut sind die Prognosen der Arbeitsgemeinschaft wirtschaftswissenschaftlicher Forschungsinstitute in der Bundesrepublik Deutschland? Review of World Economics 120: 279–300. [Google Scholar] [CrossRef]
  53. Kirchgässner, Gebhard. 1993. Testing weak rationality of forecasts with different time horizons. Journal of Forecasting 12: 541–58. [Google Scholar] [CrossRef]
  54. Kirchgässner, Gebhard, and Ulrich K. Müller. 2006. Are forecasters reluctant to revise their predictions? Some German evidence. Journal of Forecasting 25: 401–23. [Google Scholar] [CrossRef]
  55. Kirchgässner, Gebhard, and Marcel Savioz. 2001. Monetary policy forecasts for real GDP growth—An empirical investigation for the Federal Republic of Germany. German Economic Review 2: 339–65. [Google Scholar] [CrossRef]
  56. Lehmann, Robert. 2015. Survey-Based Indicators vs. Hard Data: What Improves Export Forecasts in Europe? ifo Working Paper No. 196. Munich: Ifo Institute - Leibniz Institute for Economic Research at the University of Munich. [Google Scholar]
  57. Liaw, Andy, and Matthew Wiener. 2002. Classification and regression by randomforest. R News 2: 18–22. [Google Scholar]
  58. Malloy, Christopher J. 2005. The geography of equity analysis. Journal of Finance 60: 719–55. [Google Scholar] [CrossRef]
  59. Müller, Karsten, Ulrich Fritsche, and Jörg Döpke. 2019. Has macroeconomic forecasting chnaged after the Great Recession? Panel-based evidence on forecast accuracy and forecaster behaviour from Germany. Journal of Macroeconomics. forthcoming. [Google Scholar]
  60. Murphy, Melanie A, Jeffrey S. Evans, and Andrew Storfer. 2010. Quantifying Bufo boreas connectivity in Yellowstone National Park with landscape genetics. Ecology 91: 252–61. [Google Scholar] [CrossRef]
  61. Neumann, Manfred J. M., and Herbert S. Buscher. 1980. Die Inflationsprognosen der Arbeitsgemeinschaft deutscher wirtschaftswissenschaftlicher Forschungsinstitute: Sind sie “rational”? Review of World Economics 116: 533–50. [Google Scholar] [CrossRef]
  62. Öller, Lars-Erik, and Bharat Barot. 2000. The accuracy of european growth and inflation forecasts. International Journal of Forecasting 16: 293–315. [Google Scholar] [CrossRef]
  63. Park, Sang-Chul. 2018. U.S. protectionism and trade imbalance between the U.S. and northeast asian countries. International Organisations Research Journal 13: 76–100. [Google Scholar] [CrossRef]
  64. Patton, Andrew J., and Allan Timmermann. 2007. Testing forecasting optimality under unknown loss. Journal of the American Statistical Association 102: 1172–84. [Google Scholar] [CrossRef]
  65. Pelagidis, Theodore. 2018. Free trade, protectionism, and the global economy. World Economics 19: 121–28. [Google Scholar]
  66. Pesaran, Hashem M., and Allan Timmermann. 1995. Predictability of stock returns: Robustness and economic significance. Journal of Finance 50: 1201–28. [Google Scholar] [CrossRef]
  67. Pierdzioch, Christian, and Jan-Christoph Rülke. 2013. A note on forecasting the rate of change fo the price of oil: Asymmetric loss and forecast rationality. Economies 1: 6–13. [Google Scholar] [CrossRef]
  68. Pierdzioch, Christian, Monique B. Reid, and Rangan Gupta. 2016a. Forecasting the South African inflation rate: On asymmetric loss and forecast rationality. Economic Systems 40: 82–92. [Google Scholar] [CrossRef]
  69. Pierdzioch, Christian, Jan-Christoph Rülke, and Peter Tillman. 2016b. Using forecasts to uncover the loss function of federal open market committee members. Macroeconomic Dynamics 20: 791–818. [Google Scholar] [CrossRef]
  70. R Core Team. 2019. R: A Language and Environment for Statistical Computing, R: Version 3.6.0. Vienna: R Foundation for Statistical Computing. [Google Scholar]
  71. Riad, Nagwa, Luca Errico, Christian Henn, Christian Saborowski, Mika Saito, and Jarkko Turunen. 2012. Changing Patterns of Global Trade. Washington: International Monetary Fund. [Google Scholar]
  72. Sinclair, Tara, Herman O. Stekler, and Hans Christian Müller-Dröge. 2016. Evaluating forecasts of a vector of variables: A German forecasting competition. Journal of Forecasting 35: 493–503. [Google Scholar] [CrossRef]
  73. Statistisches Bundesamt. 2018. Wirtschaft und Statistik. Available online: https://www.destatis.de/GPStatistik/receive/DESerie_serie_00000012?list=all (accessed on 28 March 2018).
  74. Sun, Yuying, Shouyang Wang, and Xun Zhang. 2018. How efficient are china’s macroeconomic forecasts? evidences from a new forecasting evaluation approach. Economic Modelling 68: 506–13. [Google Scholar] [CrossRef]
  75. Timmermann, Allan. 2007. An evaluation of the World Economic Outlook forecasts. IMF Staff Papers 54: 1–33. [Google Scholar] [CrossRef]
  76. Varian, Hal R. 1974. A Bayesian approach to real estate assessment. In Studies in Bayesian Econometrics and Statistics—In Honor of Leonard J. Savage. Edited by Arnold Zellner and Stephen E. Fienberg. Amsterdam: Nort-Holland, vol. 1, pp. 195–208. [Google Scholar]
  77. Vosen, Simeon, and Torsten Schmidt. 2011. Forecasting private consumption: Survey-based indicators vs. Google Trends. Journal of Forecasting 30: 565–78. [Google Scholar] [CrossRef]
  78. Yu, Lean, Shouyang Wang, and Kin Keung Lai. 2008. Forecasting China’s foreign trade volume with a kernel-based hybrid econometric-AI ensemble learning approach. Journal of Systems and Complexity 21: 1–19. [Google Scholar] [CrossRef]
  79. Zellner, Arnold. 1986. Bayesian estimation and prediction using asymmetric loss functions. Journal of the American Statistical Association 81: 446–51. [Google Scholar] [CrossRef]
Figure 1. Data Segmentation of an Exemplary Classification or Regression Tree—Left panel: Exemplary classification/regression tree with partitioning predictors, z t , cutpoints, c k , and regions, R j . With t = 1 , 2 , k = 1 4 , j = 1 5 . Right panel: Partitioned feature space with the respective regions. Example based on Hastie et al. (2009, Figure 9.2).
Figure 1. Data Segmentation of an Exemplary Classification or Regression Tree—Left panel: Exemplary classification/regression tree with partitioning predictors, z t , cutpoints, c k , and regions, R j . With t = 1 , 2 , k = 1 4 , j = 1 5 . Right panel: Partitioned feature space with the respective regions. Example based on Hastie et al. (2009, Figure 9.2).
Economies 07 00093 g001
Figure 2. Variable importanceplots for random forests evaluating export growth forecasts—variable importance is measured by counting the times a predictor is selected as splitting variable in a regression or classification tree. Variable importance measures of the respective trees are summed up and the share of each predictor as a splitting variable in a random forest is computed by dividing by the number of trees in the random forest, here 750. Light grey bars indicate trade variables, dark grey bars indicate other macroeconomic variables.
Figure 2. Variable importanceplots for random forests evaluating export growth forecasts—variable importance is measured by counting the times a predictor is selected as splitting variable in a regression or classification tree. Variable importance measures of the respective trees are summed up and the share of each predictor as a splitting variable in a random forest is computed by dividing by the number of trees in the random forest, here 750. Light grey bars indicate trade variables, dark grey bars indicate other macroeconomic variables.
Economies 07 00093 g002
Figure 3. Variable importance plots for random forests evaluating import growth forecasts—Variable importance is measured by counting the times a predictor is selected as splitting a variable in a regression or classification tree. Variable importance measures of the respective trees are summed up and the share of each predictor as a splitting variable in a random forest is computed by dividing by the number of trees in the random forest, here 750. Light grey bars indicate trade variables; dark grey bars indicate other macroeconomic variables.
Figure 3. Variable importance plots for random forests evaluating import growth forecasts—Variable importance is measured by counting the times a predictor is selected as splitting a variable in a regression or classification tree. Variable importance measures of the respective trees are summed up and the share of each predictor as a splitting variable in a random forest is computed by dividing by the number of trees in the random forest, here 750. Light grey bars indicate trade variables; dark grey bars indicate other macroeconomic variables.
Economies 07 00093 g003
Table 1. Descriptive statistics of forecast errors.
Table 1. Descriptive statistics of forecast errors.
ForecasterHorizonNMEANSDAC FE < 0 NMEANSDAC FE < 0
ExportsImports
DIWShorter-term360.202.350.030.47360.712.850.100.42
Longer-term48−0.785.05−0.150.6248−0.283.98−0.060.56
HWWIShorter-term31−0.112.44−0.010.58310.242.79−0.100.39
Longer-term43−0.145.04−0.310.51420.354.01−0.260.45
ifoShorter-term41−0.062.40−0.080.54410.022.29−0.200.41
Longer-term44−0.504.73−0.160.6144−0.093.52−0.150.50
IfWShorter-term390.262.76−0.070.41390.572.40−0.090.38
Longer-term47−0.844.280.080.60470.083.470.100.53
GDShorter-term46−0.583.77−0.090.5747−0.103.19−0.090.49
Longer-term48−0.165.35−0.120.56480.134.25−0.030.50
OECDShorter-term460.383.21−0.070.43460.293.54−0.250.43
Longer-term460.095.53−0.280.50460.294.20−0.190.48
Notes: N: Number of observations, MEAN: Arithmetic mean, SD: Standard deviation, AC: First, order autocorrelation, F E < 0 : Share of negative forecast errors.
Table 2. Predictor variables.
Table 2. Predictor variables.
PredictorsGrouplnLagDescriptionSource
Production GermanyM - RealY1Year-on-year rate of change of the monthlyOECD
German total manufacturing output.
Production United StatesM - RealY1Year-on-year rate of change of the monthlyOECD
U.S. total manufacturing output.
Production FranceM - RealY1Year-on-year rate of change of the monthlyOECD
French total manufacturing output.
Production United KingdomM - RealY1Year-on-year rate of change of the monthlyOECD
U.K. total manufacturing output.
Production ItalyM - RealY1Year-on-year rate of change of the monthlyOECD
Italian total manufacturing output.
Production NetherlandsM - RealY1Year-on-year rate of change of the monthlyOECD
Dutch total manufacturing output.
Order inflowM - RealY1Year-on-year rate of change of the industrial ordersBUBA
received for Germany; calendar and seasonally adjusted.
UnemploymentM - RealN1Monthly unemployment rate (in percent of civilianBUBA
labor) for Germany; calendar and seasonally adjusted.
Oil priceM - PricesY0Year-on-year rate of change of the monthlyFRED
crude oil price (WTI); dollars per barrel.
ClimateM - SurveyN0Monthly ifo business tendency survey for manufacturing forFRED
Germany; half-a-year-ahead tendency, seasonally adjusted.
Climate (expectations)M - SurveyN0Monthly ifo business tendency survey for manufacturing forFRED
Germany; situation in six months; seasonally adjusted.
OECD leading (normalized)M - CompositeN2Monthly normalized OECD compositeOECD
leading indicator for Germany.
Real effective exchange rateT - PricesY1Year-on-year rate of change of the monthly narrowBIS
effective exchange rate for Germany; CPI-based.
ExportsT - RealY1212-months-lag of the year-on-year rate of change ofFRED
German value goods exports; seasonally adjusted.
ImportsT - RealY1212-months-lag of the year-on-year rate of change ofFRED
German value goods imports; seasonally adjusted.
Export PricesT - PricesY1Year-on-year rate of change of the monthly index of GermanDESTATIS
export prices; standard international trade classification.
Import PricesT - PricesY1Year-on-year rate of change of the monthly index of GermanDESTATIS
import prices; standard international trade classification.
Consumer PricesM - PricesY0Year-on-year rate of change of the monthly GermanBUBA
consumer price index; calendar and seasonally adjusted.
Producer PricesM - PricesY0Year-on-year rate of change of the monthly GermanFRED
domestic producer price index for manufacturing.
Notes: BIS - Bank for International Settlements, https://www.bis.org/statistics/index.htm; BUBA - German Central Bank, http://www.bundesbank.de/Navigation/EN/Statistics/statistics.html; DESTATIS - Federal Statistical Office of Germany, https://www.destatis.de/EN/FactsFigures/FactsFigures.html; FRED - Federal Reserve Bank of St. Louis, https://fred.stlouisfed.org/; OECD - Organisation for Economic Co-operation and Development, http://stats.oecd.org/. ln: natural logarithmic transformation, Y - yes, N - no, M - Macroeconomic variable, T - Trade variable. − Lag: Publication lags in months added where necessary (Drechsel and Scheufele 2012, Table 3).
Table 3. Basic specification.
Table 3. Basic specification.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Shorter-term 0.8010.7650.804 0.1460.2150.166 0.8520.8130.565 0.3810.5150.415
Longer-term 0.1050.5490.196 0.0930.0570.003 0.3380.4510.484 0.3580.1300.178
HWWI
Forecast Exports Imports Exports Imports
Shorter-term 0.1640.2610.275 0.3090.3260.232 0.1020.0910.244 0.3180.2890.400
Longer-term 0.0260.0110.018 0.0180.0060.009 0.9130.6910.854 0.4240.2400.184
ifo
Forecast Exports Imports Exports Imports
Shorter-term 0.7920.7450.837 0.9450.9920.966 0.8190.8860.830 0.3130.4210.302
Longer-term 0.0010.0030.018 0.0030.0010.005 0.2600.1900.183 0.8100.8190.800
IfW
Forecast Exports Imports Exports Imports
Shorter-term 0.0540.0460.048 0.4770.4480.454 0.1820.0170.019 0.6560.5520.304
Longer-term 0.1090.1410.116 0.1620.1400.120 0.4440.2650.620 0.0870.0600.146
GD
Forecast Exports Imports Exports Imports
Shorter-term 0.3460.2980.533 0.5870.7940.591 0.8190.7730.542 0.0640.1780.091
Longer-term 0.5510.3860.343 0.0790.0640.079 0.8410.6210.622 0.5560.7150.370
OECD
Forecast Exports Imports Exports Imports
Shorter-term 0.5080.4980.342 0.1910.3090.160 0.1690.1060.158 0.4260.3170.107
Longer-term 0.1090.0790.113 0.0020.0010.002 0.0390.0910.035 0.2020.2160.145
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 4. Alternative hyperparameters—Maximum number of terminal nodes.
Table 4. Alternative hyperparameters—Maximum number of terminal nodes.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Shorter-term 0.9390.9360.864 0.2060.1590.190 0.5490.7920.800 0.3970.2820.587
Longer-term 0.3080.2930.228 0.2590.1040.091 0.2170.4500.228 0.0650.1050.181
HWWI
Forecast Exports Imports Exports Imports
Shorter-term 0.3850.2890.334 0.3330.2830.296 0.5020.1650.250 0.6600.5280.426
Longer-term 0.0200.0190.009 0.0200.0280.018 0.9000.6940.684 0.3450.2040.263
ifo
Forecast Exports Imports Exports Imports
Shorter-term 0.9390.7930.830 0.9450.9010.927 0.8350.9800.978 0.1690.1590.278
Longer-term 0.0110.0150.010 0.0210.0090.006 0.1870.3870.375 0.7900.5390.848
IfW
Forecast Exports Imports Exports Imports
Shorter-term 0.0440.0530.040 0.4610.4950.500 0.1970.0490.015 0.6390.5360.769
Longer-term 0.2200.1520.181 0.1210.2040.214 0.5770.5930.366 0.1460.2790.090
GD
Forecast Exports Imports Exports Imports
Shorter-term 0.2960.5370.423 0.5710.9270.732 0.8670.8820.701 0.0460.0840.182
Longer-term 0.3890.4530.336 0.1450.0650.067 0.5110.5120.601 0.4370.6410.639
OECD
Forecast Exports Imports Exports Imports
Shorter-term 0.3640.3340.413 0.2080.0980.164 0.0660.2220.108 0.1750.4580.251
Longer-term 0.0420.1410.100 0.0010.0010.001 0.0850.0250.015 0.0840.2110.086
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 5. Alternative hyperparameters—Half of the chosen predictors.
Table 5. Alternative hyperparameters—Half of the chosen predictors.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Shorter-term 0.9610.9070.910 0.1880.1470.218 0.8130.8800.548 0.5140.3940.282
Longer-term 0.3300.0780.185 0.0560.0490.064 0.1450.2120.363 0.1900.3540.183
HWWI
Forecast Exports Imports Exports Imports
Shorter-term 0.2940.2160.227 0.6310.3230.313 0.1080.0460.088 0.6460.5290.408
Longer-term 0.0170.0170.008 0.0390.0550.009 0.8880.8940.746 0.2790.3550.257
ifo
Forecast Exports Imports Exports Imports
Shorter-term 0.9300.9390.892 0.9760.9880.994 0.9730.7610.929 0.4280.2480.229
Longer-term 0.0090.0130.013 0.0430.0090.007 0.3630.4680.252 0.7130.7130.623
IfW
Forecast Exports Imports Exports Imports
Shorter-term 0.0260.0690.012 0.5940.4980.478 0.0440.0390.007 0.7870.7700.539
Longer-term 0.3710.1570.290 0.2900.2410.230 0.6880.7610.812 0.1810.1330.218
GD
Forecast Exports Imports Exports Imports
Shorter-term 0.6930.3670.449 0.9000.8090.786 0.7190.8950.794 0.0290.1100.067
Longer-term 0.5310.4500.303 0.0810.0660.039 0.6940.7950.532 0.4580.6360.686
OECD
Forecast Exports Imports Exports Imports
Shorter-term 0.2110.4020.322 0.0410.0810.072 0.0810.1460.103 0.0780.1140.079
Longer-term 0.0850.0600.059 0.0030.0030.001 0.0370.0260.019 0.1420.1420.144
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 6. Alternative hyperparameters—Minimum number of observations at terminal nodes.
Table 6. Alternative hyperparameters—Minimum number of observations at terminal nodes.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Shorter-term 0.9960.8990.882 0.3740.1760.198 0.5370.6090.565 0.2660.6070.494
Longer-term 0.2900.1880.294 0.1040.0880.046 0.3220.1380.148 0.2730.1290.071
HWWI
Forecast Exports Imports Exports Imports
Shorter-term 0.3170.3130.288 0.5730.3800.357 0.3420.3610.095 0.4380.3120.543
Longer-term 0.0280.0130.030 0.0110.0110.009 0.8260.8830.888 0.2560.0800.218
ifo
Forecast Exports Imports Exports Imports
Shorter-term 0.8490.9400.910 0.9310.9430.971 0.9680.9690.936 0.4280.2480.229
Longer-term 0.0160.0100.022 0.0240.0090.007 0.6120.4690.500 0.7930.7210.777
IfW
Forecast Exports Imports Exports Imports
Shorter-term 0.0300.0210.032 0.5990.5940.556 0.0940.0690.041 0.5120.4190.520
Longer-term 0.1870.2340.180 0.3080.1570.185 0.7700.7400.699 0.0950.2180.137
GD
Forecast Exports Imports Exports Imports
Shorter-term 0.2210.5280.523 0.7260.7610.834 0.8520.8530.869 0.2230.1310.102
Longer-term 0.3590.6060.331 0.1130.1090.118 0.7150.6880.443 0.6090.7170.628
OECD
Forecast Exports Imports Exports Imports
Shorter-term 0.4460.4040.348 0.2410.1010.206 0.1850.2150.170 0.7330.0670.509
Longer-term 0.1600.0920.081 0.0020.0020.002 0.0210.0300.038 0.3680.0800.155
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 7. Including trade uncertainty measures.
Table 7. Including trade uncertainty measures.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Shorter-term 0.9590.9540.835 0.1500.1780.140 0.4570.7330.737 0.4040.5810.693
Longer-term 0.1320.1940.197 0.1160.0110.025 0.3390.3500.224 0.1430.1930.088
HWWI
Forecast Exports Imports Exports Imports
Shorter-term 0.2890.2540.299 0.3750.3310.374 0.2550.1590.103 0.3950.4090.537
Longer-term 0.0150.0290.023 0.0220.0200.019 0.7480.6850.884 0.2380.2680.245
ifo
Forecast Exports Imports Exports Imports
Shorter-term 0.8580.9200.818 0.9520.9580.949 0.8550.8970.819 0.0390.0690.111
Longer-term 0.0070.0050.010 0.0070.0060.012 0.2760.5140.375 0.2420.3070.620
IfW
Forecast Exports Imports Exports Imports
Shorter-term 0.0380.0190.049 0.3460.2920.314 0.1850.0700.079 0.4030.3920.407
Longer-term 0.1840.1110.149 0.2020.1630.178 0.3970.4830.371 0.1000.1350.185
GD
Forecast Exports Imports Exports Imports
Shorter-term 0.4570.3400.224 0.7800.6970.621 0.8580.7660.803 0.1800.0680.185
Longer-term 0.7690.5540.762 0.0700.0940.346 0.4040.8690.702 0.5650.7180.782
OECD
Forecast Exports Imports Exports Imports
Shorter-term 0.5050.4040.503 0.0550.2710.294 0.1350.1790.400 0.6480.3590.392
Longer-term 0.0670.0770.080 0.0010.0010.001 0.0060.0690.015 0.0880.1980.145
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 8. Including the lagged forecast error.
Table 8. Including the lagged forecast error.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Short-term 0.9360.9330.866 0.2700.2850.162 0.7740.8770.843 0.4970.6050.359
Long-term 0.3220.4200.265 0.0100.0330.055 0.2460.3510.465 0.1760.2550.256
HWWI
Forecast Exports Imports Exports Imports
Short-term 0.1580.1510.236 0.4550.4360.403 0.1440.0770.034 0.3160.4320.440
Long-term 0.0130.0090.008 0.0090.0160.016 0.6310.8450.774 0.1590.1940.276
ifo
Forecast Exports Imports Exports Imports
Short-term 0.7590.7300.807 0.9320.8850.834 0.7740.8640.855 0.5120.3510.215
Long-term 0.0080.0690.017 0.0070.0090.007 0.1160.5040.503 0.8560.7120.843
IfW
Forecast Exports Imports Exports Imports
Short-term 0.4020.1400.113 0.4600.4110.470 0.1590.0900.108 0.2780.2890.184
Long-term 0.2790.2850.213 0.0800.1070.121 0.2980.5270.604 0.1710.0540.047
GD
Forecast Exports Imports Exports Imports
Short-term 0.5040.5110.492 0.6120.7180.821 0.4440.5350.668 0.2970.2220.034
Long-term 0.4920.3960.426 0.0830.0780.068 0.8560.8170.866 0.5440.4520.426
OECD
Forecast Exports Imports Exports Imports
Short-term 0.2120.3960.333 0.1830.1920.227 0.1360.0750.189 0.0630.2420.244
Long-term 0.0450.0720.517 0.0050.0050.004 0.0170.0430.018 0.0800.0880.133
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 9. Including the other trade forecast.
Table 9. Including the other trade forecast.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Short-term 0.7750.7410.624 0.2920.2680.173 0.6520.7420.811 0.4970.6050.359
Long-term 0.2140.1830.314 0.0150.0510.016 0.4770.3290.222 0.1760.2550.256
HWWI
Forecast Exports Imports Exports Imports
Short-term 0.2910.2000.284 0.3050.4070.456 0.3430.0950.336 0.3160.4320.440
Long-term 0.0150.0210.026 0.0050.0080.006 0.5530.6120.537 0.1590.1940.276
ifo
Forecast Exports Imports Exports Imports
Short-term 0.9190.9270.806 0.9640.9420.966 0.9220.9600.830 0.5120.3510.215
Long-term 0.0190.0110.002 0.0100.0110.019 0.2520.6360.378 0.8560.7120.843
IfW
Forecast Exports Imports Exports Imports
Short-term 0.0770.0420.037 0.4740.4790.413 0.1890.0160.038 0.2780.2890.184
Long-term 0.1790.2180.152 0.1200.1650.142 0.6130.4630.380 0.1710.0540.047
GD
Forecast Exports Imports Exports Imports
Short-term 0.6810.4610.581 0.8410.7950.902 0.7420.8730.789 0.2970.2220.034
Long-term 0.5360.4090.569 0.0850.0850.084 0.6180.5270.338 0.5440.4520.426
OECD
Forecast Exports Imports Exports Imports
Short-term 0.5470.2430.450 0.1430.1860.065 0.1540.0210.109 0.0630.2420.244
Long-term 0.1490.1490.189 0.0010.0010.001 0.0810.0250.021 0.0800.0880.133
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 10. Including additional lagged predictors.
Table 10. Including additional lagged predictors.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Short-term 0.9030.8660.819 0.3230.3390.292 0.7360.9290.614 0.4870.3730.231
Long-term 0.0790.0690.023 0.0080.0110.008 0.3100.1320.454 0.3560.1950.292
HWWI
Forecast Exports Imports Exports Imports
Short-term 0.4770.1930.247 0.2870.3600.321 0.4690.4780.256 0.4210.5440.628
Long-term 0.0370.0340.016 0.0200.0270.013 0.6880.7470.674 0.3650.1980.220
ifo
Forecast Exports Imports Exports Imports
Short-term 0.6550.8750.864 0.9850.9830.994 0.8280.9290.979 0.1740.3520.327
Long-term 0.0200.0160.012 0.0140.0190.014 0.3570.2660.372 0.8930.5320.645
IfW
Forecast Exports Imports Exports Imports
Short-term 0.0210.0100.026 0.4570.5340.552 0.2680.0440.072 0.2800.2050.203
Long-term 0.0820.0860.056 0.1960.0780.092 0.4630.5870.569 0.1980.2770.145
GD
Forecast Exports Imports Exports Imports
Short-term 0.6080.6560.663 0.8600.8870.882 0.9380.7200.797 0.2440.1720.124
Long-term 0.2790.2820.356 0.0860.1550.133 0.4170.5960.530 0.5440.7030.651
OECD
Forecast Exports Imports Exports Imports
Short-term 0.5090.3200.492 0.0450.1030.086 0.0650.0600.142 0.2900.4420.247
Long-term 0.0910.1580.145 0.0020.0010.001 0.0560.0190.038 0.3570.2040.129
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 11. Excluding the years of the financial crisis.
Table 11. Excluding the years of the financial crisis.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Short-term 0.8810.8910.916 0.1360.2450.165 0.7010.7810.908 0.4330.4020.411
Long-term 0.3260.1300.184 0.0220.0380.054 0.1510.3080.265 0.1260.2590.185
HWWI
Forecast Exports Imports Exports Imports
Short-term 0.2830.4430.539 0.3650.4080.349 0.2690.2890.278 0.3420.4960.450
Long-term 0.0190.0240.020 0.0190.0110.025 0.8060.4900.670 0.1820.1190.141
ifo
Forecast Exports Imports Exports Imports
Short-term 0.8800.7550.835 0.9520.9350.963 0.7900.8840.855 0.6350.4430.087
Long-term 0.0360.0190.024 0.0020.0090.004 0.1960.2860.175 0.3600.1950.281
IfW
Forecast Exports Imports Exports Imports
Short-term 0.0100.0480.040 0.3670.4590.495 0.0340.0290.035 0.5570.2150.301
Long-term 0.2460.2600.080 0.2350.1850.163 0.6100.3740.493 0.0660.2180.052
GD
Forecast Exports Imports Exports Imports
Short-term 0.5020.4630.391 0.5710.5170.820 0.6350.9320.950 0.1960.1600.158
Long-term 0.2830.3910.409 0.0950.0600.070 0.7820.4290.531 0.2000.3540.628
OECD
Forecast Exports Imports Exports Imports
Short-term 0.3390.2810.348 0.2290.2860.145 0.0710.0770.127 0.3560.3620.262
Long-term 0.0940.0620.053 0.0010.0020.001 0.1700.0810.044 0.0420.0890.075
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 12. Excluding the years of German reunification.
Table 12. Excluding the years of German reunification.
Trees 250500750 250500750 250500750 250500750
Loss Function Quadratic Flexible
DIW
Forecast Exports Imports Exports Imports
Short-term 0.9830.9530.956 0.4860.2220.375 0.8360.4880.375 0.1360.3340.137
Long-term 0.2550.1370.070 0.0790.0420.032 0.2270.2430.208 0.0990.1160.205
HWWI
Forecast Exports Imports Exports Imports
Short-term 0.3030.2420.298 0.6090.3820.386 0.0740.3340.475 0.5500.5460.487
Long-term 0.0100.0340.056 0.0120.0080.008 0.9020.4550.848 0.2390.1640.142
ifo
Forecast Exports Imports Exports Imports
Short-term 0.8690.7380.822 0.9700.8850.824 0.7400.8210.542 0.4450.2480.371
Long-term 0.4770.5470.383 0.2550.3800.399 0.2510.2370.167 0.6530.8580.751
IfW
Forecast Exports Imports Exports Imports
Short-term 0.0050.0100.003 0.1040.1370.145 0.1160.1280.178 0.4230.4210.324
Long-term 0.7810.5300.610 0.3670.2340.303 0.7420.6160.589 0.0500.0240.057
GD
Forecast Exports Imports Exports Imports
Short-term 0.2740.3350.312 0.2410.0790.131 0.3920.6070.391 0.1410.0760.129
Long-term 0.4300.5160.600 0.1480.1080.158 0.4950.3850.297 0.4930.2150.293
OECD
Forecast Exports Imports Exports Imports
Short-term 0.5340.5450.656 0.0500.1560.104 0.2080.2740.266 0.3790.2910.383
Long-term 0.4350.5250.488 0.0130.0060.001 0.1070.0330.030 0.2150.1380.080
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.
Table 13. Data pooled across forecasters.
Table 13. Data pooled across forecasters.
Specification - l FE t + 1 0 l FE t + 1 < 0 l FE t + 1 0 l FE t + 1 < 0
Loss Function Quadratic Flexible
Forecast Exports Imports Exports Imports
Short-term 0.001 0.001 <0.001<0.001 <0.001<0.001
Long-term 0.001 0.001 <0.001<0.001 <0.001<0.001
Notes: Reported p-values are obtained by a permutation test with 1000 replications. Bold numbers indicate significance at the 5%-level.

Share and Cite

MDPI and ACS Style

Behrens, C. A Nonparametric Evaluation of the Optimality of German Export and Import Growth Forecasts under Flexible Loss. Economies 2019, 7, 93. https://doi.org/10.3390/economies7030093

AMA Style

Behrens C. A Nonparametric Evaluation of the Optimality of German Export and Import Growth Forecasts under Flexible Loss. Economies. 2019; 7(3):93. https://doi.org/10.3390/economies7030093

Chicago/Turabian Style

Behrens, Christoph. 2019. "A Nonparametric Evaluation of the Optimality of German Export and Import Growth Forecasts under Flexible Loss" Economies 7, no. 3: 93. https://doi.org/10.3390/economies7030093

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop