In this section, we analyze the BACE and BMA results for data used in two well-known dynamic macroeconometric models: model for M1 money demand in UK (UKM1), which was proposed in

Hendry and Ericsson (

1991), and the long-term UK inflation model, as introduced in

Hendry (

2001). We focus on the modeling and forecasting of narrow money and inflation in the UK using the BACE and BMA methods along with the Autometrics program

6. We analyze the estimation results using standard posterior characteristics, such as posterior inclusion probabilities, the posterior means of regression parameters, and the posterior standard deviations, as defined in

Section 2. We compare forecasting accuracies using three measures, namely, root-mean-square error (RMSE), mean average percentage error (MAPE), and Theil’s U coefficient (see

Theil 1966, pp. 33–36) divided into three factors: bias proportion U

^{M} (measures differences between averages of actual and predicted values), regression proportion U

^{R} (evaluates the slope coefficient from a regression of changes in actual values on changes in predicted values), and disturbance proportion U

^{D} (measures proportion of forecast error associated with random disturbance). Two first factors stand for systematic error and should be 0, while disturbance proportion is an unsystematic element and should equal 1.

#### 5.1. Modeling and Forecasting Demand for Narrow Money in the UK: UKM1

We based the first empirical illustration on the UKM1 model proposed in

Hendry and Ericsson (

1991) in the following form:

where small letters indicate the log-transformed variables defined as follows

7:

${M}_{t}$: nominal narrow money, M1 aggregate in million £,

${Y}_{t}$: real total final expenditure (TFE) for 1985 prices in million £,

${P}_{t}$: deflator of TFE,

$R{n}_{t}$: net interest rate of the cost of holding money (calculated as the difference between the three-month interest rate and learning-adjusted own interest rate).

The data for the UK narrow money M1 aggregate are quarterly and span from 1964:3 to 1989:2

8.

Figure 1 presents plots of the time series used in the analysis.

Model (

27) was later replicated as an unrestricted autoregressive distributed lag (ADL) model in PcGets (see

Krolzig and Hendry 2001, p. 29). In their paper, narrow money was measured in nominal terms instead of real terms, so the ADL representation of the general unrestricted model (GUM) was defined as follows:

After reduction

9, they obtained the following empirical model:

In our research, following

Krolzig and Hendry (

2001), we estimated the GUM in the form shown in Equation (

28) using the sample 1964:1–1985:2

$(T=86)$ and used the last 4 years (1985:3–1989:2) for forecasting purposes. We compared the variable selection and forecasting accuracy of BACE and BMA with those of Autometrics, which is an alternative automatic model selection procedure.

Table 1 presents the estimation and variable selection results for UKM1 in the ADL form, Equation (

28). Model space consists of

${2}^{20}$ = 1,048,576 specifications that must be considered. The total number of variables is 20, including current values of an explanatory variables and their lags (up to order 4), lagged values of the dependent variable (up to order 4) and constant.

According to the results in

Table 1, the variables used in the BACE analysis can be divided into three groups: high-probability determinants

$({m}_{t-1},R{n}_{t},{p}_{t})$ with

$PIP\ge 2/3$, medium-probability determinants

$({m}_{t-2},{m}_{t-4},{p}_{t-1},{y}_{t-1})$ with

$1/3\le PIP<2/3$, and low-probability determinants (the remaining variables) with

$PIP<1/3$. The top four most probable variables are the same as those selected by Autometrics, although only three of them are classified as highly probable determinants (one variable, i.e.,

${y}_{t-1}$ is close to being highly probable). This discrepancy between the two selections can be explained by the fact that the Autometrics model, which is the most probable one in BACE, has only 1.53% of the total posterior probability mass (see

Table 2).

In case of BMA with stationarity restrictions, we get similar results although there are some slight changes. Again the top four most probable variables are the same as those selected by BACE and Autometrics, however the only two them

$({m}_{t-1},R{n}_{t})$ can be classified as highly probable, while in the group of medium-probability determinants we can include:

${p}_{t},{p}_{t-1},{y}_{t-1}$. The most likely model is again the model selected by Autometrics, although in this case the posterior probability for the top model is higher than in BACE and equals to 8.09% (see

Table 3). The BMA procedure without stationarity restrictions points the

${m}_{t-1}$ and

$R{n}_{t}$ as highly probable variables, while

${p}_{t}$ and

${y}_{t-1}$ are medium probable. The most probable model is still the same as indicated in BMA with stationarity restrictions, but models

$\left({M}_{2}\right)$ and

$\left({M}_{3}\right)$ have changed the order in the list (see

Table 4). In the vast majority of cases, BMA PIP coefficients take lower values than in the case of BACE. As a consequence, we find little difference in posterior mean and variance of regression parameters comparing BACE to BMA. Nevertheless, it is difficult to state clearly which point estimates are closer to the results obtained by Autometrics, however, the BACE ranking is more consistent with the Autometrics output. It seems that the both BMA methods prefer a more parsimonious specifications that do not include some variables found to be important in the BACE and Autometrics.

In the next step, we decided to compare the accuracy of forecasts.

Table 5 presents the BACE, B A (with and without stationarity restriction) and Autometrics forecasts of nominal narrow money in the UK for the period from 1985:3 to 1989:2, which covers 16 quarters. The second column includes the logs of the actual values of the dependent variable. The next columns contain the weighted averages of individual model forecasts and errors for BACE, BMA, and Autometrics, respectively. Additionally, we include results for so-called median probability models introduced by

Barbieri and Berger (

2004). Moreover, the five bottom rows of the table contain well-known measures of forecast error: RMSE, MAPE, and Theil’s U coefficients.

Two accuracy measures indicate that the BACE forecasts are relatively close to the real values of nominal narrow money in the UK. For BACE, RMSE is 0.0224 and MAPE is 0.17%, while RMSE and MAPE for BMA with stationarity restrictions are 0.0592 and 0.43%, respectively. Forecast generated by BMA without stationarity restrictions have slightly lower forecast errors then forecasts from BMA with stationarity restrictions. In the other cases, i.e., for Autometrics and median probability models the results are in the range of 0.0994–0.1018 and 0.72–0.74%, respectively, while both median BMA approaches give exact equal results. It means that the RMSE calculated for BACE is two and a half times smaller than the RMSE resulting from BMA and almost five times smaller than for Autometrics. We can see almost the same scenario for MAPE measure where BACE measure returns the smallest errors. As the last conclusion about these results is that the median probability models do not outperform the mixture of models in terms of predictive performance.

For all methods, the largest factor of forecast error is bias proportion, which has a considerable impact on forecast accuracy, although for BACE is the smallest one. This is clearly reflected in

Figure 2, which shows the actual and forecasted values of nominal narrow money. In this figure, the bias in the forecasts generated by BMA, Autometrics and two other methods substantially grows as the forecast horizon increases.

One potential explanation of this observation is that the forecasts in Autometrics and median probability models are generated by only one model. According to the BACE and BMA with stationarity restrictions results, the use of a single model $\left({M}_{1}\right)$ leaves 98.5% and 91.9% of the total posterior probability mass. On the other hand, BACE and BMA calculates forecasts from the whole model space and accounts for the mixture of all considered specifications, which are weighted by their posterior probabilities.

#### 5.2. Modeling and Forecasting Long-Term UK Inflation

In the second empirical example, we used the long-term UK inflation model developed in

Hendry (

2001) for 1875–1991 years

$(T=117)$. The data set

10 used in this research is described in

Table 6, and

Figure 3 presents the time-series plots (small letters indicate log-transformed variables).

The final specification after a number of variable transformations and model pre-reduction was as follows

11) (see

Hendry 2001):

where

${\pi}_{t}^{*}=0.25{e}_{r,t}-0.675{(c-p)}_{t}^{*}-0.075{({p}_{o}-p)}_{t}+0.11{I}_{2,t}+0.25$,

${(c-p)}_{t}^{*}={c}_{t}-{p}_{t}+0.006\times (trend-69.5)+2.37$, and

${I}_{d}$ is a combination of year indicator dummies. Model space consists of

${2}^{20}$ = 1,048,576 linear combinations that must be considered. After the reduction at a 1% significance level, specification in Equation (

30) was reduced to the following empirical model (see

Hendry 2001):

The results in

Table 7 show that the BACE, BMA, and Autometrics identify the same set of significant determinants of UK inflation as in

Hendry (

2001). Moreover, both BMA procedures, with and without stationarity restrictions, give exactly the same results. This can be explained by the fact that dependent variable

$\Delta {p}_{t}$ is far away from non-stationary region, so imposing stationarity restrictions does not result in rejecting any of draws from posterior. Hereafter, we formulate comments without division into restricted or unrestricted case. BACE and BMA indicate that the following variables are highly probable:

${\pi}_{t-1}^{*},{I}_{d,t},\Delta {p}_{e,t},{S}_{t-1},\Delta {p}_{t-1},{y}_{t-1}^{d},\Delta {R}_{s,t-1},\Delta {m}_{t-1},\Delta {p}_{o,t-1}$. Autometrics selects the same set, reducing model (

30) at the 1% significance level.

Table 8,

Table 9 and

Table 10 present the BACE and BMA posterior probability and coefficient estimates for the top 10 models. In the case of BACE the most probable model

$\left({M}_{1}\right)$ has a posterior probability of 21.9%, while the second model in the ranking

$\left({M}_{2}\right)$ has a probability of 6.4%. For the other models, the posterior probability does not exceed 4.7%. Although the posterior probability of the highest-ranked model

$\left({M}_{1}\right)$ is more than three times larger than that of the second model

$\left({M}_{2}\right)$, an inference that is based only on

${M}_{1}$ leaves 78.1% of the posterior probability mass. As a consequence, estimates of the average mean of coefficients are slightly different from those in Autometrics. We can meet a similar situation in the case of BMA, although the highest-ranked model

$\left({M}_{1}\right)$ is even more preferred by the data with posterior probability equals to 32.66%. The second model in the ranking

$\left({M}_{2}\right)$ is almost two times less likely.

Table 11 presents detailed information about the predictions for UK inflation resulting from BACE, BMA, Autometrics, and median probability models. This table includes actual values, forecast values, and forecast standard errors, as well as accuracy measures, for the period from 1982 to 1991, which covers 10 years. The actual and forecast values of UK inflation are presented in

Figure 4. For BACE, RMSE is 0.0179 and MAPE is 26.06%, for BMA we have RMSE—0.0175 and MAPE—25.85%, while RMSE and MAPE in Autometrics are 0.0151 and 25.06%, respectively. Forecast errors of the median probability models are the largest compared to other methods. As we can see BACE, BMA, and Autometrics generate forecasts of almost the same quality, but the sources of errors are different. For BACE and BMA, the greatest factor of forecast error is bias proportion, while the greatest factor for Autometrics is disturbance proportion.

One explanation of the equivalent forecasting performances is the fact that, according to the results in

Table 8,

Table 9 and

Table 10, the top 10 most probable specifications have almost the same set of nine the most probable variables and cover over 50% of the posterior probability mass, including the second-ranked model (

${M}_{2}$) selected by Autometrics.