econometrics BACE and BMA Variable Selection and Forecasting for UK Money Demand and Inﬂation with Gretl

: In this paper, we apply Bayesian averaging of classical estimates (BACE) and Bayesian model averaging (BMA) as an automatic modeling procedures for two well-known macroeconometric models: UK demand for narrow money and long-term inflation. Empirical results verify the correctness of BACE and BMA selection and exhibit similar or better forecasting performance compared with a non-pooling approach. As a benchmark, we use Autometrics—an algorithm for automatic model selection. Our study is implemented in the easy-to-use gretl packages, which support parallel processing, automates numerical calculations, and allows for efficient computations.


Introduction
In this paper we consider two procedures of variable selection and forecasting for linear dynamic single-equation models: Bayesian averaging of classical estimates (BACE), introduced by Sala-i- Martin et al. (2004) and Bayesian model averaging (BMA) (see Raftery et al. 1997). The BACE and BMA methods are natural extension of the standard Bayesian inference methods in which one does not only make inference using single model, but also allowing pooling approach with combined estimation and prediction. As a benchmark we use Autometrics procedure which is non-pooling method based on general-to-specific approach with multiple path of searching algorithm implemented in OxMetrics software (see Doornik 2009;Doornik and Hendry 2013). We present empirical results for two non-trivial UK macroeconometric models: demand for narrow money (see Krolzig and Hendry 2001), and long-term inflation (see Hendry 2001).
In the last two decades we can observe a growing number of publications related to Bayesian model averaging in many fields of science like engineering, medicine, biology, sociology and others (see Fragoso et al. 2018). Not surprisingly, this type of approach has also been long-used in economics and it is still important, especially for identification of the sources of economic growth (Fernández et al. 2001a(Fernández et al. , 2001bBłażejowski et al. 2019). The ideas stated in this paper were reviewed, among others, by Hoeting et al. (1999) and Wasserman (2000). A comprehensive study on the use of model averaging in economics, both from the frequentist and Bayesian approaches, was discussed in Steel (2019).
The BACE approach is an approximation of BMA and it is not purely Bayesian but relies on Schwarz approximation to compute the Bayes factor (see Ley and Steel 2009). Nevertheless, both the BACE and BMA approaches account for model uncertainty, which often requires the consideration of many possible linear combinations of variables and lead to a large model space that needs to be explored and then demands intensive computational effort. As an approximation, BACE is usually faster than the standard BMA approach; nevertheless, obtaining the output in a reasonable amount of time still remains a significant challenge, especially in time-series modeling and forecasting. Raftery et al. (1997) showed that standard variable selection procedures lead to different estimates and conflicting conclusions about the main questions of interest. Moreover, econometric models that are firmly based on economic theory do not always work for forecasting. The BACE and BMA approaches combine the knowledge obtained from many possible models and accounts for uncertainty by averaging the parameter estimates from different specifications. Consequently, both methods can better identify significant determinants of a dependent variable and generate more accurate forecasts without any specific knowledge.
In BACE and BMA we can penalize large dynamic models using different model prior assumptions putting higher probabilities for more parsimonious models. This type of approach does not cover all possible solutions. One of the potential alternative is assigning lower prior probabilities over lags length, such as Minnesota Prior (see Doan et al. 1984). We also assume stability of the relation between dependent and independent variables over time and, as a consequence, all slope parameters and other posterior characteristics in our BACE and BMA packages are time invariant. In the case of time-varying parameters, it is possible to employ dynamic model averaging presented in Drachal (2018); Raftery et al. (2010).
The remainder of this paper is structured as follows. In Section 2, we discuss some aspects of BACE. Section 3 briefly outlines Bayesian model averaging for dynamic linear regression models along with short information about implementation of both packages in the gretl. In Section 4 we provide basic information about Autometrics-a PcGive module for automatic model selection. Section 5 presents empirical results for two selected UK macroeconometric models: demand for narrow money and long-term inflation. In this section, we also compare the variable selection strategies and forecasting performance of BACE and BMA with those of Autometrics. Additionally, in Section 6 we analyze the robustness and computational run-times of BACE and BMA. Finally, we conclude the paper in Section 7.

The BACE Method
In Sala-i-Martin et al. (2004), the authors proposed averaging parameter estimates using a technique-Bayesian averaging of classical estimates-that enabled the measurement of the importance of particular potential regressors. In this method, parameter estimates are obtained by applying ordinary least squares (OLS) and then averaged across all possible combinations of models. The BACE approach is not purely Bayesian but relies on Schwarz approximation to compute the Bayes factor. This approach is an alternative to the familiar and earlier-applied BMA technique, from which it differs i.e., by using non-informative prior assumptions of regression parameters 1 . A full discussion that compares BACE and BMA is presented in Ley and Steel (2009).
Among the many articles that have applied the BACE technique are van Dijk (2004) for US inflation and Białowolski et al. (2014) for gross domestic product, inflation, and unemployment in Poland. Jones and Schneider (2006) used BACE analysis to verify the human capital effect on economic growth, while Mapa and Briones (2007) and Simo-Kengne (2016) used BACE analysis to obtain variables associated with economic growth. Cuaresma and Doppelhofer (2007) extended the BACE approach by allowing for the uncertainty of nonlinear threshold effects to identify determinants of long-term economic growth. Bergh and Karlsson (2010) applied BACE to investigate the relation between government size and the control of economic freedom and globalization for a panel of rich countries. In an empirical investigation of industrial production forecasting, Feldkircher (2012) focused 1 BACE is implicitly based on fixed Zellner's g-prior, whereas, in the BMA framework, g-prior can be set explicitly. on the forecasting performance resulting from model averaging by measuring the root-mean-square error (RMSE). Albis and Mapa (2014) used BACE to verify misspecification issues in vector autoregressive models for artificial data.
Let us consider the following dynamic linear regression model M j (j = 1, 2, . . . , K): is an (Tx1) vector of errors that are assumed to be normally distributed (i.e., ∼ N(0 T , σ 2 I T )) and N(µ, Σ) denotes a normal distribution with location µ and covariance Σ.
From OLS estimates (see Sala-i-Martin et al. 2004), we can calculate the approximation of the posterior probability of model M j (i.e., Pr(M j | y)) using the following formula: where SSE j and SSE i are the OLS sum of squared errors, 2 K denotes the total number of potential combinations of K independent variables, and k j and k i are the number of regression parameters β j and β i .
Prior probabilities Pr(M j ) and Pr(M i ) of models M j and M i are binomially distributed; that is, The binomial distribution implies that we only need to specify a prior expected model size E(Ξ) = Kθ, where E(Ξ) ∈ (0, K]. For example, if we define the value of E(Ξ), then our BACE package will automatically produce a value of prior inclusion probability for all competitive models. If θ = 0.5, then the prior expected model size is equal to the average of the number of potential regressors, and the model prior distribution is uniform Pr(M i ) = 2 −K and reflects a lack of previous knowledge about the models.
Using BACE, we can also easily evaluate the mean and variance of the posterior distribution of regression parameters β for the whole model space (see Leamer 1978;Sala-i-Martin et al. 2004): Another useful and popular characteristic of the BACE approach is posterior inclusion probability (PIP), which is defined as the posterior probability that the variable x i is relevant in the explanation of the dependent variable (see Leamer 1978;Mitchell and Beauchamp 1988). In our case, the PIP is calculated as the sum of the posterior model probabilities for all of the models that include a specific variable: For model averaging, a Bayesian pooling strategy can also provide useful information about future observations of the dependent variable on the basis of the whole model space: Var where E(y f | y) and Var(y f | y) denote the mean and variance of future observations y f .

The BMA Method
Another model building strategy is Bayesian model averaging wherein we can make an inference based on full posterior distribution. From Bayesian perspective uncertainty is a natural way of decision making process and therefore it can be easily included in the model selection rules (Koop 2003;Zellner 1971). Among the many seminal papers about Bayesian model averaging are Hoeting et al. (1999) and Fernández et al. (2001aFernández et al. ( ,2001b. The most recent detailed overview is presented in Steel (2019).
Once again, we are dealing with a problem which model and variables are the most appropriate in the analysis of the dependencies, but in this case we use a natural and explicit way of combining prior information with data, without any approximation of marginal data density and Bayes factors. We consider two variants of BMA framework. The first one where we impose stationary conditions for autoregressive parameters, and the second one without restrictions. Now let us consider the first one: where y is a vector of T observations, X j is (T × k j ) matrix, and β j is a (k j × 1) vector of parameters, ε is a vector of dimensions (T × 1) with a normal distribution N(0, σ 2 I T ), where σ 2 is a variance of random error ε and I T is an identity matrix of size T. Moreover , and Γ is stationary region for the parameters of autoregressive processes. We also assume that we observe initial values y (0) . Let us consider a prior density of the following form: where: and f N (β j | µ, Σ) denotes the multivariate normal density with mean µ and covariance matrix Σ, I(A) is the indicator function, β j is k j -vector of prior means for regression coefficients and V j is a k j × k j positive definite prior covariance matrix of the form: For the error precision h which is defined as h = 1/σ 2 we use noninformative prior: The factor of proportionality g j (j = 1, 2, . . . , K) is part of the so-called g-prior, as introduced in Zellner (1986). In our research we use Benchmark prior, recommended by Fernández et al. (2001a): Assuming the prior structure in Equation (10) we obtain the following joint posterior density: where c j is normalizing constant and f NG is normal-gamma density (see Koop et al. 2007). In our case constant c j plays important role to obtain the Bayes factor between competitive models and can be obtained by Monte Carlo simulations.
Using the properties of normal-gamma density, Equation (15) leads to: where and v j = T. We also have: where v j = T − k j . The marginal data density p(y | M j ) as well as posterior means and standard deviations of regression coefficients can be calculated numerically using Monte Carlo integration by sampling from the posterior distribution in Equation (15). In our case, we first draw error precision h from Equation (17) and then we draw β j from Equation (16). We only accept those candidate values which lie in stationary region for the parameters of autoregressive processes. The constant c j can be calculated as an inverse of the acceptance ratio i.e., inverse of the fraction of random numbers accepted in Monte Carlo simulation.
The posterior probability of any variant of regression model M j can be calculated by the following formula, which is crucial for Bayesian model averaging: where Pr(M 1 ), Pr(M 2 ), . . . , Pr(M K ) denote the prior probabilities of competitive models (see Equation (3)).
Other characteristics, like posterior model probabilities (PIP) as well as the mean and variance of the posterior distribution of regression parameters β for the whole model space can be calculated in the same manner as described for BACE method.
In case of BMA variant without stationary restrictions for the parameters of autoregressive processes, we assume that β y j ∈ R k y j and Equations (11), (15) and (16) takes the following form, while the other equations remain unchanged:

BACE and BMA in Gretl
In order to perform Bayesian averaging of classical estimates, we used the BACE 2.0 package 2 . This code implements an automatic BACE procedure that is available in the gretl 3 program as an open-source software. In the procedure's main window, we can specify e.g., the following parameters: the list of independent variables, the prior distribution over the model space, the number of out-of-sample forecasts, and general parameters for the Monte Carlo simulation. As a result, the BACE package prints basic posterior characteristics, such as PIP, and the posterior means of coefficients, together with their standard errors. In addition, the package presents rankings of the most probable specifications according to their explanatory power and generates forecasts of the dependent variable.
In this paper we also use a software package that implements Bayesian model averaging for Autoregressive Distributed Lag models BMA_ADL ver. 0.9 in gretl 4 . The BMA_ADL package as well as the output is similar to BACE package. Although these two packages are similar there is a principle difference between them. In BMA_ADL we draw samples from posterior distribution of slope parameters β and we check roots of characteristic polynomial of the autoregressive process, although this feature can be explicitly switched off by user. Detailed information about the package can be found in .
Since exploration of many possible models demands intensive computational effort, we run BACE and BMA_ADL through the Message Passing Interface (MPI) 5 . It is especially useful for a large number of explanatory variables, which results in a computational complexity that exceeds the computing power of modern PCs, because it performs parallel computations through MPI.
Both packages are written in gretl's internal scripting language Hansl (see Cottrell and Lucchetti 2019b) with an easy-to-use graphical user interface (GUI). Therefore, they can be treated as an automatic model selection procedure and would be a useful for users who are not familiar with model averaging.

Autometrics
Autometrics procedure of variables model selection is built on the basis of PcGets module implemented in OxMetrics software and is is fully described in Doornik (2009) and Doornik and Hendry (2013). This conception is based on general-to-specific approach with multiple path of searching (reducing) algorithm. The assumption underlying empirical model selection is discovering the local data generating process (LDGP), which is tend to explain the relations which occur in real world in the space of available variables. One needs to select the model from the set of 2 The BACE 2.0 package is available at http://ricardo.ecn.wfu.edu/gretl/cgi-bin/gretldata.cgi?opt=SHOW_FUNCS and was developed by co-authors (see Błażejowski and Kwiatkowski 2018). 3 Gretl is an open-source software for econometric analysis and is available at http://gretl.sf.net. 4 The BMA_ADL package for gretl is available in Supplementary Materials along with scripts to replicate all analysis. 5 MPI is a standard that supports running a given program simultaneously on several CPU cores, so it supports a very flexible type of parallelism of Monte Carlo integration see Lucchetti 2020, 2019a). potential specifications ensuring that model is congruent and encompasses LDGP. The crucial issue here is test-based reduction strategy (see Desboulets 2018).
The procedure of selecting variables in Autometrics is proceed in following stages. The starting point for automatic procedure is general unrestricted model (GUM), which should remain congruent and include all potentially relevant variables according to sample size, theoretical, or empirical considerations. This solution increases chance that LDGP is nested in GUM and can be discovered. The variables, which have strong importance and influence should remain in model and the less significant ones should not be retained. The mis-specification test is applied to ensure the congruence postulate and to avoid badly-specified GUM. Additionally, encompassing test is used to evaluate if small model can explain larger one, which is encompassed within. Such a procedure can be treated as progressive research strategy of reduction and defines partial order of considered model specifications (see Hendry et al. 2008).
Autometrics ensures the tree-search approach for examining the whole model space using three strategies of path evaluation: pruning, bunching and chopping (see Hendry and Doornik 2014). Before the main multi-path reduction of variables is launched, the pre-search is initiated to eliminate most insignificant and irrelevant variables. During reductions and simplifications diagnostic tests are employed to ensure the congruence and to find valid specification. The terminal model is received at the end of each branch of the tree-search algorithm. Moreover, Autometrics also take into consideration following modeling issues: cointegration, functional form, economic theory or data accuracy.
Application of Autometrics or PcGets can be found in works, for example: Clements and Hendry (2008); Hendry (2001) for UK inflation, Hendry (2011) for consumers' expenditures, Castle et al. (2012) for US real interest rates, Ericsson and Kamin (2009) for Argentine broad money demand, Marczak and Proietti (2016) for industrial production, Kamarudin and Ismail (2016) for water quality index, or Ackah and Asomani (2015) for renewable energy.

Empirical Results
In this section, we analyze the BACE and BMA results for data used in two well-known dynamic macroeconometric models: model for M1 money demand in UK (UKM1), which was proposed in Hendry and Ericsson (1991), and the long-term UK inflation model, as introduced in Hendry (2001). We focus on the modeling and forecasting of narrow money and inflation in the UK using the BACE and BMA methods along with the Autometrics program 6 . We analyze the estimation results using standard posterior characteristics, such as posterior inclusion probabilities, the posterior means of regression parameters, and the posterior standard deviations, as defined in Section 2. We compare forecasting accuracies using three measures, namely, root-mean-square error (RMSE), mean average percentage error (MAPE), and Theil's U coefficient (see Theil 1966, pp. 33-36) divided into three factors: bias proportion U M (measures differences between averages of actual and predicted values), regression proportion U R (evaluates the slope coefficient from a regression of changes in actual values on changes in predicted values), and disturbance proportion U D (measures proportion of forecast error associated with random disturbance). Two first factors stand for systematic error and should be 0, while disturbance proportion is an unsystematic element and should equal 1.
Based on the methods discussed here, we also consider two additional models associated with BACE and BMA output. Barbieri and Berger (2004) introduced median probability model, which is defined as the model consisting of those variables which have an overall posterior probability greater than or equal to 0.5 of being in a model. According to the authors, median probability model considerably outperforms the most probable model in terms of predictive accuracy. Therefore it seems reasonable to include this model in our analysis and compare its forecasting performance. 6 We used gretl version 2019d-git and PcGive version 14.2 with Ox Professional version 7.20 on a PC machine running under Debian GNU/Linux 64 bits.

Modeling and Forecasting Demand for Narrow Money in the UK: UKM1
We based the first empirical illustration on the UKM1 model proposed in Hendry and Ericsson (1991) in the following form: where small letters indicate the log-transformed variables defined as follows 7 : • M t : nominal narrow money, M1 aggregate in million £, • Y t : real total final expenditure (TFE) for 1985 prices in million £, • P t : deflator of TFE, • Rn t : net interest rate of the cost of holding money (calculated as the difference between the three-month interest rate and learning-adjusted own interest rate).
The data for the UK narrow money M1 aggregate are quarterly and span from 1964:3 to 1989:2 8 . Figure 1 presents plots of the time series used in the analysis.  Exogeneity of variables used in UKM1 model is discussed in (Hendry and Nielsen 2012, pp. 266-67;Hendry 1995, pp. 605-6;Hendry 2015, pp. 127-33) and the results show that modeling demand for narrow money in UK as a single equation is valid in general.
Model (27) was later replicated as an unrestricted autoregressive distributed lag (ADL) model in PcGets (see Krolzig and Hendry 2001, p. 29). In their paper, narrow money was measured in nominal terms instead of real terms, so the ADL representation of the general unrestricted model (GUM) was defined as follows: After reduction 9 , they obtained the following empirical model: In our research, following Krolzig and Hendry (2001), we estimated the GUM in the form shown in Equation (28) using the sample 1964:1-1985:2 (T = 86) and used the last 4 years (1985:3-1989:2) for forecasting purposes. We compared the variable selection and forecasting accuracy of BACE and BMA with those of Autometrics, which is an alternative automatic model selection procedure. Table 1 presents the estimation and variable selection results for UKM1 in the ADL form, Equation (28). Model space consists of 2 20 = 1,048,576 specifications that must be considered. The total number of variables is 20, including current values of an explanatory variables and their lags (up to order 4), lagged values of the dependent variable (up to order 4) and constant.
According to the results in Table 1, the variables used in the BACE analysis can be divided into three groups: high-probability determinants (m t−1 , Rn t , p t ) with PIP ≥ 2/3, medium-probability determinants (m t−2 , m t−4 , p t−1 , y t−1 ) with 1/3 ≤ PIP < 2/3, and low-probability determinants (the remaining variables) with PIP < 1/3. The top four most probable variables are the same as those selected by Autometrics, although only three of them are classified as highly probable determinants (one variable, i.e., y t−1 is close to being highly probable). This discrepancy between the two selections can be explained by the fact that the Autometrics model, which is the most probable one in BACE, has only 1.53% of the total posterior probability mass (see Table 2).
In case of BMA with stationarity restrictions, we get similar results although there are some slight changes. Again the top four most probable variables are the same as those selected by BACE and Autometrics, however the only two them (m t−1 , Rn t ) can be classified as highly probable, while in the group of medium-probability determinants we can include: p t , p t−1 , y t−1 . The most likely model is again the model selected by Autometrics, although in this case the posterior probability for the top model is higher than in BACE and equals to 8.09% (see Table 3). The BMA procedure without stationarity restrictions points the m t−1 and Rn t as highly probable variables, while p t and y t−1 are medium probable. The most probable model is still the same as indicated in BMA with stationarity restrictions, but models (M 2 ) and (M 3 ) have changed the order in the list (see Table 4). In the vast majority of cases, BMA PIP coefficients take lower values than in the case of BACE. As a consequence, we find little difference in posterior mean and variance of regression parameters comparing BACE to BMA. Nevertheless, it is difficult to state clearly which point estimates are closer to the results obtained by Autometrics, however, the BACE ranking is more consistent with the Autometrics output. It seems that the both BMA methods prefer a more parsimonious specifications that do not include some variables found to be important in the BACE and Autometrics. 9 Authors understand 'reduction' as a structured path of elimination insignificant variables based on t-statistics together with pre-search analysis and encompassing tests.
In the next step, we decided to compare the accuracy of forecasts. Table 5 presents the BACE, B A (with and without stationarity restriction) and Autometrics forecasts of nominal narrow money in the UK for the period from 1985:3 to 1989:2, which covers 16 quarters. The second column includes the logs of the actual values of the dependent variable. The next columns contain the weighted averages of individual model forecasts and errors for BACE, BMA, and Autometrics, respectively. Additionally, we include results for so-called median probability models introduced by Barbieri and Berger (2004). Moreover, the five bottom rows of the table contain well-known measures of forecast error: RMSE, MAPE, and Theil's U coefficients.
Two accuracy measures indicate that the BACE forecasts are relatively close to the real values of nominal narrow money in the UK. For BACE, RMSE is 0.0224 and MAPE is 0.17%, while RMSE and MAPE for BMA with stationarity restrictions are 0.0592 and 0.43%, respectively. Forecast generated by BMA without stationarity restrictions have slightly lower forecast errors then forecasts from BMA with stationarity restrictions. In the other cases, i.e., for Autometrics and median probability models the results are in the range of 0.0994-0.1018 and 0.72-0.74%, respectively, while both median BMA approaches give exact equal results. It means that the RMSE calculated for BACE is two and a half times smaller than the RMSE resulting from BMA and almost five times smaller than for Autometrics. We can see almost the same scenario for MAPE measure where BACE measure returns the smallest errors.
As the last conclusion about these results is that the median probability models do not outperform the mixture of models in terms of predictive performance.
For all methods, the largest factor of forecast error is bias proportion, which has a considerable impact on forecast accuracy, although for BACE is the smallest one. This is clearly reflected in Figure 2, which shows the actual and forecasted values of nominal narrow money. In this figure, the bias in the forecasts generated by BMA, Autometrics and two other methods substantially grows as the forecast horizon increases.
One potential explanation of this observation is that the forecasts in Autometrics and median probability models are generated by only one model. According to the BACE and BMA with stationarity restrictions results, the use of a single model (M 1 ) leaves 98.5% and 91.9% of the total posterior probability mass. On the other hand, BACE and BMA calculates forecasts from the whole model space and accounts for the mixture of all considered specifications, which are weighted by their posterior probabilities. m forecasts

Modeling and Forecasting Long-Term UK Inflation
In the second empirical example, we used the long-term UK inflation model developed in Hendry (2001) for 1875-1991 years (T = 117). The data set 10 used in this research is described in Table 6, and Figure 3 presents the time-series plots (small letters indicate log-transformed variables). 1880 1900 1920 1940 1960 1980   The final specification after a number of variable transformations and model pre-reduction was as follows 11 (see Hendry 2001): where .006 × (trend − 69.5) + 2.37, and I d is a combination of year indicator dummies. Model space consists of 2 20 = 1,048,576 linear combinations that must be considered. After the reduction at a 1% significance level, specification in Equation (30) was reduced to the following empirical model (see Hendry 2001): The results in Table 7 show that the BACE, BMA, and Autometrics identify the same set of significant determinants of UK inflation as in Hendry (2001). Moreover, both BMA procedures, with and without stationarity restrictions, give exactly the same results. This can be explained by the fact that dependent variable ∆p t is far away from non-stationary region, so imposing stationarity restrictions does not result in rejecting any of draws from posterior. Hereafter, we formulate comments without division into restricted or unrestricted case. BACE and BMA indicate that the following variables are highly probable: π * t−1 , I d,t , ∆p e,t , S t−1 , ∆p t−1 , y d t−1 , ∆R s,t−1 , ∆m t−1 , ∆p o,t−1 . Autometrics selects the same set, reducing model (30) at the 1% significance level.
Tables 8-10 present the BACE and BMA posterior probability and coefficient estimates for the top 10 models. In the case of BACE the most probable model (M 1 ) has a posterior probability of 21.9%, while the second model in the ranking (M 2 ) has a probability of 6.4%. For the other models, the posterior probability does not exceed 4.7%. Although the posterior probability of the highest-ranked model (M 1 ) is more than three times larger than that of the second model (M 2 ), an inference that is based only on M 1 leaves 78.1% of the posterior probability mass. As a consequence, estimates of the average mean of coefficients are slightly different from those in Autometrics. We can meet a similar situation in the case of BMA, although the highest-ranked model (M 1 ) is even more preferred by the data with posterior probability equals to 32.66%. The second model in the ranking (M 2 ) is almost two times less likely. Table 11 presents detailed information about the predictions for UK inflation resulting from BACE, BMA, Autometrics, and median probability models. This table includes actual values, forecast values, and forecast standard errors, as well as accuracy measures, for the period from 1982 to 1991, which covers 10 years. The actual and forecast values of UK inflation are presented in Figure 4. For BACE, RMSE is 0.0179 and MAPE is 26.06%, for BMA we have RMSE-0.0175 and MAPE-25.85%, while RMSE and MAPE in Autometrics are 0.0151 and 25.06%, respectively. Forecast errors of the median probability models are the largest compared to other methods. As we can see BACE, BMA, and Autometrics generate forecasts of almost the same quality, but the sources of errors are different. For BACE and BMA, the greatest factor of forecast error is bias proportion, while the greatest factor for Autometrics is disturbance proportion.
One explanation of the equivalent forecasting performances is the fact that, according to the results in Tables 8-10, the top 10 most probable specifications have almost the same set of nine the most probable variables and cover over 50% of the posterior probability mass, including the second-ranked model (M 2 ) selected by Autometrics.       Table 9. BMA (with stationarity restrictions) posterior probabilities and coefficient estimates for the top 10 models of UK inflation.

Robustness
In order to confirm the empirical findings for variable and model selection obtained by BACE and BMA, we performed a robustness analysis using different prior model assumptions. We apply philosophy proposed in Osiewalski and Steel (1993) and we set different variants of the prior average model size in order to penalize large models. In Section 5, the prior average model size is set to E(Ξ) = K/2 (where K is the total number of independent variables). This means that we do not prefer any specification, so all possible models are equally probable. We considered robustness scenario as different specifications of prior model size, estimating the models in Equations (28) and (30)  According to the results concerning BACE method, which are presented in Tables 12 and 13, there are no substantial differences in the output between E(Ξ) = K/4, E(Ξ) = K/5, and E(Ξ) = K/8. Similar conclusion can be formulated for BMA outcome included in Tables 14-17. Moreover, comparing the results in Tables 12-17 with those in Tables 1 and 7 reveals that the observed differences are negligible.

BACE and BMA Run Times
Tables 18-20 present computational timings 12 of BACE and BMA analysis conducted in two general variants, with and without forecasting, for two earlier considered empirical examples, i.e., UKM1 and inflation. In the case of BACE analysis, total number of iterations in MC 3 sampling algorithm equals 500,000, while for BMA is 150,000. The difference in the total number of MC 3 iterations in BACE and BMA gretl packages is related with different ways of employing MPI parallel computations, which results in different pace of convergence. CPUs denotes the total number of processors used in simulation experiment, Nrep means the total number of iterations in MC 3 sampling algorithm, and Run time denotes computational time (in seconds).
Increasing the number of CPUs decreases run times of both packages more or less linearly. The highest boost is in the case of BACE for UKM1 with almost equal ratio. Both packages slow down in forecasting, but again with different ratios. In case of BACE, timings of simulations increase approximately by 10-20%, no matter how many CPUs are used. In the case of BMA, run times are multiply by a factor ranging from 12.64 to 26.07 for restricted case and from 32.33 to 825.29 for unrestricted case. Such a big increase of computational timings is related with necessity of generating predictive distributions step-by-step in order to compute dynamic forecasts. Longer run times for BMA are also related with stationarity restrictions for autoregressive parameters. We believe that in the future we will be able to improve speed of BMA computations. 1.5 × 10 5 2470 1.5 × 10 5 56,294 1.5 × 10 5 380 1.5 × 10 5 6829 20 1.5 × 10 5 1169 1.5 × 10 5 14,771 1.5 × 10 5 136 1.5 × 10 5 3044 CPUs denotes the total number of processors used in simulation experiment, Nrep means the total number of iterations in MC 3 sampling algorithm and Run time denotes computational time (in seconds). 1.5 × 10 5 167 1.5 × 10 5 65,862 1.5 × 10 5 54 1.5 × 10 5 6556 20 1.5 × 10 5 103 1.5 × 10 5 16,630 1.5 × 10 5 55 1.5 × 10 5 1778 CPUs denotes the total number of processors used in simulation experiment, Nrep means the total number of iterations in MC 3 sampling algorithm, and Run time denotes computational time (in seconds).

Conclusions
In this paper, we discussed the possibility of using the model averaging methods-BACE and BMA as a tool for selecting variables and forecasting in dynamic econometric modeling. Empirical examples with known, non-trivial macroeconometric models confirmed that the BACE and BMA procedures, by calculating a PIP value for each independent variable, correctly indicates the determinants of the dependent process. Robustness analysis results confirm the stability of variable selection in our examples.
Moreover, the BACE and BMA take into account model uncertainty and generates reasonably close or more accurate forecasts compared with Autometrics or median probability model. The UK inflation forecasts generated by BACE, BMA, and Autometrics have similar RMSE and MAPE values, but the demand for narrow money forecasts generated by BACE and BMA have several times smaller errors than those generated by Autometrics. A similar conclusion also applies to median probability models. Forecasts generated by simple averaging of all predictive models can provide better results than forecasts from single models. This advantage over the non-pooling inference is particularly evident when there is no one dominant model but rather many competing specifications with relatively low explanatory power.