# Bayesian Model Averaging Using Power-Expected-Posterior Priors

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. BMA Point Prediction Estimates

## 3. Computation and Evaluation of BMA Prediction

- For every model ${M}_{\ell}\in \mathcal{M}$:
- (a)
- Obtain the least-squares estimates ${\widehat{\mathit{\beta}}}_{\ell}=({\widehat{\mathit{\beta}}}_{0},{\widehat{\mathit{\beta}}}_{{e}_{\ell}})$ of the regression coefficients using centered covariates.
- (b)
- Calculate the posterior expected value of w from Equation (10).
- (c)
- Calculate ${\widehat{\mathit{y}}}_{{M}_{\ell}}^{new}$ from Equation (8).

- Implement Equation (9) to calculate ${\widehat{\mathit{y}}}_{BMA}^{new}$, as the weighted average of ${\widehat{\mathit{y}}}_{{M}_{\ell}}^{new}$ over all models ${M}_{\ell}\in \mathcal{M}$ using posterior model probabilities as weights.

- Model search using $M{C}^{3}$ algorithm (Madigan and York 1995): this approach can be used since the marginal likelihood is readily available, but it is not very efficient, especially for large model spaces, since both the numerator and the denominator in Equation (13) are greatly affected by the number of visited models, and hence by the number of iterations of the algorithm.
- g–conditional $M{C}^{3}$ algorithm: hyper–parameter g is generated by its marginal posterior distribution; then, we use the conditional on g marginal likelihood to move through the model space; this is the approach used by Ley and Steel (2012). Under this setup, $f\left({y}_{i}^{\mathbb{V}}\right|{\mathit{y}}^{\mathbb{M}},{\mathbf{X}}^{\mathbb{V}})$ is estimated by$$\widehat{f}\left({y}_{i}^{\mathbb{V}}\right|{\mathit{y}}^{\mathbb{M}},{\mathbf{X}}^{\mathbb{V}})=\frac{1}{T}\sum _{t=1}^{T}\frac{f({y}_{i}^{\mathbb{V}},{\mathit{y}}^{\mathbb{M}}|{g}^{\left(t\right)},{M}^{\left(t\right)})}{f\left({\mathit{y}}^{\mathbb{M}}\right|{g}^{\left(t\right)},{M}^{\left(t\right)})},$$
- Fully Bayesian variable–selection MCMC: density $f\left({y}_{i}^{\mathbb{V}}\right|{\mathit{y}}^{\mathbb{M}},{\mathbf{X}}^{\mathbb{V}})$ is estimated by the MCMC average of the sampling–density function of each visited model ${M}^{\left(t\right)}$, evaluated at ${y}_{i}^{\mathbb{V}}$, for each generated set of the model parameters. This is the approach we used in Section 5. More specifically, we implemented the Gibbs variable–selection approach of Dellaportas et al. (2002).

## 4. Simulation Study

## 5. FLS Dataset: Cross-Country Growth GDP Study

## 6. Discussion

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Bayarri, Maria J., James O. Berger, Anabel Forte, and Gonzalo García-Donato. 2012. Criteria for Bayesian model choice with application to variable selection. Annals of Statistics 40: 1550–77. [Google Scholar] [CrossRef] [Green Version]
- Berger, James O., and Luis R. Pericchi. 2001. Objective Bayesian methods for model selection: Introduction and comparison. In Model Selection. Institute of Mathematical Statistics Lecture Notes. Monograph Series 38; Beachwood: IMS, pp. 135–207. [Google Scholar]
- Berger, James O., and Luis R. Pericchi. 2004. Training samples in objective model selection. Annals of Statistics 32: 841–69. [Google Scholar] [CrossRef] [Green Version]
- Consonni, Guido, Dimitris Fouskakis, Brunero Liseo, and Ioannis Ntzoufras. 2018. Prior distributions for objective Bayesian analysis. Bayesian Analysis 13: 627–79. [Google Scholar] [CrossRef]
- Dellaportas, Petros, Jonathan J. Forster, and Ioannis Ntzoufras. 2002. On Bayesian model and variable selection using MCMC. Statistics and Computing 12: 27–36. [Google Scholar] [CrossRef]
- Dellaportas, Petros, Jonathan J. Forster, and Ioannis Ntzoufras. 2012. Joint specification of model space and parameter space prior distributions. Statistical Science 27: 232–46. [Google Scholar] [CrossRef]
- Fernandez, Carmen, Eduardo Ley, and Mark F. J. Steel. 2001a. Benchmark priors for Bayesian model averaging. Journal of Econometrics 100: 381–427. [Google Scholar] [CrossRef] [Green Version]
- Fernandez, Carmen, Eduardo Ley, and Mark F. J. Steel. 2001b. Model uncertainty in cross-country growth regressions. Journal of Applied Econometrics 16: 563–576. [Google Scholar] [CrossRef]
- Fouskakis, Dimitris, and Ioannis Ntzoufras. 2020. Power-Expected-Posterior priors as mixtures of g-priors. arXiv arXiv:2002.05782. [Google Scholar]
- Fouskakis, Dimitris, Ioannis Ntzoufras, and David Draper. 2015. Power-expected-posterior priors for variable selection in Gaussian linear models. Bayesian Analysis 10: 75–107. [Google Scholar] [CrossRef] [Green Version]
- Hoeting, Jennifer A., David Madigan, Adrian E. Raftery, and Chris T. Volinsky. 1999. Bayesian Model Averaging: A Tutorial. Statistical Science 14: 382–417. [Google Scholar]
- Ibrahim, Joseph G., and Ming-Hui Chen. 2000. Power prior distributions for regression models. Statistical Science 15: 46–60. [Google Scholar]
- Kass, Robert E., and Larry Wasserman. 1995. A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. Journal of the American Statistical Association 90: 928–34. [Google Scholar] [CrossRef]
- Ley, Eduardo, and Mark F. J. Steel. 2009. On the effect of prior assumptions in Bayesian model averaging with applications to growth regression. Journal of Applied Econometrics 24: 651–74. [Google Scholar] [CrossRef] [Green Version]
- Ley, Eduardo, and Mark F. J. Steel. 2012. Mixtures of g-priors for Bayesian model averaging with economic applications. Journal of Econometrics 171: 251–66. [Google Scholar] [CrossRef] [Green Version]
- Liang, Feng, Rui Paulo, German Molina, Merlise A. Clyde, and Jim O. Berger. 2008. Mixtures of g priors for Bayesian variable selection. Journal of the American Statistical Association 103: 410–23. [Google Scholar] [CrossRef]
- Madigan, David, and Jeremy York. 1995. Bayesian graphical models for discrete data. International Statistical Review 63: 215–32. [Google Scholar] [CrossRef] [Green Version]
- Pérez, José M., and James O. Berger. 2002. Expected-posterior prior distributions for model selection. Biometrika 89: 491–511. [Google Scholar] [CrossRef]
- Raftery, Adrian E., David Madigan, and Jennifer A. Hoeting. 1997. Bayesian model averaging for linear regression models. Journal of the American Statistical Association 92: 179–91. [Google Scholar] [CrossRef]
- Scott, James G., and James O. Berger. 2010. Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem. The Annals of Statistics 38: 2587–619. [Google Scholar] [CrossRef] [Green Version]
- Steel, Mark F. J. 2016. Bayesian model averaging. In Wiley StatsRef: Statistics Reference Online. Edited by Balakrishnan Narayanaswamy, Theodore Colton, Brian Everitt, Walter Piegorsch, Fabrizio Ruggeri and Jef Teugels. New Jersey: John Wiley & Sons, Ltd., pp. 1–7. [Google Scholar]
- Steel, Mark F. J. 2019. Model Averaging and its Use in Economics. arXiv arXiv:1709.08221. [Google Scholar]
- Womack, Andrew J., Luis León-Novelo, and George Casella. 2014. Inference from intrinsic Bayes’ procedures under model selection and uncertainty. Journal of the American Statistical Association 109: 1040–53. [Google Scholar] [CrossRef]

**Figure 1.**Simulation scenario 1. Predictive measures for maximum–a–posteriori (MAP) and median–probability (MP) models and Bayesian model averaging (BMA) using all models (full) and highest a–posteriori models [best 10 models and models with posterior odds (PO) versus the MAP model of at least $>1/3$].

**Figure 2.**Simulation scenario 2. Predictive measures for maximum–a–posteriori (MAP) and median–probability (MP) models and Bayesian model averaging (BMA) using all models (full) and highest a–posteriori models [best 10 models and models with posterior odds (PO) versus the MAP model of at least $>1/3$].

**Figure 4.**FLS dataset: box plots of posterior mean model size of visited models over 50 prediction subsamples.

**Figure 6.**FLS dataset: box plots of posterior medians of the shrinkage factor w over 50 prediction subsamples.

**Figure 8.**Histograms of posterior standard deviations of g for all methods under comparison for the FLS data.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Fouskakis, D.; Ntzoufras, I.
Bayesian Model Averaging Using Power-Expected-Posterior Priors. *Econometrics* **2020**, *8*, 17.
https://doi.org/10.3390/econometrics8020017

**AMA Style**

Fouskakis D, Ntzoufras I.
Bayesian Model Averaging Using Power-Expected-Posterior Priors. *Econometrics*. 2020; 8(2):17.
https://doi.org/10.3390/econometrics8020017

**Chicago/Turabian Style**

Fouskakis, Dimitris, and Ioannis Ntzoufras.
2020. "Bayesian Model Averaging Using Power-Expected-Posterior Priors" *Econometrics* 8, no. 2: 17.
https://doi.org/10.3390/econometrics8020017