# Inference Using Simulated Neural Moments

^{1}

^{2}

## Abstract

**:**

## 1. Introduction

## 2. Simulated Moments, Indirect Likelihood, and MSM-MCMC Inference

## 3. Neural Moments

- Draw ${\theta}^{s}$ from the parameter space $\Theta ,$ using some prior distribution.
- Draw a sample ${Y}^{s}$ from the model $M\left(\theta \right)$ at ${\theta}^{s}$.
- Compute the vector of raw statistics $W\left({Y}^{s}\right).$

## 4. Examples

#### 4.1. Stochastic Volatility

#### 4.2. ARMA

#### 4.3. Mixture of Normals

#### 4.4. DSGE Model

## 5. Monte Carlo Results

## 6. Application: A Jump-Diffusion Model of S&P 500 Returns

## 7. Conclusions

## Funding

## Data Availability Statement

## Conflicts of Interest

## Notes

1 | These definitions and notation are loosely based on Jiang and Turnbull (2004). |

2 | It may be noted that methods other than MCMC may be used to generate the set of draws from the posterior, C. For example, one might use sequential Monte Carlo. Point estimation and inference using C remains the same regardless of how C is generated. |

3 | The function which specifies and trains the neural net is MakeNeuralMoments.jl. |

4 | See the file MCMC.jl for the details of how this proposal density is implemented. |

5 | |

6 | See the file SVlib.jl for details. |

7 | Details are in the file ARMAlib.jl. |

8 | Details are in the file MNlib.jl. |

9 | The details of the model and priors may be seen at CKlib.jl. The model is solved using third order projection, making use of the SolveDSGE.jl package. The model is discussed in more detail in Chapter 14 of the document econometrics.pdf. |

10 | These results are available for the SV and ARMA models, as well as an unreported additional model, in the WP branch of the GitHub archive. |

11 | The model is solved and simulated using the SRIW1 strong order 1.5 solver from the DifferentialEquations.jl package for the Julia language. |

12 | The workstation has 4 Opteron 6380 processors, each with 4 physical cores, running at 2500 MHz. |

13 | The data source is the Oxford–Man Institute’s realized library, v. 0.3, https://realized.oxford-man.ox.ac.uk/images/oxfordmanrealizedvolatilityindices.zip. |

## References

- Barndorff-Nielsen, Ole E., and Neil Shephard. 2002. Econometric analysis of realized volatility and its use in estimating stochastic volatility models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 64: 253–80. [Google Scholar] [CrossRef]
- Chernozhukov, Victor, and Han Hong. 2003. An MCMC approach to classical estimation. Journal of Econometrics 115: 293–346. [Google Scholar] [CrossRef] [Green Version]
- Christiano, Lawrence J., Martin S. Eichenbaum, and Mathias Trabandt. 2016. Unemployment and business cycles. Econometrica 84: 1523–69. [Google Scholar] [CrossRef]
- Christiano, Lawrence J., Mathias Trabandt, and Karl Walentin. 2010. Dsge models for monetary policy analysis. In Handbook of Monetary Economics. Amsterdam: Elsevier, vol. 3, pp. 285–367. [Google Scholar]
- Corsi, Fulvio. 2009. A simple approximate long-memory model of realized volatility. Journal of Financial Econometrics 7: 174–96. [Google Scholar] [CrossRef]
- Creel, Michael. 2017. Neural nets for indirect inference. Econometrics and Statistics 2: 36–49. [Google Scholar] [CrossRef] [Green Version]
- Creel, Michael, and Dennis Kristensen. 2015. Abc of sv: Limited information likelihood inference in stochastic volatility jump-diffusion models. Journal of Empirical Finance 31: 85–108. [Google Scholar] [CrossRef] [Green Version]
- Donald, Stephen G., Guido W. Imbens, and Whitney K. Newey. 2009. Choosing instrumental variables in conditional moment restriction models. Journal of Econometrics 152: 28–36. [Google Scholar] [CrossRef]
- Gallant, A. Ronald, and George Tauchen. 1996. Which moments to match? Econometric Theory 12: 363–90. [Google Scholar] [CrossRef]
- Gallant, A. Ronald, and George Tauchen. 2010. Emm: A Program for Efficient Method of Moments Estimation, Version 2.6, User’S Guide. Chapel Hill: University of North Carolina. [Google Scholar]
- Gourieroux, Christian, Alain Monfort, and Eric Renault. 1993. Indirect inference. Journal of Applied Econometrics 8: S85–S118. [Google Scholar] [CrossRef]
- Hall, Peter, and Joel L. Horowitz. 1996. Bootstrap critical values for tests based on generalized-method-of-moments estimators. Econometrica 64: 891–916. [Google Scholar] [CrossRef]
- Hansen, Lars Peter, John Heaton, and Amir Yaron. 1996. Finite-sample properties of some alternative GMM estimators. Journal of Business & Economic Statistics 14: 262–80. [Google Scholar] [CrossRef] [Green Version]
- Hornik, Kurt, Maxwell Stinchcombe, and Halbert White. 1989. Multilayer feedforward networks are universal approximators. Neural Networks 2: 359–66. [Google Scholar] [CrossRef]
- Jiang, Wenxin, and Bruce Turnbull. 2004. The indirect method: Inference based on intermediate statistics a synthesis and examples. Statistical Science 19: 239–63. [Google Scholar] [CrossRef]
- Lu, Zhou, Hongming Pu, Feicheng Wang, Zhiqiang Hu, and Liwei Wang. 2017. The expressive power of neural networks: A view from the width. Paper presented at 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, December 4–9; pp. 6232–40. [Google Scholar]
- Marjoram, Paul, John Molitor, Vincent Plagnol, and Simon Tavaré. 2003. Markov chain monte carlo without likelihoods. Proceedings of the National Academy of Sciences 100: 15324–28. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- McFadden, Daniel. 1989. A method of simulated moments for estimation of discrete response models without numerical integration. Econometrica 57: 995–1026. [Google Scholar] [CrossRef] [Green Version]
- Smith, Anthony A. 1993. Estimating nonlinear time-series models using simulated vector autoregressions. Journal of Applied Econometrics 8: S63–S84. [Google Scholar] [CrossRef] [Green Version]
- Tauchen, George. 1986. Statistical properties of generalized method-of-moments estimators of structural parameters obtained from financial market data. Journal of Business & Economic Statistics 4: 397. [Google Scholar] [CrossRef] [Green Version]

**Figure 3.**MCMC results for the jump-diffusion model of S&P 500 data. Posterior mean in blue, posterior median in black. The green-yellow borders define the limits of a 90% confidence interval.

Model | Parameter | True Value | W (CUE) | Z (Two-Step) | Z (CUE) |
---|---|---|---|---|---|

SV | $\varphi $ | 0.692 | 0.123 | 0.064 | 0.076 |

$\rho $ | 0.90 | 0.086 | 0.082 | 0.086 | |

$\sigma $ | 0.363 | 0.138 | 0.105 | 0.105 | |

ARMA | $\alpha $ | 0.95 | 0.030 | 0.028 | 0.047 |

$\beta $ | 0.5 | 0.078 | 0.067 | 0.068 | |

${\sigma}^{2}$ | 1.0 | 0.099 | 0.091 | 0.084 |

**Table 2.**90% confidence interval coverage for SV and ARMA models, using raw ($W)$ or neural net ($Z)$ statistics. Correct coverage rejected when outside 0.864–0.932.

Model | Parameter | W (CUE) | Z (Two-Step) | Z (CUE) |
---|---|---|---|---|

SV | $\varphi $ | 0.876 | 0.884 | 0.912 |

$\rho $ | 0.732 | 0.976 | 0.910 | |

$\sigma $ | 0.762 | 0.956 | 0.928 | |

ARMA | $\alpha $ | 0.786 | 0.988 | 0.916 |

$\beta $ | 0.814 | 0.954 | 0.918 | |

${\sigma}^{2}$ | 0.808 | 0.920 | 0.912 |

**Table 3.**95% confidence interval coverage for SV and ARMA models, using raw ($W)$ or neural net ($Z)$ statistics. Correct coverage rejected when outside 0.924–0.974.

Model | Parameter | W (CUE) | Z (Two-Step) | Z (CUE) |
---|---|---|---|---|

SV | $\varphi $ | 0.916 | 0.938 | 0.954 |

$\rho $ | 0.796 | 0.990 | 0.944 | |

$\sigma $ | 0.824 | 0.976 | 0.958 | |

ARMA | $\alpha $ | 0.838 | 0.994 | 0.966 |

$\beta $ | 0.856 | 0.984 | 0.942 | |

${\sigma}^{2}$ | 0.880 | 0.966 | 0.954 |

**Table 4.**99% confidence interval coverage for SV and ARMA models, using raw ($W)$ or neural net ($Z)$ statistics. Correct coverage rejected when outside 0.976–1.000.

Model | Parameter | W (CUE) | Z (Two-Step) | Z (CUE) |
---|---|---|---|---|

SV | $\varphi $ | 0.936 | 0.968 | 0.990 |

$\rho $ | 0.848 | 0.998 | 0.978 | |

$\sigma $ | 0.888 | 0.994 | 0.986 | |

ARMA | $\alpha $ | 0.898 | 1.000 | 0.988 |

$\beta $ | 0.916 | 0.998 | 0.986 | |

${\sigma}^{2}$ | 0.920 | 0.994 | 0.990 |

Model | Parameter | True Value | Z (Two-Step) | Z (CUE) |
---|---|---|---|---|

MN | ${\mu}_{1}$ | 1.0 | 0.019 | 0.018 |

$\sigma {}_{1}$ | 0.2 | 0.087 | 0.089 | |

${\mu}_{2}$ | 0.0 | 0.021 | 0.020 | |

$\sigma {}_{2}$ | 2.0 | 0.064 | 0.065 | |

p | 0.4 | 0.024 | 0.025 | |

DSGE | $\beta $ | 0.99 | 0.001 | 0.000 |

$\gamma $ | 2.00 | 0.083 | 0.085 | |

${\rho}_{z}$ | 0.9 | 0.009 | 0.008 | |

${\sigma}_{z}$ | 0.02 | 0.001 | 0.001 | |

${\rho}_{\eta}$ | 0.7 | 0.050 | 0.055 | |

${\sigma}_{\eta}$ | 0.01 | 0.001 | 0.001 | |

$\overline{nss}$ | 1/3 | 0.001 | 0.001 |

**Table 6.**90% confidence interval coverage for MN and DSGE models. Correct coverage rejected when outside 0.864–0.932.

Model | Parameter | Z (Two-Step) | Z (CUE) |
---|---|---|---|

MN | ${\mu}_{1}$ | 0.920 | 0.914 |

$\sigma {}_{1}$ | 0.934 | 0.922 | |

${\mu}_{2}$ | 0.906 | 0.918 | |

$\sigma {}_{2}$ | 0.934 | 0.920 | |

p | 0.922 | 0.908 | |

DSGE | $\beta $ | 0.950 | 0.914 |

$\gamma $ | 0.968 | 0.920 | |

${\rho}_{z}$ | 0.928 | 0.928 | |

${\sigma}_{z}$ | 0.910 | 0.892 | |

${\rho}_{\eta}$ | 0.892 | 0.890 | |

${\sigma}_{\eta}$ | 0.972 | 0.906 | |

$\overline{nss}$ | 0.924 | 0.902 |

**Table 7.**95% confidence interval coverage for MN and DSGE models. Correct coverage rejected when outside 0.924–0.974.

Model | Parameter | Z (Two-Step) | Z (CUE) |
---|---|---|---|

MN | ${\mu}_{1}$ | 0.956 | 0.962 |

$\sigma {}_{1}$ | 0.976 | 0.962 | |

${\mu}_{2}$ | 0.944 | 0.952 | |

$\sigma {}_{2}$ | 0.964 | 0.958 | |

p | 0.960 | 0.958 | |

DSGE | $\beta $ | 0.972 | 0.962 |

$\gamma $ | 0.990 | 0.962 | |

${\rho}_{z}$ | 0.960 | 0.958 | |

${\sigma}_{z}$ | 0.952 | 0.946 | |

${\rho}_{\eta}$ | 0.950 | 0.938 | |

${\sigma}_{\eta}$ | 0.996 | 0.952 | |

$\overline{nss}$ | 0.966 | 0.956 |

**Table 8.**99% confidence interval coverage for MN and DSGE models. Correct coverage rejected when outside 0.976–1.000.

Model | Parameter | Z (Two-Step) | Z (CUE) |
---|---|---|---|

MN | ${\mu}_{1}$ | 0.990 | 0.990 |

$\sigma {}_{1}$ | 0.996 | 0.992 | |

${\mu}_{2}$ | 9.986 | 0.984 | |

$\sigma {}_{2}$ | 0.992 | 0.994 | |

p | 0.996 | 0.986 | |

DSGE | $\beta $ | 0.996 | 0.990 |

$\gamma $ | 1.000 | 0.986 | |

${\rho}_{z}$ | 0.976 | 0.980 | |

${\sigma}_{z}$ | 0.986 | 0.990 | |

${\rho}_{\eta}$ | 0.990 | 0.982 | |

${\sigma}_{\eta}$ | 1.000 | 0.992 | |

$\overline{nss}$ | 0.988 | 0.988 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Creel, M.
Inference Using Simulated Neural Moments. *Econometrics* **2021**, *9*, 35.
https://doi.org/10.3390/econometrics9040035

**AMA Style**

Creel M.
Inference Using Simulated Neural Moments. *Econometrics*. 2021; 9(4):35.
https://doi.org/10.3390/econometrics9040035

**Chicago/Turabian Style**

Creel, Michael.
2021. "Inference Using Simulated Neural Moments" *Econometrics* 9, no. 4: 35.
https://doi.org/10.3390/econometrics9040035