# Hidden Node Detection between Observable Nodes Based on Bayesian Clustering

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Model Settings

## 3. Bayesian Clustering

## 4. Hidden Node Detection

#### 4.1. The Proposed Algorithm

**Example**

**1.**

**Algorithm**

**2.**

#### 4.2. Asymptotic Properties of the Algorithm

**Theorem**

**3.**

## 5. Numerical Experiments

## 6. Discussion

## 7. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## Appendix A

## References

- Rissanen, J. Stochastic complexity and modeling. Ann. Stat.
**1986**, 14, 1080–1100. [Google Scholar] - Schwarz, G.E. Estimating the dimension of a model. Ann. Stat.
**1978**, 6, 461–464. [Google Scholar] - Akaike, H. A new look at the statistical model identification. IEEE Trans. Autom. Control
**1974**, 19, 716–723. [Google Scholar] - Good, I.J. The Estimation of Probabilities: An Essay on Modern Bayesian Methods, Research Monograph No. 30; The MIT Press: Cambridge, MA, USA, 1965. [Google Scholar]
- Rusakov, D.; Geiger, D. Asymptotic model selection for naive Bayesian networks. J. Mach. Learn. Res.
**2005**, 6, 1–35. [Google Scholar] - Watanabe, S. Algebraic Geometry and Statistical Learning Theory; Cambridge University Press: New York, NY, USA, 2009. [Google Scholar]
- Watanabe, S. Algebraic analysis for non-identifiable learning machines. Neural Comput.
**2001**, 13, 899–933. [Google Scholar] - Yamazaki, K.; Watanabe, S. Singularities in mixture models and upper bounds of stochastic complexity. Int. J. Neural Netw.
**2003**, 16, 1029–1038. [Google Scholar][Green Version] - Yamazaki, K.; Watanabe, S. Algebraic geometry and stochastic complexity of hidden Markov models. Neurocomputing
**2005**, 69, 62–84. [Google Scholar] - Aoyagi, M. Consideration on Singularities in Learning Theory and the Learning Coefficient. Entropy
**2013**, 15, 3714–3733. [Google Scholar][Green Version] - Geiger, D.; Heckerman, D.; Meek, C. Asymptotic Model Selection for Directed Networks with Hidden Variables. In Proceedings of the Twelfth Annual Conference on Uncertainty in Artificial Intelligence, Portland, OR, USA, 1–4 August 1996; pp. 283–290. [Google Scholar]
- Drton, M.; Plummer, M. A Bayesian information criterion for singular models. J. R. Stat. Soc. Ser. B Stat. Methodol.
**2017**, 79, 323–380. [Google Scholar][Green Version] - Verma, T.; Pearl, J. Equivalence and Synthesis of Causal Models. In Proceedings of the Sixth Annual Conference on Uncertainty in Artificial Intelligence, Cambridge, MA, USA, 27–29 July 1990; Elsevier Science Inc.: New York, NY, USA, 1991; pp. 255–270. [Google Scholar]
- Richardson, T.; Spirtes, P. Ancestral Graph Markov Models. Ann. Stat.
**2000**, 30, 2002. [Google Scholar] - Zhang, J. On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias. Artif. Intell.
**2008**, 172, 1873–1896. [Google Scholar][Green Version] - Chaturvedi, I.; Ragusa, E.; Gastaldo, P.; Zunino, R.; Cambria, E. Bayesian network based extreme learning machine for subjectivity detection. J. Frankl. Inst.
**2018**, 355, 1780–1797. [Google Scholar] - Allman, E.S.; Rhodes, J.A.; Sturmfels, B.; Zwiernik, P. Tensors of nonnegative rank two. Linear Algebra Appl.
**2015**, 473, 37–53. [Google Scholar][Green Version] - Blei, D.M.; Ng, A.Y.; Jordan, M.I. Latent Dirichlet Allocation. J. Mach. Learn. Res.
**2003**, 3, 993–1022. [Google Scholar] - Heckerman, D.; Geiger, D.; Chickering, D.M. Learning Bayesian Networks: The Combination of Knowledge and Statistical Data. Mach. Learn.
**1995**, 20, 197–243. [Google Scholar][Green Version] - Buntine, W. Theory Refinement on Bayesian Networks. In Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, Los Angeles, CA, USA, 13–15 July 1991; Morgan Kaufmann Publishers Inc.: San Francisco, CA, USA, 1991; pp. 52–60. [Google Scholar][Green Version]
- Yamazaki, K. Asymptotic accuracy of Bayes estimation for latent variables with redundancy. Mach. Learn.
**2016**, 102, 1–28. [Google Scholar]

Data-Set ID | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|

Estimated size | 3 | 3 | 3 | 4 | 3 | 3 | 4 | 4 | 4 | 3 |

Data-Set ID | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|

Estimated size (n = 100) | 3 | 3 | 3 | 3 | 4 | 3 | 4 | 3 | 3 | 3 |

Estimated size (n = 500) | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 4 |

Data-Set ID | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
---|---|---|---|---|---|---|---|---|---|---|

Estimated size (skewed parent node) | 3 | 3 | 3 | 3 | 3 | 4 | 3 | 3 | 3 | 3 |

Estimated size (nearly-uniform child node) | 1 | 1 | 1 | 1 | 2 | 1 | 1 | 2 | 1 | 1 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Yamazaki, K.; Motomura, Y.
Hidden Node Detection between Observable Nodes Based on Bayesian Clustering. *Entropy* **2019**, *21*, 32.
https://doi.org/10.3390/e21010032

**AMA Style**

Yamazaki K, Motomura Y.
Hidden Node Detection between Observable Nodes Based on Bayesian Clustering. *Entropy*. 2019; 21(1):32.
https://doi.org/10.3390/e21010032

**Chicago/Turabian Style**

Yamazaki, Keisuke, and Yoichi Motomura.
2019. "Hidden Node Detection between Observable Nodes Based on Bayesian Clustering" *Entropy* 21, no. 1: 32.
https://doi.org/10.3390/e21010032