Next Article in Journal
Data-Rate Constrained Observers of Nonlinear Systems
Previous Article in Journal
Non-Geodesic Incompleteness in Poincaré Gauge Gravity
Previous Article in Special Issue
Mixture of Experts with Entropic Regularization for Data Classification
Article Menu
Issue 3 (March) cover image

Export Article

Open AccessArticle
Entropy 2019, 21(3), 281; https://doi.org/10.3390/e21030281

An Information Criterion for Auxiliary Variable Selection in Incomplete Data Analysis

1
Graduate School of Science, Hiroshima University, Hiroshima 739-8526, Japan
2
Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan
3
RIKEN Center for Advanced Intelligence Project, Tokyo 103-0027, Japan
*
Author to whom correspondence should be addressed.
Received: 21 February 2019 / Revised: 9 March 2019 / Accepted: 12 March 2019 / Published: 14 March 2019
(This article belongs to the Special Issue Information-Theoretical Methods in Data Mining)
Full-Text   |   PDF [907 KB, uploaded 15 March 2019]   |  

Abstract

Statistical inference is considered for variables of interest, called primary variables, when auxiliary variables are observed along with the primary variables. We consider the setting of incomplete data analysis, where some primary variables are not observed. Utilizing a parametric model of joint distribution of primary and auxiliary variables, it is possible to improve the estimation of parametric model for the primary variables when the auxiliary variables are closely related to the primary variables. However, the estimation accuracy reduces when the auxiliary variables are irrelevant to the primary variables. For selecting useful auxiliary variables, we formulate the problem as model selection, and propose an information criterion for predicting primary variables by leveraging auxiliary variables. The proposed information criterion is an asymptotically unbiased estimator of the Kullback–Leibler divergence for complete data of primary variables under some reasonable conditions. We also clarify an asymptotic equivalence between the proposed information criterion and a variant of leave-one-out cross validation. Performance of our method is demonstrated via a simulation study and a real data example. View Full-Text
Keywords: Akaike information criterion; auxiliary variables; Fisher information matrix; incomplete data; Kullback–Leibler divergence; misspecification; Takeuchi information criterion Akaike information criterion; auxiliary variables; Fisher information matrix; incomplete data; Kullback–Leibler divergence; misspecification; Takeuchi information criterion
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Imori, S.; Shimodaira, H. An Information Criterion for Auxiliary Variable Selection in Incomplete Data Analysis. Entropy 2019, 21, 281.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top