Next Article in Journal
Algorithmic Relative Complexity
Previous Article in Journal
Optimal Multi-Level Thresholding Based on Maximum Tsallis Entropy via an Artificial Bee Colony Approach
Entropy 2011, 13(4), 860-901; doi:10.3390/e13040860
Article

A Feature Subset Selection Method Based On High-Dimensional Mutual Information

1,2,*  and 3,*
Received: 8 January 2011; in revised form: 18 March 2011 / Accepted: 23 March 2011 / Published: 19 April 2011
Download PDF [2671 KB, updated 21 April 2011; original version uploaded 19 April 2011]
Abstract: Feature selection is an important step in building accurate classifiers and provides better understanding of the data sets. In this paper, we propose a feature subset selection method based on high-dimensional mutual information. We also propose to use the entropy of the class attribute as a criterion to determine the appropriate subset of features when building classifiers. We prove that if the mutual information between a feature set X and the class attribute Y equals to the entropy of Y , then X is a Markov Blanket of Y . We show that in some cases, it is infeasible to approximate the high-dimensional mutual information with algebraic combinations of pairwise mutual information in any forms. In addition, the exhaustive searches of all combinations of features are prerequisite for finding the optimal feature subsets for classifying these kinds of data sets. We show that our approach outperforms existing filter feature subset selection methods for most of the 24 selected benchmark data sets.
Keywords: feature selection; mutual information; Entropy; information theory; Markov blanket; classification feature selection; mutual information; Entropy; information theory; Markov blanket; classification
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Export to BibTeX |
EndNote


MDPI and ACS Style

Zheng, Y.; Kwoh, C.K. A Feature Subset Selection Method Based On High-Dimensional Mutual Information. Entropy 2011, 13, 860-901.

AMA Style

Zheng Y, Kwoh CK. A Feature Subset Selection Method Based On High-Dimensional Mutual Information. Entropy. 2011; 13(4):860-901.

Chicago/Turabian Style

Zheng, Yun; Kwoh, Chee Keong. 2011. "A Feature Subset Selection Method Based On High-Dimensional Mutual Information." Entropy 13, no. 4: 860-901.


Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert