On Normalized Mutual Information: Measure Derivations and Properties
AbstractStarting with a new formulation for the mutual information (MI) between a pair of events, this paper derives alternative upper bounds and extends those to the case of two discrete random variables. Normalized mutual information (NMI) measures are then obtained from those bounds, emphasizing the use of least upper bounds. Conditional NMI measures are also derived for three different events and three different random variables. Since the MI formulation for a pair of events is always nonnegative, it can properly be extended to include weighted MI and NMI measures for pairs of events or for random variables that are analogous to the well-known weighted entropy. This weighted MI is generalized to the case of continuous random variables. Such weighted measures have the advantage over previously proposed measures of always being nonnegative. A simple transformation is derived for the NMI, such that the transformed measures have the value-validity property necessary for making various appropriate comparisons between values of those measures. A numerical example is provided. View Full-Text
Share & Cite This Article
Kvålseth, T.O. On Normalized Mutual Information: Measure Derivations and Properties. Entropy 2017, 19, 631.
Kvålseth TO. On Normalized Mutual Information: Measure Derivations and Properties. Entropy. 2017; 19(11):631.Chicago/Turabian Style
Kvålseth, Tarald O. 2017. "On Normalized Mutual Information: Measure Derivations and Properties." Entropy 19, no. 11: 631.
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.