Approximating Mutual Information by Maximum Likelihood Density Ratio Estimation
Taiji Suzuki, Masashi Sugiyama, Jun Sese, Takafumi Kanamori;
JMLR W&P 4:5-20, 2008.
Abstract
Mutual information is useful in various data processing tasks
such as feature selection or independent component analysis.
In this paper, we propose a new method of approximating mutual information
based on maximum likelihood estimation of a density ratio function.
Our method, called Maximum Likelihood Mutual Information (MLMI),
has several attractive properties, e.g.,
density estimation is not involved,
it is a single-shot procedure,
the global optimal solution can be efficiently computed,
and
cross-validation is available for model selection.
Numerical experiments show that MLMI compares favorably
with existing methods.
