A Bayesian View of Challenges in Feature Selection: Feature Aggregation, Multiple Targets, Redundancy and Interaction
Peter Antal, Andras Millinghoffer, Gábor Hullám, Csaba Szalai, András Falus;
JMLR W&P 4:74-89, 2008.
Abstract
In the paper we discuss applications of the Bayesian approach to new challenges in relevance analysis. Earlier, we formulated a Bayesian
approach to Feature Subset Selection using Bayesian networks to jointly estimate the posteriors of Markov Blanket Memberships (MBMs),
Markov Blanket Sets (MBSs), and Markov Blanket Graphs (MBGs) for a given target variable. These results of the Bayesian Multilevel Analysis
of relevance (BMLA) correspond respectively to a model-based pairwise relevance, relevance of sets, and to the interaction models of
relevant variables. Now we formulate refined levels in BMLA by introducing the concepts of k-MBSs and k-MBGs, which are intermediate,
scalable model properties expressing relevance. We consider the extension of BMLA to multiple targets. We introduce and investigate a score
for feature redundancy and interaction based on the decomposability of the structure posterior. Finally, we overview the problems of
conditional and contextual relevance. We demonstrate the use of concepts and methods in the field of genomics of asthma.
