Information Theoretic Model Selection for Pattern Analysis

Joachim M. Buhmann, Morteza H. Chehreghani, Mario Frank, Andreas P. Streich
Proceedings of ICML Workshop on Unsupervised and Transfer Learning, PMLR 27:51-64, 2012.

Abstract

Exploratory data analysis requires (i) to define a set of patterns hypothesized to exist in the data, (ii) to specify a suitable quantification principle or cost function to rank these patterns and (iii) to validate the inferred patterns. For data clustering, the patterns are object partitionings into k groups; for PCA or truncated SVD, the patterns are orthogonal transformations with projections to a low-dimensional space. We propose an information theoretic principle for model selection and model-order selection. Our principle ranks competing pattern cost functions according to their ability to extract context sensitive information from noisy data with respect to the chosen hypothesis class. Sets of approximative solutions serve as a basis for a communication protocol. Analogous to ?, inferred models maximize the so-called approximation capacity that is the mutual information between coarsened training data patterns and coarsened test data patterns. We demonstrate how to apply our validation framework by the well-known Gaussian mixture model and by a multi-label clustering approach for role mining in binary user privilege assignments.

Cite this Paper

BibTeX
@InProceedings{pmlr-v27-buhmann12a, title = {Information Theoretic Model Selection for Pattern Analysis}, author = {Buhmann, Joachim M. and Chehreghani, Morteza H. and Frank, Mario and Streich, Andreas P.}, booktitle = {Proceedings of ICML Workshop on Unsupervised and Transfer Learning}, pages = {51--64}, year = {2012}, editor = {Guyon, Isabelle and Dror, Gideon and Lemaire, Vincent and Taylor, Graham and Silver, Daniel}, volume = {27}, series = {Proceedings of Machine Learning Research}, address = {Bellevue, Washington, USA}, month = {02 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v27/buhmann12a/buhmann12a.pdf}, url = {https://proceedings.mlr.press/v27/buhmann12a.html}, abstract = {Exploratory data analysis requires (i) to define a set of patterns hypothesized to exist in the data, (ii) to specify a suitable quantification principle or cost function to rank these patterns and (iii) to validate the inferred patterns. For data clustering, the patterns are object partitionings into k groups; for PCA or truncated SVD, the patterns are orthogonal transformations with projections to a low-dimensional space. We propose an information theoretic principle for model selection and model-order selection. Our principle ranks competing pattern cost functions according to their ability to extract context sensitive information from noisy data with respect to the chosen hypothesis class. Sets of approximative solutions serve as a basis for a communication protocol. Analogous to ?, inferred models maximize the so-called approximation capacity that is the mutual information between coarsened training data patterns and coarsened test data patterns. We demonstrate how to apply our validation framework by the well-known Gaussian mixture model and by a multi-label clustering approach for role mining in binary user privilege assignments.} }
Endnote
%0 Conference Paper %T Information Theoretic Model Selection for Pattern Analysis %A Joachim M. Buhmann %A Morteza H. Chehreghani %A Mario Frank %A Andreas P. Streich %B Proceedings of ICML Workshop on Unsupervised and Transfer Learning %C Proceedings of Machine Learning Research %D 2012 %E Isabelle Guyon %E Gideon Dror %E Vincent Lemaire %E Graham Taylor %E Daniel Silver %F pmlr-v27-buhmann12a %I PMLR %P 51--64 %U https://proceedings.mlr.press/v27/buhmann12a.html %V 27 %X Exploratory data analysis requires (i) to define a set of patterns hypothesized to exist in the data, (ii) to specify a suitable quantification principle or cost function to rank these patterns and (iii) to validate the inferred patterns. For data clustering, the patterns are object partitionings into k groups; for PCA or truncated SVD, the patterns are orthogonal transformations with projections to a low-dimensional space. We propose an information theoretic principle for model selection and model-order selection. Our principle ranks competing pattern cost functions according to their ability to extract context sensitive information from noisy data with respect to the chosen hypothesis class. Sets of approximative solutions serve as a basis for a communication protocol. Analogous to ?, inferred models maximize the so-called approximation capacity that is the mutual information between coarsened training data patterns and coarsened test data patterns. We demonstrate how to apply our validation framework by the well-known Gaussian mixture model and by a multi-label clustering approach for role mining in binary user privilege assignments.
RIS
TY - CPAPER TI - Information Theoretic Model Selection for Pattern Analysis AU - Joachim M. Buhmann AU - Morteza H. Chehreghani AU - Mario Frank AU - Andreas P. Streich BT - Proceedings of ICML Workshop on Unsupervised and Transfer Learning DA - 2012/06/27 ED - Isabelle Guyon ED - Gideon Dror ED - Vincent Lemaire ED - Graham Taylor ED - Daniel Silver ID - pmlr-v27-buhmann12a PB - PMLR DP - Proceedings of Machine Learning Research VL - 27 SP - 51 EP - 64 L1 - http://proceedings.mlr.press/v27/buhmann12a/buhmann12a.pdf UR - https://proceedings.mlr.press/v27/buhmann12a.html AB - Exploratory data analysis requires (i) to define a set of patterns hypothesized to exist in the data, (ii) to specify a suitable quantification principle or cost function to rank these patterns and (iii) to validate the inferred patterns. For data clustering, the patterns are object partitionings into k groups; for PCA or truncated SVD, the patterns are orthogonal transformations with projections to a low-dimensional space. We propose an information theoretic principle for model selection and model-order selection. Our principle ranks competing pattern cost functions according to their ability to extract context sensitive information from noisy data with respect to the chosen hypothesis class. Sets of approximative solutions serve as a basis for a communication protocol. Analogous to ?, inferred models maximize the so-called approximation capacity that is the mutual information between coarsened training data patterns and coarsened test data patterns. We demonstrate how to apply our validation framework by the well-known Gaussian mixture model and by a multi-label clustering approach for role mining in binary user privilege assignments. ER -
APA
Buhmann, J.M., Chehreghani, M.H., Frank, M. & Streich, A.P.. (2012). Information Theoretic Model Selection for Pattern Analysis. Proceedings of ICML Workshop on Unsupervised and Transfer Learning, in Proceedings of Machine Learning Research 27:51-64 Available from https://proceedings.mlr.press/v27/buhmann12a.html.

Related Material