On Construction of Hybrid Logistic Regression-Naïve Bayes Model for Classification

Yi Tan, Prakash P. Shenoy, Moses W. Chan, Paul M. Romberg
Proceedings of the Eighth International Conference on Probabilistic Graphical Models, PMLR 52:523-534, 2016.

Abstract

In recent years, several authors have described a hybrid discriminative-generative model for classification. In this paper we examine construction of such hybrid models from data where we use logistic regression (LR) as a discriminative component, and naïve Bayes (NB) as a generative component. First, we estimate a Markov blanket of the class variable to reduce the set of features. Next, we use a heuristic to partition the set of features in the Markov blanket into those that are assigned to the LR part, and those that are assigned to the NB part of the hybrid model. The heuristic is based on reducing the conditional dependence of the features in NB part of the hybrid model given the class variable. We implement our method on 21 different classification datasets, and we compare the prediction accuracy of hybrid models with those of pure LR and pure NB models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v52-tan16, title = {On Construction of Hybrid Logistic Regression-Na\"ive {B}ayes Model for Classification}, author = {Tan, Yi and Shenoy, Prakash P. and Chan, Moses W. and Romberg, Paul M.}, booktitle = {Proceedings of the Eighth International Conference on Probabilistic Graphical Models}, pages = {523--534}, year = {2016}, editor = {Antonucci, Alessandro and Corani, Giorgio and Campos}, Cassio Polpo}, volume = {52}, series = {Proceedings of Machine Learning Research}, address = {Lugano, Switzerland}, month = {06--09 Sep}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v52/tan16.pdf}, url = {https://proceedings.mlr.press/v52/tan16.html}, abstract = {In recent years, several authors have described a hybrid discriminative-generative model for classification. In this paper we examine construction of such hybrid models from data where we use logistic regression (LR) as a discriminative component, and naïve Bayes (NB) as a generative component. First, we estimate a Markov blanket of the class variable to reduce the set of features. Next, we use a heuristic to partition the set of features in the Markov blanket into those that are assigned to the LR part, and those that are assigned to the NB part of the hybrid model. The heuristic is based on reducing the conditional dependence of the features in NB part of the hybrid model given the class variable. We implement our method on 21 different classification datasets, and we compare the prediction accuracy of hybrid models with those of pure LR and pure NB models.} }
Endnote
%0 Conference Paper %T On Construction of Hybrid Logistic Regression-Naïve Bayes Model for Classification %A Yi Tan %A Prakash P. Shenoy %A Moses W. Chan %A Paul M. Romberg %B Proceedings of the Eighth International Conference on Probabilistic Graphical Models %C Proceedings of Machine Learning Research %D 2016 %E Alessandro Antonucci %E Giorgio Corani %E Cassio Polpo Campos} %F pmlr-v52-tan16 %I PMLR %P 523--534 %U https://proceedings.mlr.press/v52/tan16.html %V 52 %X In recent years, several authors have described a hybrid discriminative-generative model for classification. In this paper we examine construction of such hybrid models from data where we use logistic regression (LR) as a discriminative component, and naïve Bayes (NB) as a generative component. First, we estimate a Markov blanket of the class variable to reduce the set of features. Next, we use a heuristic to partition the set of features in the Markov blanket into those that are assigned to the LR part, and those that are assigned to the NB part of the hybrid model. The heuristic is based on reducing the conditional dependence of the features in NB part of the hybrid model given the class variable. We implement our method on 21 different classification datasets, and we compare the prediction accuracy of hybrid models with those of pure LR and pure NB models.
RIS
TY - CPAPER TI - On Construction of Hybrid Logistic Regression-Naïve Bayes Model for Classification AU - Yi Tan AU - Prakash P. Shenoy AU - Moses W. Chan AU - Paul M. Romberg BT - Proceedings of the Eighth International Conference on Probabilistic Graphical Models DA - 2016/08/15 ED - Alessandro Antonucci ED - Giorgio Corani ED - Cassio Polpo Campos} ID - pmlr-v52-tan16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 52 SP - 523 EP - 534 L1 - http://proceedings.mlr.press/v52/tan16.pdf UR - https://proceedings.mlr.press/v52/tan16.html AB - In recent years, several authors have described a hybrid discriminative-generative model for classification. In this paper we examine construction of such hybrid models from data where we use logistic regression (LR) as a discriminative component, and naïve Bayes (NB) as a generative component. First, we estimate a Markov blanket of the class variable to reduce the set of features. Next, we use a heuristic to partition the set of features in the Markov blanket into those that are assigned to the LR part, and those that are assigned to the NB part of the hybrid model. The heuristic is based on reducing the conditional dependence of the features in NB part of the hybrid model given the class variable. We implement our method on 21 different classification datasets, and we compare the prediction accuracy of hybrid models with those of pure LR and pure NB models. ER -
APA
Tan, Y., Shenoy, P.P., Chan, M.W. & Romberg, P.M.. (2016). On Construction of Hybrid Logistic Regression-Naïve Bayes Model for Classification. Proceedings of the Eighth International Conference on Probabilistic Graphical Models, in Proceedings of Machine Learning Research 52:523-534 Available from https://proceedings.mlr.press/v52/tan16.html.

Related Material