Efficient Sparse Group Feature Selection via Nonconvex Optimization

Shuo Xiang, Xiaoshen Tong, Jieping Ye
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(1):284-292, 2013.

Abstract

Sparse feature selection has been demonstrated to be effective in handling high-dimensional data. While promising, most of the existing works use convex methods, which may be suboptimal in terms of the accuracy of feature selection and parameter estimation. In this paper, we expand a nonconvex paradigm to sparse group feature selection, which is motivated by applications that require identifying the underlying group structure and performing feature selection simultaneously. The main contributions of this article are twofold: (1) computationally, we introduce a nonconvex sparse group feature selection model and present an efficient optimization algorithm, of which the key step is a projection with two coupled constraints; (2) statistically, we show that the proposed model can reconstruct the oracle estimator. Therefore, consistent feature selection and parameter estimation can be achieved. Numerical results on synthetic and real-world data suggest that the proposed nonconvex method compares favorably against its competitors, thus achieving desired goal of delivering high performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-xiang13, title = {Efficient Sparse Group Feature Selection via Nonconvex Optimization}, author = {Xiang, Shuo and Tong, Xiaoshen and Ye, Jieping}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {284--292}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {1}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/xiang13.pdf}, url = {https://proceedings.mlr.press/v28/xiang13.html}, abstract = {Sparse feature selection has been demonstrated to be effective in handling high-dimensional data. While promising, most of the existing works use convex methods, which may be suboptimal in terms of the accuracy of feature selection and parameter estimation. In this paper, we expand a nonconvex paradigm to sparse group feature selection, which is motivated by applications that require identifying the underlying group structure and performing feature selection simultaneously. The main contributions of this article are twofold: (1) computationally, we introduce a nonconvex sparse group feature selection model and present an efficient optimization algorithm, of which the key step is a projection with two coupled constraints; (2) statistically, we show that the proposed model can reconstruct the oracle estimator. Therefore, consistent feature selection and parameter estimation can be achieved. Numerical results on synthetic and real-world data suggest that the proposed nonconvex method compares favorably against its competitors, thus achieving desired goal of delivering high performance.} }
Endnote
%0 Conference Paper %T Efficient Sparse Group Feature Selection via Nonconvex Optimization %A Shuo Xiang %A Xiaoshen Tong %A Jieping Ye %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-xiang13 %I PMLR %P 284--292 %U https://proceedings.mlr.press/v28/xiang13.html %V 28 %N 1 %X Sparse feature selection has been demonstrated to be effective in handling high-dimensional data. While promising, most of the existing works use convex methods, which may be suboptimal in terms of the accuracy of feature selection and parameter estimation. In this paper, we expand a nonconvex paradigm to sparse group feature selection, which is motivated by applications that require identifying the underlying group structure and performing feature selection simultaneously. The main contributions of this article are twofold: (1) computationally, we introduce a nonconvex sparse group feature selection model and present an efficient optimization algorithm, of which the key step is a projection with two coupled constraints; (2) statistically, we show that the proposed model can reconstruct the oracle estimator. Therefore, consistent feature selection and parameter estimation can be achieved. Numerical results on synthetic and real-world data suggest that the proposed nonconvex method compares favorably against its competitors, thus achieving desired goal of delivering high performance.
RIS
TY - CPAPER TI - Efficient Sparse Group Feature Selection via Nonconvex Optimization AU - Shuo Xiang AU - Xiaoshen Tong AU - Jieping Ye BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/02/13 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-xiang13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 1 SP - 284 EP - 292 L1 - http://proceedings.mlr.press/v28/xiang13.pdf UR - https://proceedings.mlr.press/v28/xiang13.html AB - Sparse feature selection has been demonstrated to be effective in handling high-dimensional data. While promising, most of the existing works use convex methods, which may be suboptimal in terms of the accuracy of feature selection and parameter estimation. In this paper, we expand a nonconvex paradigm to sparse group feature selection, which is motivated by applications that require identifying the underlying group structure and performing feature selection simultaneously. The main contributions of this article are twofold: (1) computationally, we introduce a nonconvex sparse group feature selection model and present an efficient optimization algorithm, of which the key step is a projection with two coupled constraints; (2) statistically, we show that the proposed model can reconstruct the oracle estimator. Therefore, consistent feature selection and parameter estimation can be achieved. Numerical results on synthetic and real-world data suggest that the proposed nonconvex method compares favorably against its competitors, thus achieving desired goal of delivering high performance. ER -
APA
Xiang, S., Tong, X. & Ye, J.. (2013). Efficient Sparse Group Feature Selection via Nonconvex Optimization. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(1):284-292 Available from https://proceedings.mlr.press/v28/xiang13.html.

Related Material