EP-GIG Priors and Applications in Bayesian Sparse Learning

Zhihua Zhang; Shusen Wang; Dehua Liu; Michael I. Jordan

In this paper we propose a novel framework for the construction of sparsity-inducing priors. In particular, we define such priors as a mixture of exponential power distributions with a generalized inverse Gaussian density (EP-GIG). EP-GIG is a variant of generalized hyperbolic distributions, and the special cases include Gaussian scale mixtures and Laplace scale mixtures. Furthermore, Laplace scale mixtures can subserve a Bayesian framework for sparse learning with nonconvex penalization. The densities of EP-GIG can be explicitly expressed. Moreover, the corresponding posterior distribution also follows a generalized inverse Gaussian distribution. We exploit these properties to develop EM algorithms for sparse empirical Bayesian learning. We also show that these algorithms bear an interesting resemblance to iteratively reweighted l₂ or l₁ methods. Finally, we present two extensions for grouped variable selection and logistic regression.

EP-GIG Priors and Applications in Bayesian Sparse Learning

Abstract