New Perspectives on k-Support and Cluster Norms
Andrew M. McDonald, Massimiliano Pontil, Dimitris Stamos.
Year: 2016, Volume: 17, Issue: 155, Pages: 1−38
Abstract
We study a regularizer which is defined as a parameterized infimum of quadratics, and which we call the box-norm. We show that the $k$-support norm, a regularizer proposed by Argyriou et al. (2012) for sparse vector prediction problems, belongs to this family, and the box-norm can be generated as a perturbation of the former. We derive an improved algorithm to compute the proximity operator of the squared box-norm, and we provide a method to compute the norm. We extend the norms to matrices, introducing the spectral $k$-support norm and spectral box-norm. We note that the spectral box-norm is essentially equivalent to the cluster norm, a multitask learning regularizer introduced by Jacob et al. (2009a), and which in turn can be interpreted as a perturbation of the spectral $k$-support norm. Centering the norm is important for multitask learning and we also provide a method to use centered versions of the norms as regularizers. Numerical experiments indicate that the spectral $k$-support and box-norms and their centered variants provide state of the art performance in matrix completion and multitask learning problems respectively.