Fast Variational Bayesian Inference for Non-Conjugate Matrix Factorization Models

Matthias Seeger, Guillaume Bouchard ; JMLR W&CP 22: 1012-1018, 2012.


Probabilistic matrix factorization methods aim to extract meaningful correlation structure from an incomplete data matrix by postulating low rank constraints. Recently, variational Bayesian (VB) inference techniques have successfully been applied to such large scale bilinear models. However, current algorithms are of the alternate updating or stochastic gradient descent type, slow to converge and prone to getting stuck in shallow local minima. While for MAP or maximum margin estimation, singular value shrinkage algorithms have been proposed which can far outperform alternate updating, this methodological avenue remains unexplored for Bayesian techniques. In this paper, we show how to combine a recent singular value shrinkage characterization of fully observed spherical Gaussian VB matrix factorization with augmented Lagrangian techniques in order to obtain efficient VB inference for general MF models with arbitrary likelihood potentials. In particular, we show how to handle Poisson and Bernoulli potentials, far more suited for most MF applications than Gaussian likelihoods. Our algorithm can be run even for very large models and is easily implemented in {\em Matlab}. It outperforms MAP estimation on a range of real-world datasets.

Home Page





Editorial Board



Open Source Software



RSS Feed

Page last modified on Thu April 26 2012 13:56 2012.

Copyright @ JMLR 2012. All rights reserved.