Sequential Monte Carlo Samplers for Dirichlet Process Mixtures

Yener Ulker, Bilge Günsel, Taylan Cemgil
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:876-883, 2010.

Abstract

In this paper, we develop a novel online algorithm based on the Sequential Monte Carlo(SMC) samplers framework for posterior inference in Dirichlet Process Mixtures (DPM). Our method generalizes many sequential importance sampling approaches. It provides a computationally efficient improvement to particle filtering that is less prone to getting stuck in isolated modes. The proposed method is a particular SMC sampler that enables us to design sophisticated clustering update schemes, such as updating past trajectories of the particles in light of recent observations, and still ensures convergence to the true DPM target distribution asymptotically. Performance has been evaluated in a Bayesian Infinite Gaussian mixture density estimation problem and it is shown that the proposed algorithm outperforms conventional Monte Carlo approaches in terms of estimation variance and average log-marginal likelihood.

Cite this Paper

BibTeX
@InProceedings{pmlr-v9-ulker10a, title = {Sequential Monte Carlo Samplers for Dirichlet Process Mixtures}, author = {Ulker, Yener and Günsel, Bilge and Cemgil, Taylan}, booktitle = {Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics}, pages = {876--883}, year = {2010}, editor = {Teh, Yee Whye and Titterington, Mike}, volume = {9}, series = {Proceedings of Machine Learning Research}, address = {Chia Laguna Resort, Sardinia, Italy}, month = {13--15 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v9/ulker10a/ulker10a.pdf}, url = {https://proceedings.mlr.press/v9/ulker10a.html}, abstract = {In this paper, we develop a novel online algorithm based on the Sequential Monte Carlo(SMC) samplers framework for posterior inference in Dirichlet Process Mixtures (DPM). Our method generalizes many sequential importance sampling approaches. It provides a computationally efficient improvement to particle filtering that is less prone to getting stuck in isolated modes. The proposed method is a particular SMC sampler that enables us to design sophisticated clustering update schemes, such as updating past trajectories of the particles in light of recent observations, and still ensures convergence to the true DPM target distribution asymptotically. Performance has been evaluated in a Bayesian Infinite Gaussian mixture density estimation problem and it is shown that the proposed algorithm outperforms conventional Monte Carlo approaches in terms of estimation variance and average log-marginal likelihood.} }
Endnote
%0 Conference Paper %T Sequential Monte Carlo Samplers for Dirichlet Process Mixtures %A Yener Ulker %A Bilge Günsel %A Taylan Cemgil %B Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2010 %E Yee Whye Teh %E Mike Titterington %F pmlr-v9-ulker10a %I PMLR %P 876--883 %U https://proceedings.mlr.press/v9/ulker10a.html %V 9 %X In this paper, we develop a novel online algorithm based on the Sequential Monte Carlo(SMC) samplers framework for posterior inference in Dirichlet Process Mixtures (DPM). Our method generalizes many sequential importance sampling approaches. It provides a computationally efficient improvement to particle filtering that is less prone to getting stuck in isolated modes. The proposed method is a particular SMC sampler that enables us to design sophisticated clustering update schemes, such as updating past trajectories of the particles in light of recent observations, and still ensures convergence to the true DPM target distribution asymptotically. Performance has been evaluated in a Bayesian Infinite Gaussian mixture density estimation problem and it is shown that the proposed algorithm outperforms conventional Monte Carlo approaches in terms of estimation variance and average log-marginal likelihood.
RIS
TY - CPAPER TI - Sequential Monte Carlo Samplers for Dirichlet Process Mixtures AU - Yener Ulker AU - Bilge Günsel AU - Taylan Cemgil BT - Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics DA - 2010/03/31 ED - Yee Whye Teh ED - Mike Titterington ID - pmlr-v9-ulker10a PB - PMLR DP - Proceedings of Machine Learning Research VL - 9 SP - 876 EP - 883 L1 - http://proceedings.mlr.press/v9/ulker10a/ulker10a.pdf UR - https://proceedings.mlr.press/v9/ulker10a.html AB - In this paper, we develop a novel online algorithm based on the Sequential Monte Carlo(SMC) samplers framework for posterior inference in Dirichlet Process Mixtures (DPM). Our method generalizes many sequential importance sampling approaches. It provides a computationally efficient improvement to particle filtering that is less prone to getting stuck in isolated modes. The proposed method is a particular SMC sampler that enables us to design sophisticated clustering update schemes, such as updating past trajectories of the particles in light of recent observations, and still ensures convergence to the true DPM target distribution asymptotically. Performance has been evaluated in a Bayesian Infinite Gaussian mixture density estimation problem and it is shown that the proposed algorithm outperforms conventional Monte Carlo approaches in terms of estimation variance and average log-marginal likelihood. ER -
APA
Ulker, Y., Günsel, B. & Cemgil, T.. (2010). Sequential Monte Carlo Samplers for Dirichlet Process Mixtures. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 9:876-883 Available from https://proceedings.mlr.press/v9/ulker10a.html.

Related Material