Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs

Yarin Gal, Richard Turner
Proceedings of the 32nd International Conference on Machine Learning, PMLR 37:655-664, 2015.

Abstract

Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance function with a finite Fourier series approximation and treat it as a random variable. The random covariance function has a posterior, on which a variational distribution is placed. The variational distribution transforms the random covariance function to fit the data. We study the properties of our approximate inference, compare it to alternative ones, and extend it to the distributed and stochastic domains. Our approximation captures complex functions better than standard approaches and avoids over-fitting.

Cite this Paper

BibTeX
@InProceedings{pmlr-v37-galb15, title = {Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs}, author = {Gal, Yarin and Turner, Richard}, booktitle = {Proceedings of the 32nd International Conference on Machine Learning}, pages = {655--664}, year = {2015}, editor = {Bach, Francis and Blei, David}, volume = {37}, series = {Proceedings of Machine Learning Research}, address = {Lille, France}, month = {07--09 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v37/galb15.pdf}, url = {https://proceedings.mlr.press/v37/galb15.html}, abstract = {Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance function with a finite Fourier series approximation and treat it as a random variable. The random covariance function has a posterior, on which a variational distribution is placed. The variational distribution transforms the random covariance function to fit the data. We study the properties of our approximate inference, compare it to alternative ones, and extend it to the distributed and stochastic domains. Our approximation captures complex functions better than standard approaches and avoids over-fitting.} }
Endnote
%0 Conference Paper %T Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs %A Yarin Gal %A Richard Turner %B Proceedings of the 32nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2015 %E Francis Bach %E David Blei %F pmlr-v37-galb15 %I PMLR %P 655--664 %U https://proceedings.mlr.press/v37/galb15.html %V 37 %X Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance function with a finite Fourier series approximation and treat it as a random variable. The random covariance function has a posterior, on which a variational distribution is placed. The variational distribution transforms the random covariance function to fit the data. We study the properties of our approximate inference, compare it to alternative ones, and extend it to the distributed and stochastic domains. Our approximation captures complex functions better than standard approaches and avoids over-fitting.
RIS
TY - CPAPER TI - Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs AU - Yarin Gal AU - Richard Turner BT - Proceedings of the 32nd International Conference on Machine Learning DA - 2015/06/01 ED - Francis Bach ED - David Blei ID - pmlr-v37-galb15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 37 SP - 655 EP - 664 L1 - http://proceedings.mlr.press/v37/galb15.pdf UR - https://proceedings.mlr.press/v37/galb15.html AB - Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for the sparse spectrum approximation to avoid both issues. We model the covariance function with a finite Fourier series approximation and treat it as a random variable. The random covariance function has a posterior, on which a variational distribution is placed. The variational distribution transforms the random covariance function to fit the data. We study the properties of our approximate inference, compare it to alternative ones, and extend it to the distributed and stochastic domains. Our approximation captures complex functions better than standard approaches and avoids over-fitting. ER -
APA
Gal, Y. & Turner, R.. (2015). Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputs. Proceedings of the 32nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 37:655-664 Available from https://proceedings.mlr.press/v37/galb15.html.

Related Material