Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo

Markus Heinonen, Henrik Mannerström, Juho Rousu, Samuel Kaski, Harri Lähdesmäki
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, PMLR 51:732-740, 2016.

Abstract

We present a novel approach for non-stationary Gaussian process regression (GPR), where the three key parameters – noise variance, signal variance and lengthscale – can be simultaneously input-dependent. We develop gradient-based inference methods to learn the unknown function and the non-stationary model parameters, without requiring any model approximations. For inferring the full posterior distribution we use Hamiltonian Monte Carlo (HMC), which conveniently extends the analytical gradient-based GPR learning by guiding the sampling with the gradients. The MAP solution can also be learned with gradient ascent. In experiments on several synthetic datasets and in modelling of temporal gene expression, the non-stationary GPR is shown to give major improvement when modeling realistic input-dependent dynamics.

Cite this Paper

BibTeX
@InProceedings{pmlr-v51-heinonen16, title = {Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo}, author = {Heinonen, Markus and Mannerström, Henrik and Rousu, Juho and Kaski, Samuel and Lähdesmäki, Harri}, booktitle = {Proceedings of the 19th International Conference on Artificial Intelligence and Statistics}, pages = {732--740}, year = {2016}, editor = {Gretton, Arthur and Robert, Christian C.}, volume = {51}, series = {Proceedings of Machine Learning Research}, address = {Cadiz, Spain}, month = {09--11 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v51/heinonen16.pdf}, url = {https://proceedings.mlr.press/v51/heinonen16.html}, abstract = {We present a novel approach for non-stationary Gaussian process regression (GPR), where the three key parameters – noise variance, signal variance and lengthscale – can be simultaneously input-dependent. We develop gradient-based inference methods to learn the unknown function and the non-stationary model parameters, without requiring any model approximations. For inferring the full posterior distribution we use Hamiltonian Monte Carlo (HMC), which conveniently extends the analytical gradient-based GPR learning by guiding the sampling with the gradients. The MAP solution can also be learned with gradient ascent. In experiments on several synthetic datasets and in modelling of temporal gene expression, the non-stationary GPR is shown to give major improvement when modeling realistic input-dependent dynamics.} }
Endnote
%0 Conference Paper %T Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo %A Markus Heinonen %A Henrik Mannerström %A Juho Rousu %A Samuel Kaski %A Harri Lähdesmäki %B Proceedings of the 19th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2016 %E Arthur Gretton %E Christian C. Robert %F pmlr-v51-heinonen16 %I PMLR %P 732--740 %U https://proceedings.mlr.press/v51/heinonen16.html %V 51 %X We present a novel approach for non-stationary Gaussian process regression (GPR), where the three key parameters – noise variance, signal variance and lengthscale – can be simultaneously input-dependent. We develop gradient-based inference methods to learn the unknown function and the non-stationary model parameters, without requiring any model approximations. For inferring the full posterior distribution we use Hamiltonian Monte Carlo (HMC), which conveniently extends the analytical gradient-based GPR learning by guiding the sampling with the gradients. The MAP solution can also be learned with gradient ascent. In experiments on several synthetic datasets and in modelling of temporal gene expression, the non-stationary GPR is shown to give major improvement when modeling realistic input-dependent dynamics.
RIS
TY - CPAPER TI - Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo AU - Markus Heinonen AU - Henrik Mannerström AU - Juho Rousu AU - Samuel Kaski AU - Harri Lähdesmäki BT - Proceedings of the 19th International Conference on Artificial Intelligence and Statistics DA - 2016/05/02 ED - Arthur Gretton ED - Christian C. Robert ID - pmlr-v51-heinonen16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 51 SP - 732 EP - 740 L1 - http://proceedings.mlr.press/v51/heinonen16.pdf UR - https://proceedings.mlr.press/v51/heinonen16.html AB - We present a novel approach for non-stationary Gaussian process regression (GPR), where the three key parameters – noise variance, signal variance and lengthscale – can be simultaneously input-dependent. We develop gradient-based inference methods to learn the unknown function and the non-stationary model parameters, without requiring any model approximations. For inferring the full posterior distribution we use Hamiltonian Monte Carlo (HMC), which conveniently extends the analytical gradient-based GPR learning by guiding the sampling with the gradients. The MAP solution can also be learned with gradient ascent. In experiments on several synthetic datasets and in modelling of temporal gene expression, the non-stationary GPR is shown to give major improvement when modeling realistic input-dependent dynamics. ER -
APA
Heinonen, M., Mannerström, H., Rousu, J., Kaski, S. & Lähdesmäki, H.. (2016). Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo. Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 51:732-740 Available from https://proceedings.mlr.press/v51/heinonen16.html.

Related Material