Home Page

Papers

Submissions

News

Editorial Board

Open Source Software

Proceedings (PMLR)

Transactions (TMLR)

Search

Statistics

Login

Frequently Asked Questions

Contact Us



RSS Feed

A Risk Comparison of Ordinary Least Squares vs Ridge Regression

Paramveer S. Dhillon, Dean P. Foster, Sham M. Kakade, Lyle H. Ungar; 14(10):1505−1511, 2013.

Abstract

We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a principal component analysis) and then performs an ordinary (un- regularized) least squares regression in this subspace. This note shows that the risk of this ordinary least squares method (PCA-OLS) is within a constant factor (namely 4) of the risk of ridge regression (RR).

[abs][pdf][bib]       
© JMLR 2013. (edit, beta)