Introduction
In the last chapter we saw that flexible models — like high-degree polynomials — tend to overfit: their weights become huge, positive and negative, so the curve can wiggle through every training point.
Regularization is a way to say to the model: "Fit the data, but keep things simple." It's like a teacher who gives marks for the right answer and deducts marks for an unnecessarily complicated solution. The model is pushed to find answers that work well without extreme weights.
This chapter covers the three classic regularized linear models — Ridge, Lasso and Elastic Net — plus early stopping. We start with Ridge.