Introduction
Imagine two students preparing for a maths exam:
- Student A memorises the exact answers to every practice question. On the practice test they score 100%. On the real exam, with slightly different numbers, they are lost.
- Student B barely studies and only remembers "the answer is usually around 10." They do badly on both the practice test and the real exam.
Student A overfits: they learned the practice questions too well, including their accidental details, instead of the general method. Student B underfits: they didn't learn enough to capture the real pattern.
The goal of ML is to be like Student C, who learns the general method and does well on questions they've never seen. This topic explains how to recognise and fix both problems.