Introduction
When you bake a cake, some things are decided by you before baking — oven temperature, baking time, tin size — and some things happen during baking — how the batter rises and browns. You can't control the rising directly, but your choices beforehand strongly affect it.
Machine learning has the same split:
- parameters are learned during training (the "rising");
- hyperparameters are chosen by you before training (the "oven settings").
This topic explains how to choose hyperparameters and models fairly, what to do when your training data don't look like real-world data, and why no single algorithm is always the best.