Introduction
Overfitting usually doesn't happen all at once. During training, the validation error first goes down with the training error. Then, at some point, it levels off and starts rising while the training error keeps falling: the network has started memorizing noise.
Early stopping simply stops training at that turning point and keeps the best model seen. It's so simple and effective that Geoffrey Hinton called it a "beautiful free lunch". Think of baking bread: you don't follow the recipe's time blindly; you check the bread and take it out when it's golden, before it burns.
Max-norm is a different, gentler regularizer: rather than penalizing large weights, it puts a hard cap on how large each neuron's incoming weights can get.