Introduction
A neural network has many hyperparameters: learning rate, batch size, number of layers, neurons per layer, dropout, weight decay, optimiser, schedule… Each training run can take minutes to days. You can't try everything, so you need to tune smartly.
Think of tuning a car before a race. You don't randomly adjust every screw. You start with the settings that matter most (tyre pressure, gear ratios), test, and adjust — with a stopwatch to compare fairly.