Introduction
Neural networks are flexible — maybe too flexible. You can choose any number of layers, any number of neurons in each, any activation, any optimiser… How do you pick sensible values without trying millions of combinations?
The honest answer is that there's no formula. But experience has produced good rules of thumb, and one especially useful mindset: start from an architecture that's known to work, make it somewhat bigger than you think you need, and control overfitting with regularization.