Introduction
Before training starts, every weight in the network needs a starting value. It's tempting to think this hardly matters, since training will move the weights anyway. In a deep network it matters enormously. Bad starting weights can make the signal fade away or blow up before training even gets going (topic 1.1), and then training never recovers.
Think of tuning a long chain of amplifiers before a concert. If every amplifier is set slightly too low, the music is silent at the end of the chain; slightly too high, and it's distorted noise. You want each one set so the volume coming out matches the volume going in. Good initialization does that for each layer.