Introduction
Imagine a company where, every morning, a random half of the employees are told to stay home. Nobody can rely on one specific colleague always being there, so everyone has to learn to do a bit of everything and to cooperate with whoever turns up. The company becomes more robust: losing any one person doesn't break it.
Dropout (Hinton et al., 2012; Srivastava et al., 2014) does exactly this to neurons. At every training step, each neuron in a layer is randomly switched off with probability . The network can't rely on any single neuron or on fragile combinations of neurons, so it learns more general, redundant features. It's one of the most popular and effective regularizers in deep learning.