Introduction
Plain gradient descent is like a careful hiker in thick fog. At every step they feel the slope under their feet and take one step downhill. They have no memory: each step depends only on the ground right now. On a long, gentle slope they plod slowly; in a narrow valley they bounce from one wall to the other.
Momentum turns the hiker into a ball rolling downhill. The ball builds up speed in directions where the slope keeps pointing the same way, and the back-and-forth pushes from the valley walls partly cancel out. It gets to the bottom much faster.