Introduction
A network has millions of parameters, and they don't all need the same step size. Some weights see large, frequent gradients (e.g. connected to common features); others see tiny or rare gradients (e.g. connected to rare words). One single learning rate is too big for the first group and too small for the second.
Adaptive optimizers give every parameter its own effective learning rate, adjusted automatically based on the history of its gradients. Think of a team of hikers walking down a mountain in fog. Each one is on different terrain. Those on steep, rocky slopes take small careful steps; those on gentle grass take long strides. Each adapts their own pace instead of marching to one shared rhythm.