Introduction
Self-attention (topic 4.2) has a surprising blind spot: it doesn't know the order of the tokens. Each token attends to all others based only on their content, and the result is the same whether the sentence is "the dog bit the man" or "the man bit the dog". Mathematically, if you shuffle the input tokens, the outputs are just shuffled the same way.
RNNs never had this problem, because they read tokens one after another. Transformers process all tokens at once, so position information must be added explicitly.
Think of a stack of loose pages from a book that someone has dropped. You can read every page, but to understand the story you need the page numbers. Positional encodings are the page numbers stamped onto each token.