Introduction
In the previous topic, a decoder attended to an encoder's outputs. Self-attention applies the same idea within a single sequence: every token attends to every other token in the same sentence, to work out what it means in context.
Consider: "The animal didn't cross the street because it was too tired." What does "it" refer to? To understand "it", a model should look at "animal". Change "tired" to "wide" and "it" now refers to "street". Self-attention lets the representation of "it" pull in information from whichever words are relevant.
Think of a group discussion where everyone can hear everyone. Each person (token) listens to all the others, pays more attention to the most relevant speakers, and updates their own understanding. After a few rounds of this (layers), each person's view reflects the whole conversation.