Introduction
After tokenization, each token is just an integer ID: "cat" might be 2,341 and "dog" 5,872. Those numbers mean nothing by themselves: 2,341 isn't "closer" to 5,872 than to anything else. The network needs a representation where similar words look similar.
An embedding gives every token a dense vector of, say, 300 numbers. These vectors are learned so that words used in similar ways end up near each other: "cat" near "dog", "Paris" near "London", "happy" near "joyful". Directions in the space can even capture relationships, like male → female or country → capital.
Think of a map of a city. Each shop is placed at a location, and similar shops cluster together: the restaurants in one area, the clothing stores in another. An embedding is a map of words (in many dimensions instead of two), where distance means similarity of meaning.