Introduction
A regular autoencoder's decoder can turn codes into images, so why not generate new images by feeding it random codes? Try it and you mostly get garbage. The encoder places training examples at scattered points in the latent space with big empty gaps between them, and the decoder has never learned what to do in those gaps.
The variational autoencoder (VAE) (Kingma and Welling, 2013) fixes this. It encodes each input not as a single point but as a small cloud (a probability distribution), and it pushes all those clouds to overlap and fill a simple, well-known region, a standard normal distribution. The latent space becomes smooth and continuous: every point decodes into something plausible, and nearby points decode into similar things. Then generating new data is easy: sample a random point and decode it.
Think of a map where every city is not a dot but a blurred patch, and all patches are gently pulled towards the centre of the map until they overlap. Now there are no blank regions: wherever you drop a pin, you're in or between familiar places.