Introduction
Imagine dropping ink into a glass of water. Over time, it diffuses until the water is uniformly tinted: the original drop's shape is completely gone. Running that film backwards would show uniform tint gathering back into a crisp drop. In physics, that reverse never happens on its own. But what if a neural network could learn to run it backwards?
That's the idea of diffusion models, the technology behind Stable Diffusion, DALL·E 2 and 3, Imagen, Midjourney and many video and audio generators:
- a forward process gradually adds noise to an image over many steps until only pure noise remains. This needs no learning;
- a network learns the reverse process: given a noisy image, remove a little bit of noise. Repeating this many times turns pure random noise into a brand-new image.
Think of a sculptor who starts with a rough block of marble and removes a little at a time. Each cut is small and simple; after hundreds of cuts, a statue emerges. A diffusion model is a sculptor whose block is random noise and whose every cut is a small denoising step.