Introduction
A classic image classifier can only choose between the fixed list of classes it was trained on. To add a new class ("electric scooter"), you have to collect labelled images and retrain. Humans don't work like that: if someone describes an electric scooter to you in words, you can recognize one the first time you see it.
Vision-language models connect images and natural language. They're trained on huge numbers of images paired with the text that naturally accompanies them on the web (captions, alt text). The result: models that can match images to any text description, classify images into categories they were never explicitly trained on, and describe images in words.
Think of a museum guide who has read millions of exhibit labels next to their pictures. Show them a new painting and a few possible descriptions, and they can tell you which description fits best, or write a new label themselves.