Introduction
ImageNet's 1.2 million labelled images took years of human effort to annotate. Meanwhile, billions of unlabelled images exist. Self-supervised vision methods learn from those unlabelled images by inventing tasks whose answers come from the images themselves. The resulting backbones are then fine-tuned (or used as-is) for classification, detection, segmentation, retrieval and more, often beating supervised pretraining.
The Training tab (topic 4.2) introduced the general ideas. This topic looks at three landmark methods for images in more detail:
- SimCLR: learn that two different crops of the same photo belong together;
- MAE: hide most of an image and learn to rebuild it;
- DINO: a student network learns to match a teacher network's view of the image.
Think of a child exploring the world before learning any words. They see a dog from the front, from the side, half-hidden behind a sofa, and gradually learn that these are all the same kind of thing, long before anyone tells them it's called "a dog".