Introduction
After ResNet, the race shifted from "the most accurate network at any cost" to "the best accuracy for a given budget". A phone, a drone or a factory camera can't run a 60-million-parameter network in real time. Even in data centres, cheaper models mean lower costs and faster responses.
This topic covers three big ideas:
- depthwise separable convolutions: a cheaper way to do convolution;
- EfficientNet: a principled way to scale a network up or down;
- ConvNeXt: an updated CNN design that borrowed ideas from vision transformers and matched them.
Think of car design. Early on, makers chased raw speed with bigger engines. Later, the real wins came from efficiency: lighter materials, better aerodynamics, smarter engines. These architectures are the "fuel-efficient" generation of CNNs.