Introduction
Which is easier to memorize?
- (a) 40, 27, 82, 41, 124, 62, 31, 94, 47, 142
- (b) 50, 48, 46, 44, 42, 40, 38, 36, 34, 32
Sequence (b) looks just as long, but you only need to remember a rule: "start at 50 and count down by 2". You've compressed it by finding its structure.
An autoencoder is a neural network that learns to do exactly this. It's trained to copy its input to its output, but through a narrow bottleneck that makes simple copying impossible. To succeed, it must discover the patterns in the data and store a compact code from which the input can be rebuilt.
Autoencoders are the first family of generative models in this tab. They're used for dimensionality reduction, feature learning, denoising, anomaly detection and, in their variational form (topic 1.4), for generating brand-new data.