Introduction
A bottleneck (topic 1.1) is one way to stop an autoencoder from learning the trivial "copy" solution. There are others. This topic covers two:
- denoising autoencoders receive a corrupted input but must reconstruct the clean original. Copying would only reproduce the noise, so the network has to learn what clean data looks like;
- sparse autoencoders may have a large code, but are penalized unless most code units are near zero most of the time. Each unit must then specialize in a distinct feature.
Then we'll see one of the most practical uses of autoencoders: anomaly detection.
Think of a restorer of old photographs. They're given a scratched, faded print and must produce what the original looked like. To do this well, they must know what faces, clothes and buildings normally look like. A denoising autoencoder learns that knowledge from data.