Theory of Intelligence
DEEP LEARNING•Training Deep Networks•Regularization For Deep Nets•3.3 Data Augmentation

Introduction

Creating realistic new training examples from existing ones — image, text and audio augmentation, plus Mixup and CutMix — to reduce overfitting.

Introduction

The best cure for overfitting is more data. But collecting and labelling data is slow and expensive. Data augmentation creates new training examples by applying realistic changes to the ones you have: flip a photo of a cat, crop it, make it a little darker. It's still clearly a cat, so the label stays the same, but to the network it's a new image.

Think of learning to recognise a friend's face. You've seen them in sunlight and shade, from the left and the right, with and without glasses. Because you've seen so many variations, you'd recognise them anywhere. Augmentation gives the network that same variety, so it learns what really matters (the shape of a cat) and ignores what doesn't (where the cat sits in the frame, or the lighting).

The golden rule: an augmentation must produce examples that could realistically occur and that keep the label correct.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!