Theory of Intelligence
DEEP LEARNING•Generative Deep Learning•Generative Adversarial Networks•2.2 Training Difficulties

Introduction

Why GANs are notoriously hard to train: mode collapse, oscillation and instability, and the tricks that help — Wasserstein loss, gradient penalties, spectral normalization and more.

Introduction

On paper, the GAN game ends in a perfect equilibrium. In practice, GAN training is famous for being fragile. Runs can collapse into producing the same image over and over, oscillate without ever settling, or blow up after looking fine for hours. A lot of GAN research has been about making training stable.

Think of two people learning to play tennis by playing only against each other. If one becomes much better too fast, the other never wins a point and can't learn. If they keep exploiting each other's latest weakness, they may circle around forever without either improving overall. GANs face exactly these problems.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!