Theory of Intelligence
DEEP LEARNING•Neural Networks•Designing Networks•5.2 Learning Rate, Batch Size & Epochs

Introduction

The three training settings that matter most: how to find a good learning rate, the trade-offs of small and large batches, and how long to train.

timer1 min read•format_list_bulletedConcept 1 of 4•forumLesson Discussion

Introduction

Of all the knobs you can turn when training a network, three matter most:

  1. the learning rate — how big each update step is;
  2. the batch size — how many examples each update is based on;
  3. the number of epochs — how long you train.

Getting these right is often the difference between a network that learns beautifully and one that doesn't learn at all. The good news: there are reliable ways to set them.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!
arrow_backPrevious Lesson
5.1 Hidden Layers & Neurons
Next Lessonarrow_forward
5.3 Hyperparameter Tuning for Neural Networks