Theory of Intelligence
DEEP LEARNING•Training Deep Networks•Transfer Learning And Fine Tuning•5.1 Transfer Learning & Fine-Tuning

Introduction

Reusing a network trained on a big dataset for your own task: freezing and unfreezing layers, discriminative learning rates, and when fine-tuning pays off.

timer1 min read•format_list_bulletedConcept 1 of 4•forumLesson Discussion

Introduction

Training a large network from scratch needs huge datasets and lots of compute. Most real projects have neither: maybe a few thousand labelled images or sentences. Transfer learning solves this by starting from a network that has already been trained on a large, general dataset (like ImageNet's 1.2 million images, or billions of words of text), and adapting it to your task.

Think of hiring a chef. A chef trained in a big restaurant already knows how to chop, season and cook. To work in your small café, they don't need to relearn cooking from zero; they only need to learn your menu. A pretrained network already knows how to "see" (edges, textures, shapes) or "read" (grammar, word meanings); you only teach it your specific task.

This is one of the most important practical techniques in deep learning: it gets better results with less data, in less time.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!
Next Lessonarrow_forward
5.2 Self-Supervised Pretraining