Theory of Intelligence
DEEP LEARNING•Reinforcement And Frontier•Frontier Architectures•2.4 Neural Architecture Search

Introduction

Letting algorithms design networks: defining a search space of possible architectures, search strategies from reinforcement learning to evolution, and efficient methods like weight sharing and DARTS.

timer1 min read•format_list_bulletedConcept 1 of 3•forumLesson Discussion

Introduction

Every architecture in this course, from ResNet to the transformer, was designed by people: trying ideas, running experiments, keeping what worked. Neural architecture search (NAS) automates part of this: an algorithm explores many possible architectures and finds ones that perform well, sometimes better than human designs.

Several well-known models were found or refined this way: NASNet, AmoebaNet, EfficientNet's baseline (B0) and MobileNetV3 (Computer Vision tab, topic 2.4).

Think of designing a new recipe. A chef can taste and adjust by intuition. Alternatively, a kitchen robot could systematically try thousands of ingredient combinations and cooking times, keeping the best-rated dishes. NAS is the robot. The challenges are the same: what combinations are allowed (the search space), how to choose what to try next (the search strategy), and how to rate each dish quickly (performance estimation).

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!
arrow_backPrevious Lesson
2.3 State Space Models