Theory of Intelligence
MACHINE LEARNING•Classic Algorithms•Ensemble Learning•3.2 Bagging & Pasting

Introduction

Create diverse models from one algorithm by training each on a different random sample of the data, and get a free validation score from the out-of-bag examples.

Introduction

The last topic said ensembles need diverse models. One way is to use different algorithms. Another, often more powerful way: use the same algorithm many times, but give each copy a different random sample of the training data. Each model sees a slightly different picture of the world, so each makes slightly different mistakes — and averaging cancels them out.

It's like asking ten students to each study a different random selection of past exam papers, then combining their answers. Each student has gaps, but they're in different places.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!