Introduction

Introduction

Bagging trains many models independently, in parallel, and averages them. Boosting takes a different approach: it trains models one after another, and each new model tries to fix the mistakes of the ones before it.

Think of a student preparing for an exam with a stack of practice questions:

1. They attempt every question.
2. They mark which ones they got wrong.
3. Next round, they spend extra time on those questions.
4. Repeat — each round focuses more on what's still difficult.

That's boosting. The two most popular methods are AdaBoost (this topic) and gradient boosting (next topic).