Introduction
Ask one person to guess the number of sweets in a jar and they'll probably be quite far off. Ask a thousand people and average their guesses, and the result is often amazingly close — closer than almost every individual guess. This is the wisdom of the crowd.
Ensemble learning applies the same idea to ML: combine the predictions of several models (an ensemble) to get a better prediction than any single one. Ensembles regularly win machine learning competitions, and random forests and gradient boosting — the stars of this chapter — are ensembles of decision trees.
We start with the simplest ensemble: letting different models vote.