Introduction
Think of the game "20 Questions": one person thinks of an animal, and the other asks yes/no questions — "Does it live in water?", "Is it bigger than a cat?" — narrowing down the answer with each question.
A decision tree is a model that plays this game with data. It learns a sequence of yes/no questions about the features — "Is the petal length ≤ 2.45 cm?" — that leads to a prediction.
Decision trees are:
- easy to understand — you can draw the model and follow it with your finger;
- versatile — they handle classification, regression and multi-output tasks;
- low-maintenance — no feature scaling needed;
- the building block of random forests and gradient boosting, some of the most powerful algorithms for tabular data (Chapter 03).