Introduction
Decision trees aren't only for categories — they can predict numbers too. And now that we know how trees work, it's time for an honest look at their limitations:
- they can only make cuts parallel to the axes, which is awkward for rotated or diagonal patterns;
- they're unstable: small changes in the data can produce a very different tree.
These weaknesses are exactly what ensembles (next chapter) fix — so understanding them explains why random forests and gradient boosting are so popular.