Theory of Intelligence
MACHINE LEARNING•Classic Algorithms•Decision Trees•2.2 Predictions, Gini & Entropy

Introduction

How a tree makes predictions, how Gini impurity and entropy measure how mixed a node is, and how trees estimate class probabilities.

Introduction

A tree's goal is to split the data into groups that are as "pure" as possible — each leaf containing (ideally) only one class. To do that, it needs a way to measure how mixed a group is. The two standard measures are Gini impurity and entropy.

Think of sorting a bag of mixed sweets into bowls. A bowl with only red sweets is perfectly sorted (pure). A bowl with an even mix of red, green and yellow is as unsorted as it gets. Gini and entropy put a number on "how unsorted" a bowl is.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!