Introduction
Correlation only detects linear relationships. Mutual information measures any kind of dependence: how much knowing one variable reduces uncertainty about another. It is used to pick informative features, to build decision trees, and as the theoretical motivation behind contrastive learning methods like CLIP and SimCLR.