Introduction
Imagine the heights of all the people in a large school: children in one grade cluster around one height, teenagers around another, adults around a third. Each group forms a bell curve, and the overall histogram is a mixture of those bells.
A Gaussian mixture model (GMM) assumes the data were generated exactly like that: from several Gaussian (bell-shaped) clusters, each with its own centre, shape, size and orientation. Compared with k-means, a GMM:
- handles elliptical clusters of different sizes and orientations;
- gives soft memberships: "this point is 80% cluster A, 20% cluster B";
- is a probabilistic model of the data, so it can estimate the density at any point — which makes it useful for anomaly detection (topic 3.3) and even generating new data.