Introduction
The "widest street" idea from the last topic assumed the classes could be separated perfectly — every point on the correct side, none in the street. That's called hard margin classification, and it has two serious problems:
- It only works if the data are perfectly linearly separable. One mislabelled or unusual point can make a perfect separation impossible.
- It's very sensitive to outliers. A single strange point close to the other class forces the street to become very narrow, which hurts generalisation.
Real data are never perfectly clean. So practical SVMs use soft margin classification: find a good balance between a wide street and few violations.