Introduction
Orthogonal means "perpendicular", generalised to any dimension and any inner product. Orthogonal directions don't interfere with each other: moving along one tells you nothing about the other. That makes orthogonal bases and orthogonal matrices the cleanest tools in numerical linear algebra — and the reason PCA's components are so easy to work with.