Introduction

Introduction

The SVD writes a matrix as a sum of rank-1 layers ordered by importance ($\sigma_1 \ge \sigma_2 \ge \dots$). Keep only the first few layers and you get a much smaller approximation that still captures most of the matrix. This is the idea behind image compression, denoising, recommender systems, PCA, and LoRA fine-tuning of large language models.