Theory of Intelligence
MATH FOR AI•Calculus•Multivariate Calculus•2.1 Partial Derivatives & Gradients

Introduction

Partial derivatives of functions of many variables, the gradient vector, and directional derivatives.

Introduction

A model's loss depends on many parameters at once — millions of them. To improve it, we need to know how the loss changes when each parameter moves. Partial derivatives answer this one variable at a time, and the gradient collects them into a single vector that points uphill. Gradient descent simply walks in the opposite direction.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!