Introduction
Neural-network parameters are matrices (weight matrices) and the data come in batches (also matrices). So we need derivatives with respect to matrices. The mathematics is the same partial derivatives as before — the challenge is bookkeeping the shapes. This topic gives a reliable way to do that and works through the most important case: a linear layer.