Theory of Intelligence
MACHINE LEARNING•Supervised Learning•Logistic And Softmax Regression•5.4 Softmax Regression

Introduction

Extend logistic regression to many classes: one score per class, softmax turns the scores into probabilities, and cross-entropy trains the model.

Introduction

Logistic regression answers yes/no. But often there are many classes: which of three iris species, which of ten digits, which of 50,000 words comes next in a sentence.

Softmax regression (also called multinomial logistic regression) generalises logistic regression to many classes directly — no need for one-vs-rest tricks. The same idea is the output layer of almost every neural-network classifier, including the language models that choose the next word. So understanding softmax pays off far beyond this topic.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!