Introduction
Think of two ways to stay up to date with the news:
- Once a year, you sit down and read every newspaper from the past year in one go. In between, your knowledge doesn't change.
- Every morning, you read today's paper and update what you know, a little at a time.
Machine learning systems face the same choice. They can learn from all the data at once (batch learning) or continuously, a bit at a time as new data arrive (online learning). The choice affects cost, speed, and how quickly the system notices that the world has changed.