Theory of Intelligence
MACHINE LEARNING•Foundations•An End To End Ml Project•4.2 Get the Data & Create a Test Set

Introduction

Load the dataset, take a first look at its structure, and set aside a fair test set with random or stratified sampling — before you get tempted to peek.

Introduction

With the problem framed, it's time to get the data. In a company, this might mean finding the right database, asking for access, and writing a script that downloads a fresh copy whenever you need it. Here we'll use the California housing dataset that comes with scikit-learn, so you can follow along on any computer.

This topic has one surprisingly important lesson: set aside a test set almost immediately, before you look at the data in detail. The reason is a sneaky problem called data snooping, which we'll explain below.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!