Theory of Intelligence
DEEP LEARNING•Reinforcement And Frontier•Deep Reinforcement Learning•1.3 Markov Decision Processes

Introduction

The mathematical framework behind RL: states, actions, transition probabilities and rewards, the Bellman equations that define how good each state and action is, and solving small problems with value iteration.

timer1 min read•format_list_bulletedConcept 1 of 4•forumLesson Discussion

Introduction

Policy gradients (topic 1.2) improve a policy directly. Another family of methods takes a different route: first work out how good each situation and each action is, then simply pick the best action. To do that, we need a precise description of the problem. That's a Markov decision process (MDP).

Think of planning a road trip with a map that shows, for every city, which roads you can take, where each road might lead (sometimes a road is closed and you end up somewhere else), and how pleasant or costly each trip is. If you know all that, you can work out the best route from every city. An MDP is that map; the Bellman equations are the rules for working out the best routes.

Discuss theoretical concepts, mathematical formulas, and architectural foundations.

Supports inline code with `backticks`
No comments yet. Be the first to start the discussion!
arrow_backPrevious Lesson
1.2 Policy Gradients
Next Lessonarrow_forward
1.4 Q-Learning & DQN