Introduction
Value iteration (topic 1.3) needs a complete map of the environment: every transition probability and reward. Real agents usually don't have that. A robot doesn't know the physics equations of its motors; a game-playing agent doesn't know the game's code. They only have experience: "I was here, I did this, I got that reward, and I ended up there."
Q-learning learns Q-values directly from such experience, one step at a time. Deep Q-networks (DQN) replace the table of Q-values with a neural network, so the same idea works for huge state spaces like video-game screens. In 2015, DeepMind's DQN learned to play dozens of Atari games from raw pixels, many at or above human level: a landmark for deep RL.
Think of learning a new city's restaurants. You don't have a guidebook (no model). Each evening you try a place, note how good it was, and update your opinion, also taking into account how good the neighbourhood's other options are. Over time your opinions become reliable, and you mostly go to the best ones while occasionally trying something new.