Introduction
The original transformer (topic 4.4) had two halves: an encoder and a decoder. Soon researchers found that you can use just one half for many tasks. This produced three families of models:
- encoder-only (like BERT): read a whole text and understand it. Great for classification, search and extracting information;
- decoder-only (like GPT): generate text one token at a time, left to right. The basis of today's chatbots and large language models;
- encoder–decoder (like T5): read an input and write a different output. Great for translation and summarization.
Think of three kinds of jobs in a newsroom: an editor who reads articles carefully and judges them (encoder), a writer who produces text word by word (decoder), and a translator who reads a full article in one language and rewrites it in another (encoder–decoder).