Transformers
The transformer is the design behind ChatGPT, Claude, Gemini and most other large language models. In this lesson you follow one sentence through a transformer, step by step. You also learn why it replaced older designs.
New words
- Transformer
- A neural network design built on attention, made in 2017.
- Embedding
- A vector, a list of numbers, that stands for one token.
- Block
- One layer of a transformer: attention plus a small neural network.
- Decoder-only
- A model that reads text and writes the next token, like GPT.
- RNN
- An older network that reads one word at a time, in order.
- Parameters
- The numbers inside a model that it learns during training.
What you will learn
- Why the old design was slow
- The 2017 idea, and the path of a sentence
- What is inside one block?
- Three kinds of transformers
- How big is a transformer?
- Try it: follow "The chai is ___"
- Trade-offs
- In a real system
- Interview tip
- Key takeaways
10 quiz questions at the end.
The full lesson, with interactive demos and the quiz, opens after you sign in with Google. It is free.