Token Budgets
Every LLM request is paid for, limited and slowed down by tokens. A token budget decides in advance how many tokens each part of a request may use. Then requests fit the context window, answers are not cut off, and the bill stays predictable.
New words
- Token
- A small piece of text that the model reads and writes.
- Context window
- The most tokens the model can handle in one request.
- Token budget
- A plan for how many tokens each part of a request may use.
- max_tokens
- A setting that caps how long the model's answer can be.
- Prefill and decode
- Prefill reads the input at once. Decode writes the answer token by token.
- Prompt caching
- Reusing an unchanged start of the prompt at a lower price.
What you will learn
- The problem: requests quietly grow
- What a token is
- Where the tokens in a request come from
- Try it: plan a token budget
- Agent loops: the cost curve bends upward
- Trade-offs and failure modes
- Production practices
- In code
- In a real system
- Interview tip
- Key takeaways
10 quiz questions at the end.
The full lesson, with interactive demos and the quiz, opens after you sign in with Google. It is free.