Chunking Documents
A model cannot search a 200-page PDF as one block. So we cut it into small pieces called chunks. How you cut decides if the right answer can be found later.
New words
- Chunk
- One small piece of a long document, saved and searched on its own.
- Token
- A small piece of text the model reads. About 4 tokens for 3 English words.
- Overlap
- Text that is repeated at the end of one chunk and the start of the next.
- Metadata
- Extra facts saved with a chunk, such as its source, page and heading.
- Retrieval
- Searching the saved chunks to find the ones that match a question.
- Embedding
- A list of numbers that stands for the meaning of a text.
What you will learn
- Why not search the whole document?
- How big should a chunk be?
- What is overlap?
- Where do we cut?
- Try it: cut the hostel rules
- Keep metadata with every chunk
- Tables and code are special
- Trade-offs and how to choose
- In a real system
- Interview tip
- Key takeaways
10 quiz questions at the end.
The full lesson, with interactive demos and the quiz, opens after you sign in with Google. It is free.