Gradient Descent
In the last lesson you found the best line with one NumPy call. Gradient descent is how a computer finds the best numbers by itself, step by step. Almost every AI model, from a small line to a large language model, learns this way.
New words
- Loss
- One number that says how wrong the model is. Smaller is better.
- Gradient
- The slope of the loss. It shows which way is uphill.
- Learning rate
- The size of each step. It is a number you choose.
- Epoch
- One full pass over all the training data.
- Batch
- The group of examples used to make one step.
- Local minimum
- A low point that is not the lowest point of all.
What you will learn
- How can a computer find the best line?
- What is the update rule?
- How do you get the slope for a line?
- How does the training loop work?
- What is a good learning rate?
- How do you see that the model is learning?
- Batch, stochastic and mini-batch
- Try it: walk down the valley
- Common mistakes
- Explain it to a friend
- Key takeaways
10 quiz questions at the end.
The full lesson, with interactive demos and the quiz, opens after you sign in with Google. It is free.