Evaluation Metrics
A model can look 98% correct and still be useless. In this lesson you learn why accuracy alone can fool you. You also learn the metrics (the numbers that score a model) that tell the true story: precision, recall, F1 and AUC.
New words
- Confusion matrix
- A small table that counts the right and wrong guesses of a model.
- Precision
- Of the items the model flagged, how many were truly positive.
- Recall
- Of all the truly positive items, how many the model caught.
- F1 score
- One number that mixes precision and recall. It is low if either is low.
- Threshold
- The score above which the model says "yes".
- ROC curve and AUC
- A picture and a number that show how well a model separates two groups.
What you will learn
- Why can 98% accuracy be useless?
- What is a confusion matrix?
- What are accuracy, precision and recall?
- What is the F1 score?
- What happens when the threshold moves?
- What are ROC and AUC?
- Which metric should you pick?
- Try it: move the threshold
- Common mistakes
- Explain it to a friend
- Key takeaways
10 quiz questions at the end.
The full lesson, with interactive demos and the quiz, opens after you sign in with Google. It is free.