04
Domain 04
Classification & Decisions
Odds, confusion matrices, precision and recall, ROC versus PR, thresholds, cost matrices, calibration, and lift.
6 questions
- Q19What are the model's precision and recall?150 flagged, 120 right, 200 real churners. Precision and recall by hand.
- Q20Which metric should decide, and which model wins?Identical ROC-AUC, wildly different PR-AUC. Which one does fraud care about?
- Q21At what probability threshold does contacting a customer break even?A $50 offer, a $600 save. The break-even probability is not 0.5.
- Q22What does this coefficient mean?A logistic coefficient of 0.69. Say it in odds.
- Q23What is the problem, and why does it matter for the retention budget?AUC 0.88, but the 0.85 bucket churns at 55%. What broke?
- Q24What is the lift in the top decile, and what does it mean?Top decile captures 38% of buyers. Compute the lift and cash it in.