Your Prediction Story
Pick a real question about the future. Model it honestly. Turn the prediction into a decision with a number on it. Present Mon Dec 14 in class.
The brief, in one paragraph
Find one question about the future that an organization — your employer, a business you know, a nonprofit, a team — answers today by guesswork or a spreadsheet: who churns, who defaults, how many, how much, which ones. Build one predictive model in Python that answers it: a leak-proof pipeline, an honest validation, a baseline beaten. Then do the part most projects skip — turn the scores into a decision with a number on it, and show why the decision-maker should trust it: why the model says what it says, how sure it is, and where it fails. That last translation — from probability to decision to trust — is the skill this course exists to teach.
Five Beats of a Prediction Story
Every good predictive analytics story has the same five beats. Your project hits all five, in miniature.
Unit, target, horizon, and a decision someone actually owns and makes badly today.
Wrangled, understood, leaks removed — with the split before any fitted transformer.
The right metric on a held-out set that mirrors production, beating the obvious baseline.
A threshold, a list, or a reorder point — and what it is worth, estimated honestly.
Why (SHAP), how sure (calibration, conformal), where it fails (slices), and what to monitor.
What to Submit
Three things on Canvas by Mon Dec 14 at 5:00 PM — notebook, memo, and repo link — then the presentation in class that evening. Four small milestones along the way keep the weeks before from piling up.
Working notebook
A Colab notebook that runs top to bottom: data in, leak-proof pipeline, honest validation, a tuned decision threshold or interval, and a short model card at the end.
Decision memo
Two pages to a named decision-maker: the question, the model in plain English, what it predicts, the decision and the dollars, and the trust story. No code, no math notation.
Public GitHub repository
A public repo with a README that tells the story (summary, objectives, dataset, tools, Colab links, how to run), the notebook, a visualizations folder, and the memo as PDF or Markdown. Optional: publish it as a GitHub Pages site with the template below. Check every link in an incognito window before you submit.
Final presentation
Eight minutes plus two for questions on Mon Dec 14: your prediction story, told the way a data science lead tells it to an executive — question, data, model, evaluation, decision, trust.
The decision memo, structured
Five labeled sections, about a paragraph each, written to a named decision-maker. Plain business English — the code lives in the notebook.
- The questionOne or two sentences: what is being predicted (unit, target, horizon), who owns the decision it changes, and what they do today without it.
- The data & the modelWhere the data came from, what you had to fix, what you dropped as leakage, and which model won — described in business English, with the pipeline in the notebook.
- The honest evaluationThe metric that matches the decision, on a held-out set that matches production, against the obvious baseline. State the number and what it means in plain words.
- The decision & the moneyWhat the model tells the decision-maker to do differently — the threshold, the top-k list, the reorder point — and what it is worth: dollars saved, revenue gained, hours freed. Estimated honestly.
- The trust storyWhy the model says what it says (SHAP for one case), how sure it is (calibration or a conformal interval), where it fails (sliced error rates), and what you would monitor after launch.
Four Small Steps, Pass/Fail
Each milestone is small on purpose: it exists so the last week is a polish, not a scramble. Pass means the piece is there and the notebook runs. Together they are 20 of the project's 120 points. Every date is on the syllabus and in the course calendar file.
Problem & dataset
One paragraph on Canvas: the prediction (unit, target, horizon), who owns the decision it changes, and where the data comes from. Posted as a discussion, so you can borrow ideas from other teams.
Baseline with a metric
A notebook that loads the data, splits before anything is fitted, computes the naive baseline, and scores a first model with the metric the decision needs.
Tuned model + error analysis
Hyperparameters tuned inside cross-validation, calibration if you classify, SHAP for three cases, error rates by the slices that matter, and a half-page model card.
Dry run
Draft slides (PDF, ten at most) on Canvas before class; four-minute dry runs in pairs during Session 12, with written notes from the instructor before the real thing.
How It Is Graded
Forty percent of the course grade, split five ways. Modeling is a quarter of it; the decision is another quarter.
Eight Domains to Adapt
Starting points, not assignments — the best projects come from questions you have actually watched someone guess at.
- Who cancels next month at a subscription business you know — with a retention offer priced into the threshold
- Which gym members, students, or app users go dormant in 60 days, and what a win-back message is worth
- Loan default on Lending Club or Home Credit data: a scorecard, a boosted model, and the expected-loss threshold that separates them
- Invoice non-payment for a small business: who to call first this week
- Daily sales for a store, restaurant, or product line (Favorita, M5, or your own data), validated in time order
- Hourly demand for a service — bikes, ER visits, support tickets — with staffing as the decision
- What is this listing worth? A pricing model for homes, used cars, or short-term rentals with honest error bars
- Which hotel or airline bookings will cancel, and how much to overbook
- Transaction fraud with a 0.2% base rate: PR-AUC, a cost matrix, and an alert budget the review team can handle
- Anomalies in sensor, expense, or claims data with Isolation Forest — and a triage rule for the top 1%
- 30-day readmission or appointment no-shows: a leak-proof model and the outreach list it produces
- Emergency-department volume by hour for staffing, with holidays and weather
- Who responds to a campaign (Bank Marketing, Olist, or your CRM export) with a gain chart and a contact budget
- Customer segments from RFM on real transactions, with one action per segment and a plan to measure it
- Employee attrition or late deliveries — with error rates sliced by group and a fairness note in the model card
- Which support tickets will escalate, with GroupKFold by customer and a routing decision
Public datasets that work well: Telco churn and Hotel bookings (in the course repo), Lending Club, Home Credit, Credit-Card Fraud, Ames and Kaggle House Prices, Olist e-commerce, Instacart, Favorita and M5 forecasting, UCI Bank Marketing, Online Retail II, Bike Sharing. Projects touching credit, hiring, insurance, or health must include sliced error rates and a fairness note in the model card.
Where to Begin
Each session gives you a reusable notebook pattern. Match your project to its session and start from that notebook.
Common Questions
Does my project need real data?
Can I use a Kaggle dataset or competition?
How big does the model have to be?
Can I use AI tools like Claude, ChatGPT, or Colab's Data Science Agent?
Team or solo?
What makes an A project?
When are things due?
The Signal Award
On the last night, after all presentations, the class votes for the project that best found the signal: a real question, an honest model, a decision that changed because of it, and a number attached. Netflix credits its models with a billion dollars a year in retained subscribers; Stripe with a third less fraud. Your project is your first entry in that tradition.