Encoders, scalers, imputers, and the scikit-learn Pipeline — the machinery that turns a messy table into a model input without ever peeking at the test set
By the end of tonight you'll have both the concept and the working code — every objective maps to a cell you run yourself.
Numeric columns get imputed and scaled, categorical columns get encoded, and the model sits at the end — one object that fits on train and transforms test, so leakage becomes structurally impossible.
One-hot for a handful of categories, ordinal when order is real, target encoding for 3,000 zip codes — and why the last one must be fit inside cross-validation.
Logistic regression, kNN, and SVMs care about scale; trees do not. Scale where it counts, and always after the split.
Ratios (charges per month of tenure), buckets, datetime parts, and interaction terms — the features a good analyst invents are usually worth more than the algorithm choice.
A business question, a dataset, a model — and the judgment to evaluate it.
Where the Churn Detective left off: the columns you would not know on sign-up day, and why that matters. Then the classic mistake live: scale the whole dataset, then split. The score looks great. It is a lie.
45,211 phone calls from a Portuguese bank's term-deposit campaign, 16 features, 11.7% said yes. Numeric and categorical columns mixed together, one column ('duration') that is pure leakage — and a business question: who should the call center dial first?
Categories are text; models need numbers. OneHotEncoder for 'job' (12 values), OrdinalEncoder for education (order is real), and TargetEncoder for high-cardinality columns — with the cross-fitting trick that keeps it from leaking the label.
Stretch, coffee, compare notes on the Churn Detective and HW1 (due next Monday).
ColumnTransformer routes each column to its transformer; Pipeline chains preprocessing to the model; cross_val_score evaluates the whole thing honestly. One object, one fit call, zero leakage — and set_output(transform='pandas') keeps column names readable.
Back to Telco: charges-per-tenure-month, a 'new customer' flag, a count of add-on services, contract × payment interactions. Each one is a hypothesis about why customers leave. We add them with FunctionTransformer and measure whether AUC moves.
The reusable template: ColumnTransformer → Pipeline → cross_val_score. Next week we point it at 30,000 credit-card customers and learn to predict who defaults.
Notebooks, readings, explorers, and the stories behind the methods.
The wrong way and the right way side by side, then the full ColumnTransformer + Pipeline on Bank Marketing, target encoding, and engineered Telco features with before/after AUC.
Open in ColabTonight's live notebook: pandas continued from where Session 2 stopped — describe, value_counts, sorting, loc/iloc, new columns, missing values, groupby — on Titanic first, then the same moves on Telco through to who to call first.
Open in ColabSeven case files on the Telco data, one per part of Homework 1: the tampered TotalCharges column, groupby interrogations, engineered clues, the sign-up-day time machine, a model that beats the baseline, and framing your own prediction problem. Guess before you run, then a check cell prints ✓. 60–90 minutes; ends with a scorecard.
Open in ColabThe reference for every transformer we use tonight: StandardScaler, OneHotEncoder, OrdinalEncoder, TargetEncoder, and the imputers. Bookmark it.
Open the guideThe course taught by scikit-learn's own developers. The 'Predictive modeling pipeline' module is the best free walkthrough of ColumnTransformer that exists.
Open the MOOCFour short lessons: mutual information, creating features, k-means features, and target encoding. Ninety minutes, free, with exercises.
Start the courseTableVectorizer picks an encoder for every column automatically; the Cleaner handles dates and dirty strings. The 2025 answer to 'I have a table, give me features.'
See the docsThe Bank Marketing dataset's own documentation warns that 'duration' should be discarded for a realistic model — the outcome is known once the call ends. A leak so well known it comes with a label.
Read the dataset notesSearch the StatQuest index for 'one-hot' and 'standardization'. Ten minutes each, and the pictures stick.
Open the indexProf. Arslan's earlier notebook on line plots, scatter plots, histograms, and seaborn pair plots — the tools you use to decide which features to engineer. Hosted in the course repo — opens in Colab with one click.
Open in ColabOpen the notebook and build it with us.
Build a single scikit-learn object that imputes and scales the numeric columns, one-hot encodes the categoricals, and fits a logistic regression on the Bank Marketing data — then evaluate it with 5-fold cross-validation without leaking a single row.
select_dtypes(include='number') for numeric, the rest for categorical. Drop 'duration' — it is known only after the outcome.
Numeric branch: SimpleImputer(median) → StandardScaler. Categorical branch: OneHotEncoder(handle_unknown='ignore').
Pipeline([('prep', preprocessor), ('model', LogisticRegression(max_iter=1000))]). Display it — read the wiring diagram.
cross_val_score with StratifiedKFold(5) and scoring='roc_auc'. Every fold fits the scaler and encoder on its own training portion.
from sklearn.datasets import fetch_openml
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score, StratifiedKFold
bank = fetch_openml('bank-marketing', version=1, as_frame=True)
X, y = bank.data, (bank.target == '2').astype(int) # '2' = subscribed
X = X.drop(columns=['V12']) # V12 = call duration: leakage
num = X.select_dtypes(include='number').columns
cat = X.columns.difference(num)
prep = ColumnTransformer([
('num', Pipeline([('imp', SimpleImputer(strategy='median')), ('sc', StandardScaler())]), num),
('cat', OneHotEncoder(handle_unknown='ignore'), cat),
])
model = Pipeline([('prep', prep), ('clf', LogisticRegression(max_iter=1000))])
scores = cross_val_score(model, X, y, cv=StratifiedKFold(5, shuffle=True, random_state=835), scoring='roc_auc')
print(f'AUC: {scores.mean():.3f} ± {scores.std():.3f}') Notice that the model line is the least interesting part. Swap LogisticRegression for HistGradientBoostingClassifier and every other line stays put — the pipeline is the asset, the model is a parameter. And notice what dropping duration costs: AUC falls from the low 0.90s to the high 0.70s. The honest number is the one you can act on before the phone rings.
Every preprocessing step that learns from data — scalers, imputers, encoders — is a model. Fit it on the training fold only, and let a Pipeline enforce that for you.
Encoders are choices, not defaults: one-hot for few categories, ordinal when order is real, target encoding (cross-fit) for many categories. handle_unknown='ignore' because production will surprise you.
A feature is a hypothesis about the business. Charges per tenure-month, add-on counts, contract × payment interactions — the features you invent from domain knowledge usually move the score more than the algorithm you pick.
Complete these before next Monday — they are the practice that makes the next session land.
Exam-style questions for the ideas in this session — Data & Leak-Proof Features. Every wrong answer names the misconception behind it.
Logistic regression as odds, kNN and Naive Bayes as contrasts, the confusion matrix in depth, precision vs. recall, ROC vs. PR curves, and what to do when only 22% of your credit-card customers default.