An insurer's claims dataset has a zip_code column with 2,400 distinct values. The model is a regularized logistic regression on 60,000 rows.
What is the most appropriate way to encode zip_code?
Why did you pick that answer? Two or three sentences. The act of articulating it is what builds the judgment — not the click that follows.
Target encoding compresses a 2,400-level category into one informative number — the ZIP's smoothed claim rate — estimated with internal cross-fitting so a row never sees its own target. It is the standard tool for high-cardinality categories in linear and boosted models, and scikit-learn's TargetEncoder handles the cross-fitting for you.
2,400 sparse columns, most with a handful of rows, give the model thousands of noisy coefficients to overfit. One-hot suits tens of categories, not thousands.
Ordinal encoding tells a linear model that ZIP 02115 is "more" than 02114. That invented order is noise for a linear model (trees tolerate it better).
Geography is often the strongest claims signal an insurer has. The column is valuable; only the naive encodings are the problem.