Why random splits lie when time matters, lag and rolling features that turn forecasting into regression, classical baselines you must beat, and the zero-shot foundation models that changed the field in 2025
By the end of tonight you'll have both the concept and the working code — every objective maps to a cell you run yourself.
TimeSeriesSplit and rolling-origin backtests: train on the past, test on the future, never the reverse. The 4%-offline-14%-live gap has one cause.
Lags, rolling means, calendar features, holidays — then any regressor you already know. Gradient boosting on lag features is the workhorse of industrial demand forecasting.
Seasonal naive, exponential smoothing, AutoARIMA via StatsForecast. If your model can't beat 'same hour last week,' it isn't a model.
Chronos-2 and TimesFM forecast a series they have never seen, with no training. When they win, when they don't, and how to use one as the instant baseline.
A business question, a dataset, a model — and the judgment to evaluate it.
Meet the data: 17,379 hours of bike rentals in Washington DC, 2011–2012, with weather and calendar. A random train/test split gives MAPE in the single digits. A time-ordered split gives triple that. The first was interpolation; only the second is forecasting.
Naive (last value), seasonal naive (same hour last week), and a moving average. On the bike data seasonal naive lands at MAE ≈ 70 rides per hour over the last eight weeks. Everything we build tonight is judged against that number.
Exponential smoothing (AutoETS), AutoARIMA, and Theta from Nixtla's StatsForecast — fast, well-tested, and the right second baseline. Fit on the training window, forecast the horizon, backtest with cross_validation. Then the limits: one series at a time, no weather, no holidays.
Stretch, coffee, guess tomorrow's 8 AM demand.
Build features that were knowable at forecast time: rentals 24 hours ago, 168 hours ago, the 24-hour rolling mean, hour-of-day, day-of-week, holiday flag, and the weather forecast. Then HistGradientBoostingRegressor with TimeSeriesSplit. MAE drops well below the seasonal-naive baseline — and the model can use weather, which no classical model could.
A transformer pretrained on billions of time points forecasts your series with no training at all. Chronos-2 (Amazon, Oct 2025) handles covariates and led the GIFT-Eval benchmark at release; TimesFM 2.5 (Google) is Apache-licensed. We run Chronos on the bike series and compare to the boosted model. The evidence: strong on periodic and trending series, weaker on erratic ones — and instant.
The forecasting stack in 2026: seasonal naive → StatsForecast → lag features + GBM → a foundation model as the challenger, all validated with rolling-origin backtests. Project notebooks and memos are due in two weeks; your dry run is next Monday.
Notebooks, readings, explorers, and the stories behind the methods.
Random vs. time-ordered splits, seasonal-naive and StatsForecast baselines, lag/rolling/calendar features with HistGradientBoostingRegressor and TimeSeriesSplit, MAE by hour of day, and an optional Chronos-2 zero-shot cell.
Open in ColabFanaee-T & Gama (2013): hourly and daily rental counts from Capital Bikeshare, 2011–2012, with weather and calendar variables. hour.csv is the file we use.
Open the dataset pageSix lessons: trend, seasonality, lags, hybrid models, and forecasting with ML — with exercises on a store-sales dataset. The best free companion to tonight.
Start the courseThe free, definitive forecasting textbook. Chapters 5 (toolbox: benchmarks, residuals, evaluation) and 13.6 (forecasting with ML) are tonight; it uses R, but the ideas are language-free.
Read online (free)StatsForecast for AutoARIMA/ETS/Theta at scale, mlforecast for lag features and gradient boosting across many series with recursive forecasting built in. The free industrial stack.
Open the repoThe foundation model that forecasts zero-shot with covariates and multivariate inputs, leading the GIFT-Eval and fev-bench leaderboards at release. Apache-2.0 weights on Hugging Face.
Read the paperpip install chronos-forecasting; three lines to a zero-shot forecast. The README's quick-start is the optional cell in tonight's notebook.
Open the repoA 2025 engineering post on predicting the hardest days of the year: gradient-boosted models with holiday multipliers layered on top. Lag features, calendar effects, and business overrides in production.
Read the postA permanent competition on Ecuadorian grocery sales with promotions, oil prices, and holidays — the natural next dataset after bikes, and a strong final-project candidate.
Open the competitionOpen the notebook and build it with us.
Forecast hourly rentals one day ahead. Build the seasonal-naive baseline, then lag, rolling, and calendar features for HistGradientBoostingRegressor, validate with TimeSeriesSplit, and report MAE against the baseline — overall and by hour of day.
Sort by datetime. Last 8 weeks are the test set; TimeSeriesSplit(5) on the rest for tuning. Never shuffle.
Seasonal naive: predict the count from exactly 168 hours earlier. Compute MAE on the test weeks.
lag_24, lag_168, rolling mean of the previous 7 days ending 24 hours ago, hour, weekday, holiday, workingday, and the weather columns (treated as the forecast).
HistGradientBoostingRegressor with early stopping. MAE overall, then groupby hour — where does the model still miss? Plot a test week: actual vs. baseline vs. model.
import pandas as pd, numpy as np
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.metrics import mean_absolute_error
bike = pd.read_csv('hour.csv') # UCI Bike Sharing, hourly
bike['ts'] = pd.to_datetime(bike['dteday']) + pd.to_timedelta(bike['hr'], unit='h')
bike = bike.set_index('ts').asfreq('h') # fill the missing hours
bike['cnt'] = bike['cnt'].interpolate()
y = bike['cnt']
feat = pd.DataFrame({
'lag_24': y.shift(24), 'lag_168': y.shift(168),
'roll_7d': y.shift(24).rolling(168).mean(),
'hour': bike.index.hour, 'dow': bike.index.dayofweek,
'holiday': bike['holiday'].ffill(), 'workingday': bike['workingday'].ffill(),
'temp': bike['temp'].interpolate(), 'hum': bike['hum'].interpolate(), 'weathersit': bike['weathersit'].ffill(),
}).dropna()
y = y.loc[feat.index]
cut = feat.index.max() - pd.Timedelta(weeks=8) # last 8 weeks = test
X_tr, X_te, y_tr, y_te = feat[:cut], feat[cut:], y[:cut], y[cut:]
print(f'seasonal naive MAE: {mean_absolute_error(y_te, X_te["lag_168"]):.1f} rides/hour')
gbm = HistGradientBoostingRegressor(learning_rate=0.05, max_iter=2000, early_stopping=True, random_state=835).fit(X_tr, y_tr)
print(f'GBM MAE: {mean_absolute_error(y_te, gbm.predict(X_te)):.1f} rides/hour ({gbm.n_iter_} trees)') The boosted model beats seasonal naive by a wide margin — but look at the by-hour MAE: the biggest misses are the 8 AM and 5 PM commute peaks on days when the weather turned. That is the honest story for the operations team: we are good on normal days and we know exactly which hours to over-stage bikes on bad-weather days. Then run the optional Chronos cell: with zero features and zero training it lands between the baseline and the GBM. In 2026 that is where you start, not where you finish.
Time series must be validated in time order. A random split lets lag features see the future; TimeSeriesSplit and rolling-origin backtests are the only honest scoreboards.
Forecasting is regression on features that were knowable at forecast time: lags that respect the horizon, rolling statistics, calendar, holidays, and forecast weather. Gradient boosting on those features is the industrial workhorse.
Beat seasonal naive, then StatsForecast, then run a foundation model. Chronos-2 and TimesFM forecast zero-shot and are strong on periodic series — the new instant baseline, not yet the finish line.
Complete these before next Monday — they are the practice that makes the next session land.
Exam-style questions for the ideas in this session — Trust & Structure. Every wrong answer names the misconception behind it.
A multilayer perceptron on tabular data (and why it usually loses to boosting), text as features — TF-IDF, sentence embeddings, and zero-shot LLM labels — and the AI-assisted analyst's workflow: generate, verify, own.