A team has a 40,000-row, 30-feature table of loan applications. They compare a tuned LightGBM model, a default XGBoost model, a 3-layer neural network, and a tabular foundation model (TabPFN-2.5 / TabICL) used zero-shot.
Based on the 2022–2026 benchmark evidence, which outcome is most likely?
Why did you pick that answer? Two or three sentences. The act of articulating it is what builds the judgment — not the click that follows.
Grinsztajn et al. (2022) showed tree ensembles beat neural networks on typical tabular data, and that finding has held. What changed in 2025–26 is in-context tabular foundation models: TabPFN-2.5 reports winning against default XGBoost on most small-to-medium datasets, and TabICLv2 (open source) matches tuned GBDTs on most of TabArena. At 40k rows, a tuned GBDT and a foundation model are the two contenders; tuning still matters for the GBDT, and licensing and latency still favor it in production.
On tabular business data, plain MLPs usually trail tree ensembles unless heavily engineered. The evidence is consistent across benchmarks.
Defaults are a strong baseline, but learning rate, depth, and regularization typically buy several points. Tuned beats default reliably.
Tabular foundation models are pretrained on millions of synthetic tables specifically for this setting — and they are the main 2025–26 development in the field.