Beyond IID: How General Are Tabular Foundation Models, Really?
cs.LG, cs.AI
Submitted: 2026-06-29
Updated: 2026-10-30
Comments: Accepted at NeurIPS 2026 Datasets and Benchmarks Track. v2: NeurIPS camera-ready version
Code: https://github.com/TabArena/data-foundry
Project page: https://tabarena.github.io/data-foundry
License: http://creativecommons.org/licenses/by/4.0/
The gist: Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry.
Terminology
Abstract
Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry. Research communities across disciplines are increasingly evaluating tabular foundation models on diverse datasets and tasks. However, these task- and discipline-specific evaluations remain largely inaccessible to model researchers because benchmark software and evaluation protocols are fragmented. As a result, model researchers rely on standard benchmarks, which are mostly defined for tasks where tabular foundation models already excel. The most challenging scenarios are excluded, limiting meaningful progress in the field by focusing on marginal improvements on IID data rather than on broader, more demanding challenges. To overcome this, we introduce BeyondArena, the first unified holistic benchmark for tabular data that supports diverse task types (IID, temporal, grouped), across sample size and feature dimensionality scales, with diverse feature types (with text, with high cardinality) from a broad range of disciplines. To enable unified benchmarking beyond standard benchmarks, we introduce DataFoundry, a Python framework and metadata schema for curating tabular datasets for predictive machine learning. Our results across 11 models and 142 curated datasets show that current open-source tabular foundation models are competitive or excel on datasets up to 100k rows, while traditional tree-based and deep learning models still dominate on grouped or large data. BeyondArena guides model research for the most demanding challenges in tabular data, enabling progress towards truly foundational tabular models.
Sources
- Can TabPFN Compete with GNNs for Node Classification via Graph Tabularization?
- Turning Tabular Foundation Models into Graph Foundation Models
- Bringing Graphs to the Table: Zero-shot Node Classification via Tabular Foundation Models
- TFMLinker: Universal Link Predictor by Graph In-Context Learning with Tabular Foundation Models
- GIT-BO: High-Dimensional Bayesian Optimization with Tabular Foundation Models
- TALENT: A Tabular Analytics and Learning Toolbox
- LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence
- TabReD: Analyzing Pitfalls and Filling the Gaps in Tabular Deep Learning Benchmarks
- Towards Benchmarking Foundation Models for Tabular Data With Text
- A Closer Look at TabPFN v2: Understanding Its Strengths and Extending Its Capabilities
- Comparing Task-Agnostic Embedding Models for Tabular Data
- Realistic Evaluation of TabPFN v2 in Open Environments
- No Need to Train Your RDB Foundation Model
- Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules
- TabPFN: One Model to Rule Them All?
- Risk In Context: Benchmarking Privacy Leakage of Foundation Models in Synthetic Tabular Data Generation
- Adaptation and Fine-tuning with TabPFN for Travelling Salesman Problem
- Bridging Streaming Continual Learning via In-Context Large Tabular Models
- FIRE: Multi-fidelity Regression with Distribution-conditioned In-context Learning using Tabular Foundation Models
- From Tables to Signals: Revealing Spectral Adaptivity in TabPFN
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis