TabICLv2: A better, faster, scalable, and open tabular foundation model
cs.LG
Submitted: 2026-02-11
Updated: 2026-09-16
Comments: Published at ICML 2026. Updates in v2: More experiments in Appendix L, smaller corrections
Code: https://github.com/soda-inria/tabicl
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Orion-MSP: Multi-Scale Sparse Attention for Tabular In-Context Learning
- Fine-tuned In-Context Learning Transformers are Excellent Tabular Data Classifiers
- Cautious Weight Decay
- Critical attention scaling in long-context transformers
- Sample Path Regularity of Gaussian Processes from the Covariance Kernel
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- Efficient Autoregressive Inference for Transformer Probabilistic Models
- MachineLearningLM: Scaling Many-shot In-context Learning via Continued Pretraining
- TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second
- AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data
- TabPFN-Wide: Continued Pre-Training for Extreme Feature Counts
- YaRN: Efficient Context Window Extension of Large Language Models
- Sparse Sequence-to-Sequence Models
- Muon is Scalable for LLM Training
- TabPFN Unleashed: A Scalable and Effective Solution to Tabular Classification Problems
- Foundation Models for Causal Inference via Prior-Data Fitted Networks
- Theory, Analysis, and Best Practices for Sigmoid Self-Attention
- Transformers Can Do Bayesian Inference
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks