Weight Space Learning for Certifiable Few-shot Transfer Learning
STEEL uses a discrete hypothesis space approach to provide non-trivial performance guarantees for few-shot transfer learning in LLMs and vision tasks. It trains a diffusion model over upstream parameters to create a finite set of PEFT candidates, then selects the best one via sample-and-evaluate. This yields tight risk certificates that prove performance bounds, outperforming continuous methods in low-shot regimes.