TextReg: Mitigating Prompt Distributional Overfitting via Regularized Text-Space Optimization
TextReg introduces a framework to stop prompts from overfitting to training data by penalizing representational inefficiency. It measures inefficiency through prompt length and narrowness of rules, which causes poor generalization. The method uses dual-evidence gradient purification and semantic edit regularization to guide prompt updates, leading to significant improvements in out-of-distribution performance.