Evaluating Rubric Generation with Interventional Transfer
The authors introduce Interventional Transfer (IT), a method to evaluate rubric generation by testing how changing responses affects scores across different rubrics. IT reveals that rubrics are useful for detecting quality degradation rather than just serving as benchmarks for improvement, highlighting the importance of tying rubric evaluation to the specific downstream task.