OVD: On-policy Verbal Distillation
On-policy Verbal Distillation (OVD) transfers reasoning skills from large teacher models to smaller student models using discrete verbal feedback instead of complex token matching. It replaces memory-intensive logit storage with simple 0-9 scores, drastically reducing memory usage while maintaining or improving performance on web Q&A and math reasoning tasks.