Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
The Canonical-Context On-Policy Distillation (CCOPD) method fixes LLM inconsistency when information is revealed gradually across multiple turns. It trains a student model by aligning its behavior on messy, partial conversation histories with a teacher model trained on complete, clean prompts. This enforces consistency by minimizing the drift caused by early assistant replies.