Adaptive-Policy Distillation: Knowledge Injection under Information Asymmetry
cs.CL, cs.LG
Submitted: 2026-10-31
Updated: 2026-10-31
Terminology
Sources
- On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
- A General Language Assistant as a Laboratory for Alignment
- Evaluating Large Language Models Trained on Code
- The Llama 3 Herd of Models
- MiniLLM: On-Policy Distillation of Large Language Models
- Distilling the Knowledge in a Neural Network
- Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
- ToDi: Token-wise Distillation via Fine-Grained Divergence Control
- DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
- DistiLLM: Towards Streamlined Distillation for Large Language Models
- SWITCH: Studying with Teacher for Knowledge Distillation of Large Language Models
- Efficient Knowledge Injection in LLMs via Self-Distillation
- CADENCE: Closing the Reasoning Gap via Coverage-Adaptive On-Policy Distillation
- EDGE-OPD: Internalizing Privileged Context with Evidence Guided On-Policy Distillation
- MixSD: Mixed Contextual Self-Distillation for Knowledge Injection
- AdaSwitch: Balancing Exploration and Guidance in Knowledge Distillation via Adaptive Switching
- TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models
- Learning by Distilling Context
- A Survey of On-Policy Distillation for Large Language Models
- GATES: Self-Distillation under Privileged Context with Consensus Gating
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data