Reasoning Supervision and Capability Retention in LoRA Domain Adaptation
cs.AI, cs.CL, cs.LG
Submitted: 2026-11-12
Updated: 2026-11-12
Terminology
Sources
- Qwen3 Technical Report
- QLoRA: Efficient Finetuning of Quantized LLMs
- LoRA Learns Less and Forgets Less
- Scaling Laws for Forgetting When Fine-Tuning Large Language Models
- BloombergGPT: A Large Language Model for Finance
- FinGPT: Open-Source Financial Large Language Models
- PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Finance
- Scaling Laws for Task-Specific LLM Distillation
- Distilling the Knowledge in a Neural Network
- Sequence-Level Knowledge Distillation
- Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
- Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation
- Instruction-Following Evaluation for Large Language Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents