Understanding Layer Patching in Model Size Interpolation
cs.LG
Submitted: 2026-07-09
Updated: 2026-11-13
Terminology
Sources
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks
- The Pile: An 800GB Dataset of Diverse Text for Language Modeling
- The Llama 3 Herd of Models
- Distilling the Knowledge in a Neural Network
- Training Compute-Optimal Large Language Models
- Instruction-Following Pruning for Large Language Models
- Scaling Laws for Neural Language Models
- Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models
- ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
- LLM Pruning and Distillation in Practice: The Minitron Approach
- Kimi K2.5: Visual Agentic Intelligence
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- Qwen3 Technical Report
- FinerCut: Finer-grained Interpretable Layer Pruning for Large Language Models
- Instruction-Following Evaluation for Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis