Output-Aware Block Influence: Jacobian-Lens Weighting for Depth Pruning
cs.LG, cs.AI
Submitted: 2026-09-30
Updated: 2026-09-30
Comments: 11 pages, 7 figures, NeurIPS 2026 AXIOM Workshop
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- The Unreasonable Ineffectiveness of the Deeper Layers
- Verbalizable Representations Form a Global Workspace in Language Models
- Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs
- Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methods
- Persona-Pruner: Sculpting Lightweight Models for Role-Playing
- ShortGPT: Layers in Large Language Models are More Redundant Than You Expect
- LLM Pruning and Distillation in Practice: The Minitron Approach
- Wanda++: Pruning Large Language Models via Regional Gradients
- LaCo: Large Language Model Pruning via Layer Collapse
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis