ASAP: Attention Sink Anchored Pruning
cs.LG
Submitted: 2026-05-21
Updated: 2026-11-05
Terminology
Sources
- PPT: Token Pruning and Pooling for Efficient Vision Transformers
- Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations
- SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference
- Efficient Streaming Language Models with Attention Sinks
- When Attention Sink Emerges in Language Models: An Empirical View
- Why do LLMs attend to the first token?
- What are you sinking? A geometric approach on attention sink
- How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers
- The Kinetics Human Action Video Dataset
- MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
Related papers
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis
- (Mis)generalization of Helpful-Only Fine-tuning