Beyond FLOPs: Benchmarking Real Inference Acceleration of LLM Pruning under a GEMM-Centric Taxonomy
cs.LG, cs.CL
Submitted: 2026-06-08
Updated: 2026-08-27
Comments: 28 pages, 21 figures, accepted by EMNLP2026 main
Code: https://github.com/EIT-NLP/LLM-Pruning
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Fluctuation-based Adaptive Structured Pruning for Large Language Models
- ELANA: A Simple Energy and Latency Analyzer for LLMs
- BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- From LLMs to LRMs: Rethinking Pruning for Reasoning-Centric Models
- What Do Visual Tokens Really Encode? Uncovering Sparsity and Redundancy in Multimodal Large Language Models
- SkipOPU: An FPGA-based Overlay Processor for Large Language Models with Dynamically Allocated Computation
- A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers
- Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methods
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- T'yr-the-Pruner: Structural Pruning LLMs via Global Sparsity Distribution Optimization
- The Llama 3 Herd of Models
- Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective
- Informed Routing in LLMs: Smarter Token-Level Computation for Faster Inference
- ViCA: Efficient Multimodal LLMs with Vision-Only Cross-Attention
- What Matters in Transformers? Not All Attention is Needed
- LLaMA: Open and Efficient Foundation Language Models
- Pointer Sentinel Mixture Models
- Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
- Mixture-of-Depths: Dynamically allocating compute in transformer-based language models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks