GSTEP: Global Spatio-Temporal Density-Driven Visual Token Pruning for Efficient Video Large Language Models

arXiv:2608.03083 · cs.CV, cs.CL · Submitted 2026-08-04 · Read on arXiv

Mengjie Zhang, Qihui Zhu, Tao Zhang, Shuangwu Chen, Huihuang Qin, Yu Guo, Shenghao Ye, Zijian Wen, Yunpeng Hou, Dong Jin, Xiaobin Tan, Huasen He, Jian Yang

cs.CV, cs.CL

Submitted: 2026-08-04

Comments: 4 figures, accepted to ACM MM 26'

Code: https://github.com/yeluoy/GSTEP

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Pruning — 96× in this paper · explained in 4 episodes
• Benchmark — 30× in this paper · explained in 2 episodes
• Redundancy — 8× in this paper · explained in 2 episodes
• Transformer — 6× in this paper · explained in 2 episodes
• Transformers — 5× in this paper · explained in 2 episodes
• Accuracy — 4× in this paper · explained in 1 episode
• Generalization — 4× in this paper · explained in 14 episodes
• Vision-language models — 4× in this paper · explained in 2 episodes
• Ablation — 3× in this paper · explained in 1 episode
• Ablation Study — 2× in this paper · explained in 3 episodes
• Decoupling — 2× in this paper · explained in 2 episodes
• Farthest Point Sampling — 2× in this paper
• Instruction Tuning — 2× in this paper · explained in 3 episodes
• KV cache — 2× in this paper · explained in 4 episodes
• Benchmarking — 1× in this paper · explained in 4 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Evaluation Protocol — 1× in this paper · explained in 2 episodes
• Few-Shot Learning — 1× in this paper · explained in 6 episodes
• Hyperparameters — 1× in this paper · explained in 2 episodes
• MLLMs — 1× in this paper · explained in 2 episodes
• Papers — 1× in this paper · explained in 3 episodes
• Temporal Reasoning — 1× in this paper · explained in 3 episodes
• Test-Time Scaling — 1× in this paper · explained in 4 episodes
• Transferability — 1× in this paper · explained in 5 episodes

Sources

• MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens
• Qwen3-VL Technical Report
• Qwen2.5-VL Technical Report
• Token Merging: Your ViT But Faster
• VideoLLM: Modeling Video Sequence with Large Language Models
• LongVILA: Scaling Long-Context Visual Language Models for Long Videos
• Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
• FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• LLaVA-OneVision: Easy Visual Task Transfer
• Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior
• Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations
• Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model
• HoliTom: Holistic Token Merging for Fast Video Large Language Models
• FastVID: Dynamic Density Pruning for Fast Video Large Language Models
• AdaTP: Attention-Debiased Token Pruning for Video Large Language Models
• PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction
• Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving
• PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning
• Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models