LAVE: Latent Visual Evidence-Enhanced Planning for Video Tool-use Agents

arXiv:2608.07585 · cs.CV, cs.LG, cs.MA · Submitted 2026-08-05 · Read on arXiv

Zijian Wang, Junnan Zhu, Rongzhen Li, Xiao Liu, Guohui Xiang, Quan Lu, Lijia Liu, Yining Wang, Jiang Zhong, Kaiwen Wei

cs.CV, cs.LG, cs.MA

Submitted: 2026-08-05

Updated: 2026-08-11

Comments: 16 pages, 6 figures, 9 tables. Includes appendix

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Planner — 90× in this paper · explained in 3 episodes
• Benchmark — 14× in this paper · explained in 2 episodes
• Provenance — 13× in this paper · explained in 5 episodes
• Decoder — 12× in this paper · explained in 2 episodes
• Redundancy — 7× in this paper · explained in 2 episodes
• Ablation — 6× in this paper · explained in 1 episode
• MLLMs — 4× in this paper · explained in 2 episodes
• Accuracy — 3× in this paper · explained in 1 episode
• Cosine Similarity — 3× in this paper · explained in 3 episodes
• Training-Free Framework — 3× in this paper · explained in 2 episodes
• Transformer — 3× in this paper · explained in 2 episodes
• Transformers — 3× in this paper · explained in 2 episodes
• Verification — 3× in this paper · explained in 2 episodes
• Ablation Study — 2× in this paper · explained in 3 episodes
• Evaluation Protocol — 2× in this paper · explained in 2 episodes
• Latent Visual Reasoning — 2× in this paper · explained in 2 episodes
• Localization — 2× in this paper · explained in 2 episodes
• Reinforcement Learning — 2× in this paper · explained in 4 episodes
• Tokenization — 2× in this paper · explained in 2 episodes
• Abstraction — 1× in this paper · explained in 1 episode
• Allocator — 1× in this paper · explained in 2 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Hyperparameters — 1× in this paper · explained in 2 episodes
• Iterative Refinement — 1× in this paper · explained in 5 episodes
• Multi-Agent System — 1× in this paper · explained in 7 episodes
• Multi-agent Collaboration — 1× in this paper · explained in 2 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Semantic Alignment — 1× in this paper · explained in 6 episodes
• Steering — 1× in this paper · explained in 2 episodes
• Vision-language models — 1× in this paper · explained in 2 episodes

Sources

• Scaling Video Understanding via Compact Latent Multi-Agent Collaboration
• Video-R1: Reinforcing Video Reasoning in MLLMs
• VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning
• FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learning
• Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
• SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learning
• VideoMind: A Chain-of-LoRA Agent for Temporal-Grounded Video Reasoning
• GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding
• Watch, Remember, Reason: Human-View Video Understanding with MLLMs
• DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs
• GPT-4o System Card
• Steering Language Models With Activation Engineering
• Video-EM: Event-Centric Episodic Memory for Long-Form Video Understanding
• A Glimpse to Compress: Dynamic Visual Token Pruning for Large Vision-Language Models
• Latent Visual Cache for Video Reasoning

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models