UTOPI: Efficient Egocentric Long-Video Understanding in AR via User-Guided Token Pre-Compression
cs.LG
Submitted: 2026-11-08
Updated: 2026-11-08
Code: https://github.com/SAI-Lab-NYU/utopi-code
Project page: https://sai-lab-nyu.github.io/utopi
Terminology
Sources
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
- Prompt Cache: Modular Attention Reuse for Low-Latency Inference
- Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding
- End-to-End Video Question Answering with Frame Scoring Mechanisms and Adaptive Sampling
- Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
- Aria Everyday Activities Dataset
- Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models
- SmolVLM: Redefining small and efficient multimodal models
- In the Eye of MLLM: Benchmarking Egocentric Video Intent Understanding with Gaze-Guided Prompting
- GazeQwen: Lightweight Gaze-Conditioned LLM Modulation for Streaming Video Understanding
- Attend Before Attention: Efficient and Scalable Video Understanding via Autoregressive Gazing
- MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
- PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning
- Generative Frame Sampler for Long Video Understanding
- FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
- Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding
- DynTok: Dynamic Compression of Visual Tokens for Efficient and Effective Video Understanding
- Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMs
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis