Visual Cipher Bench: Benchmarking LLM Agents on Long-Horizon Visual Reasoning
cs.CV
Submitted: 2026-11-12
Updated: 2026-11-16
Project page: https://lab.einsia.ai/visual-cipher-bench
Terminology
Sources
- SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
- Measuring AI Ability to Complete Long Software Tasks
- GPT-4 Technical Report
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Qwen3-VL Technical Report
- Visual Programming: Compositional visual reasoning without training
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning
- BLINK: Multimodal Large Language Models Can See but Not Perceive
- Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs
- Vision language models are blind: Failing to translate detailed visual features into words
- MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
- MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark
- MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?
- Benchmarking PhD-Level Coding in 3D Geometric Computer Vision
- RAVEN: A Dataset for Relational and Analogical Visual rEasoNing
- ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems
- WebArena: A Realistic Web Environment for Building Autonomous Agents
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs