MARBLE: An Agentic Benchmark for Spatial Reasoning and Visual Abstraction
cs.AI, cs.CL, cs.CV
Submitted: 2025-06-28
Updated: 2026-11-18
Project page: https://marble-benchmark.github.io
Terminology
Sources
- Qwen2.5-VL Technical Report
- MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks
- PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patterns
- ARC Prize 2024: Technical Report
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark
- GPT-4o System Card
- OpenAI o1 System Card
- LLaVA-OneVision: Easy Visual Task Transfer
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
- BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
- Seed1.5-VL Technical Report
- Evaluating Large Language Models with Grid-Based Game Competitions: An Extensible LLM Benchmark and Leaderboard
- Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey
- EscapeCraft: A 3D Room Escape Environment for Benchmarking Complex Multimodal Reasoning Ability
- A Surprising Failure? Multimodal LLMs and the NLVR Challenge
- Multimodal Inconsistency Reasoning (MMIR): A New Benchmark for Multimodal Reasoning Models
- V-MAGE: A Game Evaluation Framework for Assessing Vision-Centric Capabilities in Multimodal Large Language Models
- InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents