SUREON: A Benchmark and Vision-Language Model for Surgical Reasoning
cs.CV, cs.AI
Submitted: 2026-03-06
Updated: 2026-10-30
Comments: WACV 2027
Project page: https://aperezr20.github.io/sureon
License: http://creativecommons.org/licenses/by/4.0/
The gist: A capable surgical AI model should reason from visual evidence; yet, many existing surgical benchmarks can be partially solved without visual input…
Terminology
Abstract
A capable surgical AI model should reason from visual evidence; yet, many existing surgical benchmarks can be partially solved without visual input, as models may exploit procedural priors or common anatomy-instrument interactions to infer answers from language alone. To address this, we introduce SUREON, a video question-answering (QA) dataset and benchmark designed to evaluate visually grounded surgical reasoning. Our pipeline generates QA pairs from narrated surgical academic videos, refines incorrect options using multiple models to reduce linguistic biases, and uses trained adversarial guessers to filter out questions answerable without visual evidence. The resulting dataset contains 76K training pairs covering 168 procedures and 10 specialties, alongside a benchmark of 340 questions across 10 categories verified by expert surgeons. Benchmarking leading commercial and open-source models shows that surgical visual reasoning remains largely unsolved--no model exceeds 53% accuracy and even expert surgeons reach only 64.9%. We also introduce SUREON-VLM, a baseline trained on our curated data that avoids language shortcuts, outperforms existing open-source models on SUREON, and matches larger specialist models on multiple standard surgical benchmarks. Our training ablations show that distractor refinement and blind curation strengthen visual grounding, increasing the multimodal gain from 8.7 to 29.1 percentage points while reducing the training set from 176.6K to 76K filtered samples. SUREON thus offers both a verified testbed for surgical visual reasoning and training data that strengthens visual grounding.
Sources
- 2017 Robotic Instrument Segmentation Challenge
- 2018 Robotic Scene Segmentation Challenge
- Qwen3-VL Technical Report
- LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settings
- Video-R1: Reinforcing Video Reasoning in MLLMs
- CholecSeg8k: A Semantic Segmentation Dataset for Laparoscopic Cholecystectomy Based on Cholec80
- DeepSeek-V3 Technical Report
- EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT
- SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge
- Systematic Evaluation of Large Vision-Language Models for Surgical Artificial Intelligence
- Video Dataset for Surgical Phase, Keypoint, and Instrument Recognition in Laparoscopic Surgery (PhaKIR)
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligence
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs