Surg-UniWorld: A Unified Surgical World Model with Multimodal Control Experts
Rulin Zhou, Wanhao Liu, Guoheng Ma, Liangjin Shao, Qiujie Song, Yidu Wang, Guankun Wang, Tong Chen, Long Bai, Luping Zhou, Hongliang Ren
cs.AI, cs.CV
Submitted: 2026-08-07
Updated: 2026-08-10
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Controllable surgical world models can provide a generative foundation for surgical artificial intelligence and simulation by synthesizing realistic instrument--tissue interactions.
Terminology
Abstract
Controllable surgical world models can provide a generative foundation for surgical artificial intelligence and simulation by synthesizing realistic instrument--tissue interactions. However, existing methods lack a unified multimodal control paradigm, while direct fusion of heterogeneous visual conditions often causes anatomical distortion, instrument appearance drift, and temporally inconsistent interactions. In this work, we propose Surg-UniWorld, a unified surgical world model with multimodal control experts. Surg-UniWorld first constructs a Hierarchical Surgical Anchor from first-frame appearance and hierarchical semantic masks to preserve persistent scene identity, anatomical organization, and interaction boundaries. Anchor-Relative Modality Experts then interpret edge, depth, and optical-flow evidence relative to the shared anchor, capturing complementary boundary, geometric, and motion information. A Multimodal Control Expert further performs contribution-preserving stage-wise composition of the activated modality increments and generates control hints for the Wan2.2 video diffusion backbone. To support multimodal surgical world modeling, we further construct Cholec80-SurgWAM, a benchmark for controllable surgical video generation. Extensive experiments demonstrate that Surg-UniWorld consistently outperforms existing controllable video generation methods and surgical world-model baselines in generation quality, temporal consistency, and multimodal controllability.
Sources
- WorldSimBench: Towards Video Generation Models as World Simulators
- Cosmos-H-Surgical: Learning Surgical Robot Policies from Videos via World Modeling
- How Far Are Surgeons from Surgical World Models? A Pilot Study on Zero-shot Surgical Video Generation with Expert Assessment
- SAW: Toward a Surgical Action World Model via Controllable and Scalable Video Generation
- World Simulation with Video Foundation Models for Physical AI
- LTX-Video: Realtime Video Latent Diffusion
- Wan: Open and Advanced Large-Scale Video Generative Models
- SurGen: Text-Guided Diffusion Model for Surgical Video Generation
- WAFT: Warping-Alone Field Transforms for Optical Flow
- Depth Anything 3: Recovering the Visual Space from Any Views
- Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems