TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception
cs.CV, cs.AI
Submitted: 2026-09-25
Updated: 2026-09-29
Comments: Published at ECCV 2026, 49 pages, 20 figures
License: http://creativecommons.org/licenses/by/4.0/
The gist: We present TriO, a multi-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow and LiDAR.
Terminology
Abstract
We present TriO, a multi-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow and LiDAR. In contrast to prior work, TriO utilizes three distinct sensor modalities (camera, LiDAR, and RADAR) as both inputs and sources of self-supervision, eliminating the need for additional human annotations. Thanks to its novel supervision, the model is able to segment any occupancy from the drivable surface, overcoming the limitations of existing open-set methods in handling long-tail objects. TriO achieves state-of-the-art results in multiple 3D and 4D tasks, including occupancy, flow, and LiDAR prediction, as well as zero-shot road obstacle segmentation across multiple datasets such as Argoverse 2, and Spotting the Unexpected.
Sources
- ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimation
- Submanifold Sparse Convolutional Networks
- DOME: Taming Diffusion Model into High-Fidelity Controllable Occupancy World Model
- FSF-Net: Enhance 4D Occupancy Forecasting with Coarse BEV Scene Flow for Autonomous Driving
- GASP: Unifying Geometric and Semantic Self-Supervised Pre-training for Autonomous Driving
- GroundNet: Monocular Ground Plane Normal Estimation with Geometric Consistency
- GaussianFusionOcc: A Seamless Sensor Fusion Approach for 3D Occupancy Prediction Using 3D Gaussians
- Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
- SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
- Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting
- UniPAD: A Universal Pre-training Paradigm for Autonomous Driving
- OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments
- Self-Supervised Representation Learning with Joint Embedding Predictive Architecture for Automotive LiDAR Object Detection
- Deformable ConvNets v2: More Deformable, Better Results
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models