Test-time Correction: An Online 3D Detection System via Visual Prompting
cs.CV
Submitted: 2024-12-10
Updated: 2026-10-29
Code: https://github.com/open-mmlab/mmdetection3d
License: http://creativecommons.org/licenses/by/4.0/
The gist: This paper introduces Test-time Correction (TTC), an online 3D detection system designed to rectify test-time errors using various auxiliary feedback…
Terminology
Abstract
This paper introduces Test-time Correction (TTC), an online 3D detection system designed to rectify test-time errors using various auxiliary feedback, aiming to enhance the safety of deployed autonomous driving systems. Unlike conventional offline 3D detectors that remain fixed during inference, TTC enables immediate online error correction without retraining, allowing autonomous vehicles to adapt to new scenarios and reduce deployment risks. To achieve this, we equip existing 3D detectors with an Online Adapter (OA) module -- a prompt-driven query generator for real-time correction. At the core of OA module are visual prompts: image-based descriptions of objects of interest derived from auxiliary feedback such as mismatches with 2D detections, road descriptions, or user clicks. These visual prompts, collected from risky objects during inference, are maintained in a visual prompt buffer to enable continuous correction in future frames. By leveraging this mechanism, TTC consistently detects risky objects, achieving reliable, adaptive, and versatile driving autonomy. Extensive experiments show that TTC significantly improves instant error rectification over frozen 3D detectors, even under limited labels, zero-shot settings, and adverse conditions. We hope this work inspires future research on post-deployment online rectification systems for autonomous driving.
Sources
- MP3: A Unified Model to Map, Perceive, Predict and Plan
- BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View
- Kosmos-2: Grounding Multimodal Large Language Models to the World
- GPT-4 Technical Report
- Gemini: A Family of Highly Capable Multimodal Models
- SAM3D: Zero-Shot 3D Object Detection via Segment Anything Model
- SAM2Point: Segment Any 3D as Videos in Zero-shot and Promptable Manners
- OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning
- SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking
- Anything-3D: Towards Single-view Anything Reconstruction in the Wild
- Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
- Track Anything: Segment Anything Meets Videos
- Segment Everything Everywhere All at Once
- Tent: Fully Test-time Adaptation by Entropy Minimization
- Continual Test-Time Adaptation for Object Detection with Adaptive Monitoring and Randomized Restoration
- Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detection
- Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection
- Sparse4D v2: Recurrent Temporal Fusion with Sparse Model
- Open-World Entity Segmentation
- Decoupled Weight Decay Regularization
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs