TAP-Out: Tracking Any Point in 360 via Outpainting

arXiv:2611.00406 · cs.CV · Submitted 2026-10-30 · Read on arXiv

cs.CV

Submitted: 2026-10-30

Updated: 2026-11-12

Comments: Accepted at BMVC 2026. 22 pages, including supplementary material. Project page: https://finlay-hudson.github.io/tapout/ Code: https://github.com/finlay-hudson/TAP-Out

License: http://creativecommons.org/licenses/by/4.0/

The gist: Tracking points that leave the camera's field of view (FoV) requires reasoning about unobserved scene dynamics, a fundamentally different problem from tracking visible correspondences.

Terminology

Abstract

Tracking points that leave the camera's field of view (FoV) requires reasoning about unobserved scene dynamics, a fundamentally different problem from tracking visible correspondences. We present TAP-Out, an offline method that makes this out-of-view (OOV) reasoning explicit by leveraging generative video outpainting as a world-model prior. Given a narrow-FoV input video, TAP-Out outpaints a complete 360° equirectangular video, locates target objects within the panoramic scene via video object segmentation, and synthesises object-centred perspective crops on which standard in-frame trackers operate. This decoupled design composes off-the-shelf foundation models without requiring task-specific architectural modifications. On the TAPVid360 benchmark, TAP-Out achieves state-of-the-art performance, improving OOV tracking accuracy by approximately 3x over the best prior method. We demonstrate its flexibility on out-of-frame object segmentation and detection and we provide detailed diagnostic analyses across motion regimes, static versus dynamic content, and hallucination quality, characterising when and why the approach succeeds or fails. Code is publicly available at https://github.com/finlay-hudson/TAP-Out

Sources

Related papers