UniffordInteractor: Reasoning the Unified Affordance for Human and Multi-Object Interaction
cs.GR
Submitted: 2026-10-29
Updated: 2026-10-29
Comments: 13 pages, 11 figures. Published in ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2026)
DOI: 10.1145/3842530
License: http://creativecommons.org/licenses/by/4.0/
The gist: Deriving plausible and dynamic Human Object Interaction (HOI) has gained tremendous progress in Embodied AI and Immersive AR/VR.
Abstract
Deriving plausible and dynamic Human Object Interaction (HOI) has gained tremendous progress in Embodied AI and Immersive AR/VR. While existing HOI methods have effectively explored human interaction with individual or fixed objects, they still show limitations in complex multi-object interaction scenes. Meanwhile, previous approaches focus more on the contact information held by human-object relation, overlooking the significance of the contact situation of various entities (i.e., object-object). To address these limitations, we propose UniffordInteractor, a Reasoning-then-Interacting framework that leverages unified affordance for realistic and long-horizon human and multi-object interaction modeling. At the reasoning stage, we devise a Unified Affordance Reasoner (UAR) that employs a fine-tuned 3D Multimodal Large Language Model for unified object-centric affordance prediction. At the interacting stage, we propose Hierarchical Interaction Generator (HIG), which formulates the interaction generation as a hierarchical (e.g., root to body to hand to objects) autoregressive prediction, conditioned on the obtained unified affordance. Extensive experiments demonstrate that UniffordInteractor surpasses SOTA counterparts in human and multi-object interaction generation.
Related papers
- AtlasLC: Fast Codec-Ready Compression of Object-Centric 3D Gaussian Splatting
- Velocity-Space 3D Asset Editing
- Physics-Driven Extraction of Dynamic Fluid Asset from Monocular Video with Generative 3DGS
- SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-Based Humanoid Control
- Delaunay-Based Nearest Neighbor Search with Grid-Accelerated Queries and Parallel Construction
- Macrofacet Theory for Gaussian Process Statistical Surfaces