Instance-anchored interaction evidence: Grounding robot plans in human pointing and handling
Instance-anchored interaction evidence (IAE) registers scene objects to public identifiers and tracks identities across video using backward mask propagation. It describes frames by hand/forearm geometry relative to these instances to ground robot plans in human pointing and handling, significantly improving plan success on implicit-intent tasks over existing vision-language models.