GROW squared: Grounding Which and Where for Robot Tool Use
GROW2 introduces a method for robot tool use that grounds affordances in open-world environments. It breaks down the task into semantic selection using vision-language models and geometric grounding using vision foundation models based on object parts. This allows the robot to generalize tool use across many objects without needing specific annotations.