Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Tasks
This work evaluated how input perturbations affect Vision-Language-Action (VLA) models during robot manipulation tasks, moving beyond simple task success rates. The study found that successful trajectories are not robust to these changes; perturbations alter the execution behavior of successful paths. This means task success alone does not capture how a robot actually performs a successful action.