Probing an Embodied LLM: When Higher Observation Fidelity Hurts Problem Solving
Researchers tested how different visual inputs affect an LLM agent solving a mechanical puzzle called Lockbox. They found that increasing observation fidelity did not improve performance; raw RGB input was best, and ground-truth labels were worst. Moderate noise in observations actually improved success by reducing repetitive action loops, suggesting measured performance reflects error interaction rather than pure reasoning.