Can MLLMs Find Their Way in a City? Exploring Emergent Navigation from Web-Scale Knowledge
Researchers tested multimodal large language models (MLLMs) for complex city navigation using a benchmark called CityNav, which requires agents to navigate 50+ decision points based only on visual input. They introduced Verbalization of Path (VoP) and multi-component memory systems to ground the MLLMs' internal reasoning. Results showed that these methods significantly improved agent success rates over base models, proving MLLMs can leverage world knowledge for sequential decision-making in real environments, though limitations remain in handling complex graph inconsistencies.