Geo-Spatial Concept Probing of Large Language Models: Abstraction, Compositionality, and Grounding
This episode reviews a paper testing whether large language models truly understand spatial concepts like direction, distance, and topology. The authors built a benchmark using UK wards, generating questions from real geometry. Results show moderate accuracy but poor consistency, weak abstraction, and failed grounding, suggesting models often pattern-match rather than genuinely represent concepts.