SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agents
SubtleMemory is a new benchmark testing how long-running AI agents handle fine-grained relational memory. It introduces three relation types: Complementary (aggregating evidence), Nuanced (distinguishing similar facts under conditions), and Contradictory (recognizing conflicts). Findings show current systems struggle, especially with contradictory relations, highlighting the need for agents to preserve subtle relational details during long tasks.