ChaosBench-Logic: A Benchmark for Logical and Symbolic Reasoning on Chaotic Dynamical Systems
ChaosBench-Logic introduces a benchmark to test how large language models reason about complex chaotic dynamical systems using formal first-order logic. It found that while models are accurate on individual facts, they fail at maintaining consistent global reasoning and compositional tasks. This reveals a weakness in their ability to build and uphold a coherent world model necessary for scientific understanding.