Time Present and Time Past: Benchmarking Large Language Models on Temporally Evolving Document Understanding
cs.AI
Submitted: 2026-08-09
Updated: 2026-08-25
Code: https://github.com/icsetepa44/TIDE
License: http://creativecommons.org/licenses/by/4.0/
The gist: Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates.
Terminology
Abstract
Evolving documents, such as laws, tax codes, and software documentation, are amended, replaced, and sometimes reverted over time, so a question has different correct answers at different dates. In contrast to encyclopedic knowledge, where an old fact is simply overwritten, an amendment is itself an official text that states what it replaces and when it takes effect, and the earlier version stays correct for its validity period. The central challenge is therefore version resolution, that is, identifying the version in force on the queried date. Existing temporal QA datasets treat time only as an annotation, so version resolution stays untested. We present TIDE, an expert-verified benchmark of 3,050 QA pairs over 644 official customs instruments issued between 1969 and 2025 by the Government of Bangladesh, covering eight task types over deeply code-mixed documents that are heterogeneous in layout and dated in two calendars. In addition, we evaluate nine recent LLMs under a single protocol across parametric, gold-context, and retrieval access, scored by a three-judge LLM council with a hard date gate separating correct meaning from correct time. The best macro-averaged accuracy is only 68.5%. Resolving a version from an implicit date reaches 59.7%, and detecting that the supplied version does not govern the query reaches only 26.7%. Models are more likely to find correct versions than to reject incorrect ones, and they tend to follow a confident parametric answer over the supplied authoritative text. All code and data are available at https://github.com/icsetepa44/TIDE
Sources
- XOR QA: Cross-lingual Open-Retrieval Question Answering
- Dated Data: Tracing Knowledge Cutoffs in Large Language Models
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
- StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models
- VersionRAG: Version-Aware Retrieval-Augmented Generation for Evolving Documents
- DyKnow: Dynamically Verifying Time-Sensitive Factual Knowledge in LLMs
- TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models
- When Facts Change: Probing LLMs on Evolving Knowledge with evolveQA
- When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation
- gpt-oss-120b & gpt-oss-20b Model Card
- Harnessing Temporal Databases for Systematic Evaluation of Factual Time-Sensitive Question-Answering in Large Language Models
- Gemini Embedding: Generalizable Embeddings from Gemini
- EvoWiki: Evaluating LLMs on Evolving Knowledge
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
- Gemma 4 Technical Report
- Kimi K2.5: Visual Agentic Intelligence
- TRAM: Benchmarking Temporal Reasoning for Large Language Models
- Qwen3 Technical Report
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems