What Do LLM Judges Actually Verify? Matched Controls and Record Gaps for Document Currency
cs.AI, cs.CL
Submitted: 2026-10-31
Updated: 2026-10-31
Code: https://github.com/unnatibshah/what-do-llm-judges-verify
Terminology
Sources
- Stale-Document Poisoning: When Outdated Retrieval Overrides Correct Model Answers
- A Judge Should Know What Changed:Construct Validity for LLM-as-a-Judge Evaluation
- A Dataset for Answering Time-Sensitive Questions
- Temporal Misgrounding in Legal RAG: A Versioned-Corpus Benchmark for French Tax Law
- VersionRAG: Version-Aware Retrieval-Augmented Generation for Evolving Documents
- Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence
- HALLMARK: How to Diagnose LLM Citation Verifiers and When to Deploy Them
- Judging Against the Reference: Uncovering Knowledge-Driven Failures in LLM-Judges on QA Evaluation
- Trusting the Tool: Asymmetric Deference of LLM Vulnerability Auditors to Static-Analysis Evidence
- Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
- Scoring-Rule Sensitivity in LLM Evaluation: Evidence from Indian Financial Regulatory Text
- Asking For An Old Friend: Diagnosing and Mitigating Temporal Failure Modes in LLM-based Statutory Question Answering
- JudgeBench: A Benchmark for Evaluating LLM-based Judges
- ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence
- Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
- Agents' Overreliance on Unreliable Tools
- Scalable Stewardship of an LLM-Assisted Clinical Benchmark with Physician Oversight
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents