ChainCheck: Near-Perfect Evidence-Sufficiency Scores Can Conceal Sensitivity to Construction Artifacts -- Paired Conditional Contrasts that Expose an Evaluation Confound
cs.AI
Submitted: 2026-11-09
Updated: 2026-11-09
Terminology
Sources
- Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts
- Benchmarking Large Language Models in Retrieval-Augmented Generation
- Understanding Dataset Design Choices for Multi-hop Reasoning
- The Power of Noise: Redefining Retrieval for RAG Systems
- Evaluating Models' Local Decision Boundaries via Contrast Sets
- Shortcut Learning in Deep Neural Networks
- Annotation Artifacts in Natural Language Inference Data
- Designing and Interpreting Probes with Control Tasks
- Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
- LoRA: Low-Rank Adaptation of Large Language Models
- Avoiding Reasoning Shortcuts: Adversarial Evaluation, Training, and Model Development for Multi-Hop QA
- Sufficient Context: A New Lens on Retrieval Augmented Generation Systems
- Learning the Difference that Makes a Difference with Counterfactually-Augmented Data
- How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks
- Entity-Based Knowledge Conflicts in Question Answering
- Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
- Compositional Questions Do Not Necessitate Multi-hop Reasoning
- Hypothesis Only Baselines in Natural Language Inference
- Measuring and Narrowing the Compositionality Gap in Language Models
- Know What You Don't Know: Unanswerable Questions for SQuAD
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents