Global Consistency Repair for Natural-Language Claim Sets
cs.AI
Submitted: 2026-01-20
Updated: 2026-10-09
Comments: Accepted to EMNLP 2026 (Main Conference)
License: http://creativecommons.org/licenses/by/4.0/
The gist: NLP systems increasingly aggregate claims from retrieved passages, documents, and generated reasoning steps.
Terminology
Abstract
NLP systems increasingly aggregate claims from retrieved passages, documents, and generated reasoning steps. Yet current factuality methods mostly verify claims individually or compare pairs of claims, leaving open the question of whether the entire aggregated claim set can be retained coherently. This misses a fundamental failure mode: claims may be locally plausible and pairwise compatible while still being jointly inconsistent. We formalize global consistency repair: given an inconsistent natural-language claim set, preserve the largest coherent subset while localizing the compact sources of inconsistency. We introduce QXR, an adaptive diagnosis-and-repair framework that treats an LLM as a noisy set-level consistency judge, localizes minimal conflicting subsets, and repairs the claim set by deleting a small hitting set of claims. Across LLM judges and benchmarks derived from VitaminC, FEVER, and conflicting-evidence RAG, QXR preserves substantially more valid claims than direct all-at-once judging while maintaining high precision. These results suggest that factuality in claim aggregation systems should be treated as a set-level repair problem, not only as local verification.
Sources
- Enhancing Multi-Hop Fact Verification with Structured Knowledge-Augmented Large Language Models
- GraphCheck: Breaking Long-Term Text Barriers with Extracted Knowledge Graph-Powered Fact-Checking
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- A Survey on LLM-as-a-Judge
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
- Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models
- Halu-J: Critique-Based Hallucination Judge
- JudgeLM: Fine-tuned Large Language Models are Scalable Judges
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems