NxN E-valuation: Hypothesis Certification via a Conformal CRT Null
cs.AI
Submitted: 2026-08-06
Updated: 2026-10-26
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: We propose NxN E-valuation, a handy, e-value-based hypothesis-certification algorithm that lets a hypothesis be verified without building any case-specific certification procedure---such as
Terminology
Abstract
We propose NxN E-valuation, a handy, e-value-based hypothesis-certification algorithm that lets a hypothesis be verified without building any case-specific certification procedure---such as constructing a dedicated null hypothesis---as long as a large enough dataset is available. The method is especially suited to LLM-based exploration systems, where LLMs are remarkably good at proposing hypotheses but suffer badly from hallucination; this hallucination prevents us from harvesting LLM outputs directly, and existing remedies each fall short. The most common solutions include letting the LLM verify or correct itself circular verification and held-out testing (where false hypotheses can still pass via spurious correlations), among other remedies detailed in the introduction. To resolve this, NxN E-valuation exploits the naturally existing large training set and lets different samples serve as null hypotheses for one another. This design directly realizes a conditional randomization test (CRT) that certifies each hypothesis. The approach can be a universally better replacement for at least LLM circular verification and held-out-data testing, provided the LLM's generations are hypotheses that apply to each individual sample.
Sources
- A Survey on Hypothesis Generation for Scientific Discovery in the Era of Large Language Models
- Grounding LLM Reasoning with Knowledge Graphs
- BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model
- Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectification
- Unifying approach to selective inference with applications to cross-validation
- OneReason Technical Report
- The Clever Hans Mirage: A Comprehensive Survey on Spurious Correlations in Machine Learning
- Attribution Techniques for Mitigating Hallucinated Information in RAG Systems: A Survey
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems