Halluscoring 2026: The first shared task on llms hallucination detection and answer verification

arXiv:2609.38355 · cs.CL · Submitted 2026-09-29 · Read on arXiv

cs.CL

Submitted: 2026-09-29

Updated: 2026-10-01

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Hallucination — 114× in this paper · explained in 3 episodes
• Hallucination Detection — 56× in this paper · explained in 2 episodes
• Verification — 19× in this paper · explained in 2 episodes
• AUC — 18× in this paper · explained in 2 episodes
• Generalization — 15× in this paper · explained in 13 episodes
• Distillation — 13× in this paper · explained in 2 episodes
• Robustness — 9× in this paper · explained in 5 episodes
• Fine-Tuning — 8× in this paper · explained in 5 episodes
• Benchmark — 5× in this paper · explained in 2 episodes
• Knowledge Distillation — 5× in this paper · explained in 11 episodes
• Alignment — 3× in this paper · explained in 1 episode
• Contrastive Learning — 3× in this paper · explained in 5 episodes
• Parameter-Efficient Fine-Tuning — 3× in this paper · explained in 2 episodes
• Annotator Agreement — 2× in this paper · explained in 1 episode
• Calibration — 2× in this paper · explained in 7 episodes
• Distribution Shift — 2× in this paper · explained in 5 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Cross-Model Transfer — 1× in this paper · explained in 2 episodes
• Data Augmentation — 1× in this paper · explained in 7 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Generative AI — 1× in this paper · explained in 2 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Generative Models — 1× in this paper · explained in 2 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Model-Agnostic — 1× in this paper · explained in 2 episodes
• RAG — 1× in this paper · explained in 3 episodes
• Retrieval-Augmented Generation (RAG) — 1× in this paper · explained in 16 episodes
• Teacher-Student Distillation — 1× in this paper · explained in 2 episodes

Sources

• Evaluating Multilingual and Code-Switched Alignment in LLMs via Synthetic Natural Language Inference
• CrossHallu: Do Hallucination Signals Generalize Across Languages and Domains in Large Language Model's Internals?
• HalluScore: Large Language Model Hallucination Question Answering Benchmark
• Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gaps
• HalluTruthQA-4K: A Fine-Grained Corpus and Annotation Process for Arabic Hallucination Detection and Truth Verification
• HalluTruthQA: A Fine-Grained Benchmark for Hallucination Detection, Localization, and Explanation in Arabic Question Answering
• SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakes

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering