Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models
cs.CL, cs.LG
Submitted: 2026-04-02
Updated: 2026-11-07
Code: https://github.com/ayushjhaveri/llm-confirmation-bias
Terminology
Sources
- An AI system to help scientists write expert-level empirical software
- WILT: A Multi-Turn, Memorization-Robust Inductive Logic Benchmark for LLMs
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- Language models show human-like content effects on reasoning tasks
- A Rational Analysis of the Speech-to-Song Illusion
- What is a Number, That a Large Language Model May Know It?
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- Unveiling Confirmation Bias in Chain-of-Thought Reasoning
- Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
- The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
- Qwen3 Technical Report
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data
- BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation
- Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking