Not Until the Evidence Says So: Teaching LLM Investigators When to Close a Case

cs.CL, cs.AI, cs.LG

Submitted: 2026-10-02

Updated: 2026-10-02

Comments: 23 pages. Dataset: https://huggingface.co/datasets/etigerstudio/Nautil ; Models: https://huggingface.co/etigerstudio/Nautil-SFT , https://huggingface.co/etigerstudio/Nautil-RLVR ; Demo: https://huggingface.co/spaces/etigerstudio/Nautil-Demo ; Code: https://github.com/etigerstudio/Nautil

Code: https://github.com/etigerstudio/Nautil

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Sources

• JustDiag!: A Diagnostic Justification Engine for Accountable Root Cause Analysis
• Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning
• Holmes: An Evidence-Grounded LLM Agent for Auditable DDoS Investigation in Cloud Networks
• Quantifying and Mitigating Premature Closure in Frontier LLMs
• Why Language Models Hallucinate
• CaRT: Teaching LLM Agents to Know When They Know Enough
• AgentAbstain: Do LLM Agents Know When Not to Act?
• Agentic Abstention: Do Agents Know When to Stop Instead of Act?
• LLM-as-an-Investigator: Evidence-First Agentic Reasoning for Robust Interactive Problem Diagnosis
• Sequential Diagnosis with Language Models
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning
• MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reports

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering