One note in three: a verified census of three deployed AI scribes, and the instrument that counted it

arXiv:2608.31017 · cs.CL, cs.AI, cs.CY · Submitted 2026-08-31 · Read on arXiv

cs.CL, cs.AI, cs.CY

Submitted: 2026-08-31

Updated: 2026-08-31

Comments: 47 pages, 3 figures. Dataset: https://huggingface.co/datasets/ComposoAI/OmissionBench . Code: https://github.com/composo-ai/omission-bench . Companion paper: "LLM Judges Verify Presence, Not Absence: Omission Blindness in AI Clinical Notes and What Recovers It"

Code: https://github.com/composo-ai/omission-bench

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Verification — 27× in this paper · explained in 2 episodes
• Clustering — 12× in this paper · explained in 2 episodes
• Provenance — 9× in this paper · explained in 3 episodes
• Abstention — 7× in this paper · explained in 1 episode
• Benchmark — 7× in this paper · explained in 2 episodes
• Generative AI — 5× in this paper · explained in 2 episodes
• LLM judge — 5× in this paper · explained in 2 episodes
• Accountability — 4× in this paper · explained in 2 episodes
• Accuracy — 4× in this paper · explained in 1 episode
• Calibration — 4× in this paper · explained in 7 episodes
• Hallucination — 4× in this paper · explained in 3 episodes
• Actor — 2× in this paper · explained in 3 episodes
• Benchmarking — 2× in this paper · explained in 3 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Redundancy — 2× in this paper · explained in 2 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Adversarial Model — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Failure Modes — 1× in this paper · explained in 3 episodes
• Failure Taxonomy — 1× in this paper · explained in 2 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• RAG — 1× in this paper · explained in 3 episodes

Sources

• Is GraphRAG Needed? From Basic RAG to Graph-/Agentic Solutions with Context Optimization
• Extrinsically-Focused Evaluation of Omissions in Medical Summarization
• Beyond Literal Summarization: Redefining Hallucination for Medical SOAP Note Evaluation

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering