Guideline-as-Oracle: Zero-Annotation Training of an Ophthalmic Telephone Triage Agent

arXiv:2608.04772 · cs.CL, cs.AI · Submitted 2026-08-18 · Read on arXiv

Chenyu Wang, Yi Liu, Baoqing Li, Min Tu, Diping Song

cs.CL, cs.AI

Submitted: 2026-08-18

Updated: 2026-08-19

Code: https://github.com/tatsu-lab/stanford_alpaca

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Benchmark — 16× in this paper · explained in 2 episodes
• Accuracy — 9× in this paper · explained in 1 episode
• GRPO — 6× in this paper · explained in 6 episodes
• American Academy of Ophthalmology — 4× in this paper
• Augmentation — 3× in this paper · explained in 2 episodes
• Fine-Tuning — 3× in this paper · explained in 5 episodes
• Grounding — 3× in this paper · explained in 3 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Generalization — 2× in this paper · explained in 14 episodes
• Generator — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Harness — 1× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 1× in this paper · explained in 4 episodes
• Localization — 1× in this paper · explained in 2 episodes
• POMDP — 1× in this paper · explained in 2 episodes
• Papers — 1× in this paper · explained in 3 episodes
• Provenance — 1× in this paper · explained in 5 episodes
• Pruning — 1× in this paper · explained in 4 episodes
• Robustness — 1× in this paper · explained in 6 episodes
• Supervised Fine-Tuning (SFT) — 1× in this paper · explained in 13 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• Constitutional AI: Harmlessness from AI Feedback
• From Promising Capability to Pervasive Bias: Assessing Large Language Models for Emergency Department Triage
• Sequential Diagnosis with Language Models
• Capabilities of GPT-4 on Medical Challenge Problems
• AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Towards Conversational Diagnostic AI
• Benchmark Data Contamination of Large Language Models: A Survey
• MedResearcher-R1: Expert-Level Medical Deep Researcher via A Knowledge-Informed Trajectory Synthesis Framework
• Don't Make Your LLM an Evaluation Benchmark Cheater

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering