Do AI chatbots find what experts would? Effects of model, user role, and sample size on study retrieval for medical questions
Researchers tested three advanced AI chatbots (Claude Sonnet 5, Gemini 3.1 Pro, and GPT-5.5) answering clinical questions based on Cochrane review topics using different user roles (patient, clinician, researcher). Findings showed ChatGPT had the best recall, but retrieval success was primarily predicted by the sample size of the cited studies rather than other factors.