Auditable Release Control for Pedagogical Leakage in LLM Tutors
Nizam Kadir
cs.CR, cs.AI, cs.CL
Submitted: 2026-08-01
Comments: 9 pages, 1 figure, 6 tables. Preprint; not peer reviewed
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language model tutors can be correct and helpful yet disclose an answer or decisive reasoning before that disclosure is authorized.
Terminology
Abstract
Large language model tutors can be correct and helpful yet disclose an answer or decisive reasoning before that disclosure is authorized. We formalize this state- and action-dependent failure as pedagogical leakage and introduce an authorization-aware complete-mediation boundary. A selector emits one of five disclosure contracts, trusted policy gates privileged modes, and a renderer proposes language. A single release function applies inspectable checks, optional cumulative verification, and action-specific fallback; replayable traces separate selection, generation, verification, and enforcement failures. Matched component attribution exposes a safety-utility frontier. On 599 fixed Gemini 3.5 proposals, strict mediation reduces blinded three-model panel-majority leakage flags from 181 to 0 (paired problem-cluster difference-30.22 points, 95% CI [-35.00,-25.72]), while replacing 581 responses and lowering helpfulness. Checker-triggered fallback alone yields 11 majority flags; adding the semantic verifier yields 14 and no reliable marginal gain. A global A1 scaffold yields 0 majority and 54 any-judge flags, outperforming fitted Q on automatic safety and utility. In an externally timestamped replication over 40 unseen problem clusters and 480 attack sequences, high-assurance release reduces majority flags from 42 to 8 (-7.08 points, 95% CI [-13.13,-2.29]); seven failures persist, one is introduced, and mean helpfulness falls by.192. These results establish an auditable release boundary and failure attribution under declared contracts, not universal semantic safety or learning gains.
Sources
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
- Training Verifiers to Solve Math Word Problems
- SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems
- LearnLM: Improving Gemini for Learning
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
- Training LLM-based Tutors to Improve Student Learning Outcomes in Dialogues
- TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Models
- The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialogues
- Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models
Related papers
- SoK: AI-Augmented Binary Reversing
- Relaxed Sender Anonymity for CBDC Interbank Settlement: A Zero-Knowledge Approach on Permissioned EVM
- Calibration-Family Overfit: Why Trusted Sabotage Monitors Don't Transfer Across Lineages
- Estimating the Social Cost of Corporate Data Breaches
- Efficient Fuzzy PSI under One-Sided Assumptions
- Sealing the Audit-Runtime Gap for LLM Skills