Does Personalization Break Trust? Open-Set Rejection Calibration Under Few-Shot On-Device Intent Adaptation

Name: Does Personalization Break Trust? Open-Set Rejection Calibration Under Few-Shot On-Device Intent Adaptation
Uploaded: 2026-10-18T12:54:34.126901+00:00
Duration: 479 s
Description: Personalizing AI assistants using few-shot learning (LoRA) to learn new commands degrades their ability to correctly reject out-of-scope inputs, which is crucial for user trust. The study shows this degradation occurs across all tested benchmarks. A fix involving Mahalanobis distance scoring instead of standard confidence scores recovers most of this lost rejection quality with minimal overhead.

Personalizing AI assistants using few-shot learning (LoRA) to learn new commands degrades their ability to correctly reject out-of-scope inputs, which is crucial for user trust. The study shows this degradation occurs across all tested benchmarks. A fix involving Mahalanobis distance scoring instead of standard confidence scores recovers most of this lost rejection quality with minimal overhead.

Open the full episode