WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales
The episode discusses 'WATCH,' a monitoring system for AI deployments that acts as a 'watchdog.' Developed by Prinster et al., WATCH helps determine if an AI model is failing due to dangerous concept shifts or benign data changes. It provides both detection and diagnosis, offering a practical tool for high-stakes fields like healthcare.