Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content

Name: Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content
Uploaded: 2026-10-03T23:53:29.772167+00:00
Duration: 393 s
Description: Researchers used mechanistic interpretability to find attention heads responsible for toxicity classification and adversarial vulnerability in BERT and RoBERTa models on Jigsaw and ToxiGen datasets. They found that zeroing a single head can recover significant accuracy, revealing structural fairness gaps where vulnerability is unequally distributed across demographic groups.

Researchers used mechanistic interpretability to find attention heads responsible for toxicity classification and adversarial vulnerability in BERT and RoBERTa models on Jigsaw and ToxiGen datasets. They found that zeroing a single head can recover significant accuracy, revealing structural fairness gaps where vulnerability is unequally distributed across demographic groups.

Open the full episode