Train on prefill, monitor at decode: Using sparse autoencoders for topic monitoring
cs.AI, cs.LG
Submitted: 2026-11-06
Updated: 2026-11-06
Terminology
Sources
- kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrail
- Understanding intermediate layers using linear classifier probes
- An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
- Probing Classifiers: Promises, Shortcomings, and Advances
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detection
- Sparse Autoencoders Find Highly Interpretable Features in Language Models
- Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreaks
- Sparse Autoencoder Features for Classifications and Transferability
- Gemma 3 Technical Report
- Sparse Autoencoders are Topic Models
- Detecting Strategic Deception Using Linear Probes
- BERTopic: Neural topic modeling with a class-based TF-IDF procedure
- WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
- Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
- Predict, Don't React: Value-Based Safety Forecasting for LLM Streaming
- Text Segmentation as a Supervised Learning Task
- Fence: Specialized SLM Guardrails for LLM Applications
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents