Watch the Model Think: On-Policy Extraction of Activation Steering Vectors

Name: Watch the Model Think: On-Policy Extraction of Activation Steering Vectors
Uploaded: 2026-10-03T14:00:03.196112+00:00
Duration: 319 s
Description: ROAST estimates control vectors for large language models during inference by using a three-stage process. It first generates contrastive pairs from model rollouts to find directions aligned with the model's natural distribution, then uses Continuous Soft Scaling to avoid information loss from masking, and finally employs Grouped Mean Normalization to create a robust final steering vector.

ROAST estimates control vectors for large language models during inference by using a three-stage process. It first generates contrastive pairs from model rollouts to find directions aligned with the model's natural distribution, then uses Continuous Soft Scaling to avoid information loss from masking, and finally employs Grouped Mean Normalization to create a robust final steering vector.

Open the full episode