RAS: Measuring LLM Safety Through Refusal Alignment
summary
The gist
Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy.
In short
SafeVec is a white-box method to measure LLM safety by analyzing internal representations instead of generated outputs. It extracts a 'refusal direction' from safe models and scores target models based on how well their hidden states align with this refusal direction. The resulting Refusal Alignment Score (RAS) provides an efficient, calibrated 0-100 metric to distinguish aligned models across different LLM families like Llama and Gemma.
Key concepts
- SafeVec
- A white-box procedure for measuring safety by analyzing the internal hidden states of a model during a harmful prompt. It works by extracting a 'refusal direction' from a safe model and scoring target models based on how their internal states move along that refusal path when processing unsafe or jailbreak prompts.
- Refusal Direction Extraction
- The first stage of SafeVec where the direction of refusal is calculated for each layer. This is done by finding the difference between the hidden state vectors of a model under an unsafe prompt and its safe counterpart, creating a vector that represents how a model should internally react to harmful requests.
- RAS (Refusal Alignment Score)
- The final calibrated safety metric, mapped to a 0-100 scale. It measures the alignment between a target model's internal representations and the established refusal direction. A high RAS score indicates stronger internal alignment with safety protocols, suggesting better refusal behavior.
- Architecture-Specific Severity Slope
- A factor used in the RAS calculation to account for differences between LLM families. It adjusts the severity of a normalized refusal drop based on how much attack success rates (ASR) increase for a specific architecture, helping to correct for varying internal scales and refusal geometries across models.
Terminology used across episodes
This episode discusses
The paper
RAS: Measuring LLM Safety Through Refusal Alignment · Read on arXiv
National Yang Ming Chiao Tung University · Hon Hai Research Institute
Repeated safety evaluation of open-weight large language models requires generating and judging responses for many related checkpoints. We introduce RAS (Refusal Alignment Score), a representation-based, reference-relative indicator of harmful-request refusal computed by the five-stage SafeVec procedure. SafeVec extracts layer-wise refusal directions from an aligned reference model, measures their alignment in related targets, and calibrates a score from 0 to 100 using behavioral attack success rates (ASR) from reference and calibration models. Subsequent target scoring uses input forward passes alone. Across 18 held-out variants from the Llama, Gemma, and Qwen families, RAS correlates with 100(1-ASR) (Pearson 0.934, Spearman 0.851) and achieves 83.0% pairwise ordering agreement. Across five prompt configurations for the Llama reference, directions have mean cosine similarity 0.939. With calibration ASR already available, setup-inclusive evaluation of five targets achieves a speedup of 3.2 to 80.5 times over the specified behavioral protocol. These results support refusal alignment as an efficient signal for screening related checkpoints and prioritizing behavioral audits.
Transcript
Introduction to the show: ident: Security Radio. Generated commentary on the latest security and cryptography papers.
Nadia: We're looking at the paper: "RAS: Measuring LLM Safety Through Refusal Alignment".
Elias: Here's the gist of the paper. Safety evaluation of large language models (LLMs) is commonly performed by querying models with unsafe or jailbreak prompts and judging whether their outputs violate a safety policy.
Nadia: We propose SafeVec, a white-box evaluation procedure that measures safety from internal representations rather than generated outputs.
Elias: The resulting metric, RAS (Refusal Alignment Score), maps representation-level refusal alignment to a calibrated 0–100 safety score.
Nadia: Across Llama, Gemma, and Qwen families, RAS separates aligned models from uncensored and abliterated variants.
Title and authors: Nadia: So, let's talk about who wrote this and what the title actually means when you read "RAS: Measuring LLM Safety Through Refusal Alignment." It seems like the core idea is using internal states to see if the model is aligned with a refusal direction.
Elias: Right. Chang-Chieh Huang, Yan-Lun Chen, Chia-Mu Yu, and Wei-Bin Lee are the authors; they bring different expertise here—they have security researchers, cryptographers, and measurement researchers all involved in this work.
Priya: It's good to see that mix because it suggests they're thinking about both the security aspect of exploitation and the measurement aspect of how we actually interpret those internal states.
Nadia: So what's the big picture here? It’s about creating a way to score safety using internal representations, which they call Refusal Alignment Score, or RAS.
Elias: And that RAS score maps this representation-level alignment onto a calibrated number from zero to one hundred, specifically designed to compare models across different families like Llama, Gemma, and Qwen.
The paper's summary: Nadia: So the paper explains that SafeVec is a five stage procedure. It starts with extracting those refusal directions from a safe reference model and then scores the target model based on how its hidden states match those directions under both unsafe and jailbreak prompts.
Elias: And they detail how they build that signal, starting with calculating layer-wise refusal directions, which involves taking the difference between the mean activations of unsafe versus safe prompts at each layer.
Priya: I see that means it's really looking at how much a model's internal representation shifts in a way that suggests refusal when it sees harmful input. That’s a lot of data to process for one evaluation run.
Nadia: And they also have this calibration model scoring step where they use cosine similarity between the target model's hidden states and the raw scores from those prompts to get the initial RAS values.
Elias: The crucial part is that they then apply a RAS calibration step, using a small set of models with known safety levels—aligned, uncensored, and abliterated variants—to map those raw cosine similarities onto that final zero to one hundred safety score <ref:2606.25750#pg1>.
The paper's improvements: Nadia: So what are the real advantages they're pushing for? They argue that this procedure is substantially faster than judge-based evaluation because it only requires forward passes through the target model and doesn't need long-form generation or external judges.
Elias: That speedup is quite significant; they report an average speedup of two thousand one hundred and three times compared to the slowest judge methods. It’s not just a little bit quicker; it’s a major efficiency gain for auditing.
Priya: I'm interested in how this relates to what the paper claims about actual behavior, because if we can measure internal alignment, does that actually predict whether the model will fail an attack?
Nadia: Yes, they show that RAS correlates strongly with output-level safety behavior; models that score high on RAS generally have low attack success rates. So it suggests this representation signal already reflects real differences in output safety before you even look at the generated text.
Elias: They also address the issue of generalization across different model families by defining an architecture specific severity slope and a global severity slope, which helps account for differences in how activation scales vary between Llama, Gemma, and Qwen.
Conclusion: Nadia: To wrap this up, RAS gives us a representation-level safety metric that's efficient for white-box auditing and posttraining comparison. A high RAS score means stronger refusal alignment based on internal states.
Elias: It’s useful for fast white-box auditing but the authors are careful to state that it doesn't prove the model will refuse every single harmful request; it measures alignment with a reference direction, not guaranteed refusal on any input.
Priya: That limitation is important because we have to remember that if future models use completely different internal mechanisms for safety, they might need to re-estimate or extend that initial refusal direction.
Nadia: So the overall implication is that RAS is a solid tool for comparing model variants internally and quickly assessing safety posture without relying on slow, external judgment calls.
Elias: We have to remember it's calibrated within model families, so when you compare a Llama model to a Qwen model, you need those calibration models for the cross-family comparison to be reliable.
Priya: It’s helpful for getting started with auditing, but the ethical statement is key: we shouldn't just use this to ignore vulnerabilities; it should be part of a pipeline that includes behavioral testing and human review.
Nadia: Exactly. So, in short, this paper on "RAS: Measuring LLM Safety Through Refusal Alignment" gives us a fast way to audit internal alignment across Llama, Gemma, and Qwen using representation data instead of expensive output generation tests.
More episodes
- 2608.27496-ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection
- Daily Summary for 2026-11-19
- 2607.07085-Is Randomness Necessary for Adaptive Data Analysis?
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2609.10601-Threshold Choice, Not Sample Size, Bounds Trustless Verification of Nondeterministic Compound AI Workflows
- 2610.38786-Adversarial Process Disturbances at a Deployable Alarm Budget: Detector Evaluation on a Simulator and a Real Plant
- 2511.08660-Binary and Multiclass Cyberattack Classification on GeNIS Dataset
- 2604.27302-Static Attribution of Android Residential Proxy Malware Using Graph Kernels
- Daily Summary for 2026-11-18