You Only Pass Once: Answering and Abstaining Together in a Single Forward Pass of a Frozen Language Model

summary

The gist

A frozen language model on reasoning tasks has two coupled weaknesses: it under-uses evidence that its own residual stream already encodes…

In short

The system YOPO combines answering, steering, and abstaining into a single forward pass of a frozen language model. It uses a conditional steering probe to improve reasoning accuracy and a zero-shot sufficiency direction to detect when context is insufficient, preventing confabulation. This approach outperforms two-pass methods while respecting deployment budgets.

Key concepts

Conditional Steering Probe
A small probe that writes the residual stream of the frozen model at specific mid-stack layers. It is trained on answer cross-entropy to guide the model's reasoning capabilities, boosting its performance by up to 21.7 percentage points on smaller models.
Sufficiency Direction (d)
A zero-shot direction that reads the residual stream and detects insufficient context far ahead of the model's own behavior. It is trained using labeled source examples to compute class means, allowing it to abstain when information is lacking, leading to high cross-task stability.
YOPO Architecture
The resulting system that answers, steers, and abstains in one forward pass. It uses a cost curve where supervised capacity (the boost) buys in-domain accuracy but is paid for in transfer debt. The architecture ensures sufficiency labels never enter the reconstruction objective.
Budget-Conditioned Recipe
A deployment strategy based on inference budget. If memory is tight, use the flagship one-pass system. If budget allows a second pass, use the clean read as a gate. For small models or high task shifts, fall back to the label-free map to manage transfer costs.

Terminology used across episodes

This episode discusses

The paper

You Only Pass Once: Answering and Abstaining Together in a Single Forward Pass of a Frozen Language Model · Read on arXiv

Ziyang Luo, Zhongyao Chu, Xinjie He, Youting Wang, Xukui Qin, Runxiong Wu, Yan-Syuan Chen

Gatech

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "You Only Pass Once".

Jane: Here's the gist of the paper. A frozen language model on reasoning tasks has two coupled weaknesses: it under-uses evidence that its own residual stream already encodes, and it fails to detect when the input is insufficient to answer, so it confabulates<ref:2608.14465#pg2>.

Tom: This paper consolidates two research lines—developed separately by its authors, reported here together as one system—that address these on the same residual stream<ref:2608.14465#pg2>.

Jane: The Core Problem and Solution A deployable reasoning system must do both: maximally extract what the context supports, and abstain when the context supports nothing<ref:2608.14465#pg3>.

Tom: The paper addresses two failures of frozen language models asked to perform abductive inference<ref:2608.14465#pg4>.

Title and authors: Tom: So Jane, this paper is called "You Only Pass Once: Answering and Abstaining Together in a Single Forward Pass of a Frozen Language Model". It sounds like they're trying to fix two big problems in frozen models at once.

Jane: It really does sound like that. The core idea is that these models have two weaknesses: first, they don't use all the evidence already stored in their residual stream, and second, they can't tell when the context they're given isn't enough information to give a good answer, so they just start making things up.

Lu: Yeah, and this paper says it solves both using one system. They build two research lines that work on the same thing: a conditional steering probe writes the residual stream at some middle layers, and a zero-shot sufficiency direction reads that stream to decide whether to answer or not.

Meng: So they're trying to get the model to do both tasks—reasoning and abstaining—in just one pass through a frozen backbone, right? What does that mean for deployment costs?

Tom: Well, it means we can potentially get both the reasoning accuracy up and the abstention capability in there without running two separate passes over the model. The title itself suggests this single-pass approach is key to their solution.

Jane: Exactly. And what's interesting is that they are looking at how these two parts interact during that single pass, which they call interference.

Lu: They show a small conditional steering probe writing the stream at mid-stack layers lifts abductive accuracy by up to twenty-one point seven percent on Qwen2 point 5-1 point 5B and four point nine six percent on the hardest full precision 7B wall, according to page four of this paper >

Meng: Twenty one point seven percent is a pretty big lift for such a small model. But what about the models that are already quite large?

Tom: It shows that even on those larger models, like the 7B variant, there's still room for improvement with this steering probe <ref:2608.14465#pg3>. It's not just about getting one thing right; it’s about how these two mechanisms work together.

The paper's summary: Jane: So to summarize what they're actually doing, the paper is presenting a fused system called YOPO, which stands for You Only Pass Once. It combines the steering probe and the sufficiency reader into one forward pass on a frozen Qwen2 point 5 backbone, which is one point five billion, three billion, or seven billion parameters depending on what you're testing >

Tom: Right. And the main contribution they highlight is this label-free correction of the residual stream that lets the two components share one forward pass and doubles end-to-end three way accuracy over just having a frozen baseline >

Lu: The technical core is learning a small map, let's call it M, that removes the steering perturbation so it can reconstruct the clean sufficiency layer residual from what the steered residual is >

Meng: So this map M doesn't have to be trained on any labels for this correction part, which is interesting because they say that means nothing about when to abstain on something specific like NLI tasks can leak into that reconstruction process >

Jane: That’s a crucial point. The sufficiency direction d, which reads the stream and detects insufficient context, is fixed—never retrained—which preserves its transfer capabilities across different tasks >

Tom: So they're essentially using one forward pass to get a reasoning writer and a trustworthy abstention reader working together because the architecture respects this asymmetry where the abstention stays as a calibrated read, but you train away the perturbation that the write leaves on the read >

Lu: That's what page three of this paper says, and it shows that they compare three different one pass fusion architectures on a shared injection module to show how severe the interference is depending on what protocol you follow >

Meng: How does all this single pass system actually translate into something useful for a deployable AI? Does it solve the cost issue they are worried about?

Jane: It aims to provide an answer, steer, and abstain capability in one forward pass at no accuracy cost compared to the two pass alternative >

The paper's improvements: Tom: Okay, let's talk about what they actually improved over existing methods. They have a lot of metrics here. For instance, they show that their one-pass system beats the two pass reference at every scale, like zero point seven nine eight versus zero point seven five three on Qwen2 point 5-1 point 5B >

Jane: That’s a consistent improvement across different sizes and model families. They're looking at ten backbones across six model families and getting ten to ten results, which is pretty comprehensive testing > <ref:2608.14465#pg1>

Lu: What’s really interesting is the contamination follows a scale law—the clean versus steered transfer gap falls from zero point zero eight two down to zero point zero three one across the models one point five billion, three billion, and seven billion >

Meng: That scale law sounds promising for deployment because it suggests that as you go bigger, the gap between what's clean and what's steered gets smaller >

Tom: It actually inverts at the 7B level where the steered read becomes better than the clean one, with a result of zero point nine nine five versus zero point nine eight five on Qwen2 point 5-7B >

Jane: And they tie it up by saying that this steers read is already better for the read substrate at 7B, and the identity initialized correction learns to do nothing there > <ref:2608.14465#pg3>

Lu: The design law, conditioned on scale and located at the write, survives as an architecture rule rather than a prohibition >

Meng: So what's the practical implication for my pipeline? If I'm shipping models to production, is this something I should be doing or not? Does it add too much complexity?

Tom: It’s a budget-conditioned recipe. If you can afford a second partial forward pass, the clean read is still the transfer optimal gate >

Jane: But under a hard one pass budget, their flagship configuration—which is the multi layer de contamination plus boost—is actually the score leader in domain accuracy at every scale and on transfer at three billion and above >

Conclusion: Tom: So to wrap up, this paper on "You Only Pass Once: Answering and Abstaining Together in a Single Forward Pass of a Frozen Language Model" shows that we can get an answer, steer, and abstain capability in one forward pass with no loss compared to the two pass alternative >

Jane: The main implication is that this system provides a budget conditioned recipe for deployment—you use the flagship configuration when inference budget is tight or if you can afford a second partial forward pass >

Lu: And under those hard one pass budgets, their default is the flagship, which gets the highest in domain accuracy on every scale and transfer at three billion and above >

Meng: If things get small-scale, like on a one point five billion model, you should probably fall back to the purely label free map when dealing with worst case task shifts because that's where the boost has a real cost in transfer debt >

Tom: That makes sense. It’s about choosing the right tool for the job based on what you can afford to run.

Jane: It really is about finding an architecture rule rather than a prohibition, and I think this work on "You Only Pass Once" gives us a solid framework for how to build these kinds of systems.

Lu: It's interesting because they show that the residual stream can host both a reasoning writer and a sufficiency reader in one forward pass only if the architecture respects one asymmetry >

More episodes

← Home