Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results

arXiv:2607.11183 · cs.CL, cs.AI, cs.LG · Submitted 2026-07-13 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Amplitude-Only FFN Intervention for Tool-Structured LLM Inference".

Jane: The gist The Amplitude Gating (AG) method improves tool-structured LLM inference by preserving pretrained FFN weight directions and modulating activation magnitudes during decoding,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're wrapping up this paper "Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results". The title itself suggests they are focusing only on the amplitude—the magnitude of activation—without changing the core direction of the weights.

Jane: And what does that mean for us when we think about making these agents more reliable? It seems like they’ve found a way to intervene during decoding that respects the pretrained knowledge while trying to mitigate specific errors in tool use.

Lu: The authors are being careful not to overstate their findings, emphasizing that the strongest current selector evidence comes from exploratory offline results on Qwen3 point 5 tool-structured inference rather than prospective online A/B validation <ref:2607.11183#pg1>.

Meng: From a production standpoint, this means we can't just deploy this as a universal quality booster yet because you still need a good candidate generator and an acceptance policy that can retain the baseline if the intervention goes wrong.

Tom: Exactly. They are not claiming it solves everything; they’re providing an evidence bounded framework where you have to look at how classical effectiveness and PFHE safety and selector validity are reported separately. It’s about targeted intervention, not a blanket improvement for every single thing the model does.

Conclusion: Tom: So, we’re wrapping up this chat about Amplitude Gating, or AG for short. The authors wrote this paper titled "Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results."

Jane: It seems like they’ve spent a lot of time looking at how to tweak the internal workings of these large language models during the moment they are trying to use tools.

Lu: Specifically, they focus on Amplitude Gating, which basically masks where the model is activating inside its own network while keeping all those massive pretrained weights exactly as they are.

Meng: So it’s an inference-time trick to see if we can guide the AI's decision-making without having to retrain the whole thing?

Tom: Exactly. The core idea is that tool failures often happen because of a last-mile transformation error, and this method intervenes by clipping or gating those activation magnitudes during decoding. It’s very conservative, keeping the useful parts of prior work while avoiding some bigger changes.

Jane: I see them talking about these six different SwiGLU sites—those are like different points where they can apply this intervention across the network structure.

Lu: They introduce a taxonomy with things like Hard AG and Soft AG, which are just different ways to choose how much of that activation signal to retain, using concepts like L2 energy or tau shrinkage.

Meng: But what’s the actual payoff? Does this make the model reliably use tools on a real website?

Tom: The results they show are mostly exploratory. They found positive trends on Qwen3 point five-9B tool routes and Hermes function calls, showing gains like plus two point six percentage points for certain endpoints under frozen local text scorers.

Jane: But there’s a big caveat they mention—the JSON gains were small, and the L-Eval effects were actually negative. It’s not a universal fix yet.

Lu: They are very clear that the strongest current evidence comes from offline results on Qwen3 point five and they flag that it lacks the statistical significance needed for broad deployment right now.

Meng: So what does this mean for a real product? Is this something we can just plug in and expect better tool use across the board?

Tom: Not at all. They stress that successful deployment needs a good candidate generator, a valid selector, and an acceptance policy that can still retain the baseline if the intervention messes up.

Jane: It’s less about making it perfect and more about providing a safe surface for intervention where you can measure the harm versus the gain.

Lu: They also use this protocol called PFHE, which tracks four different types of outcomes: correction, harm, preserved correct, and preserved wrong. That helps separate what actually works from what just looks good on paper.

Tom: So it’s an evidence-bounded framework for FFN intervention right now. It shows where the causal mechanisms might be hiding but tells us we still have a lot of work to do before we call this a universal quality booster.

Alibaba Cloud 2 University of Toronto

cs.CL, cs.AI, cs.LG

Submitted: 2026-07-13

Updated: 2026-10-15

Comments: 48 pages, 11 figures, 29 tables. Revised abstract, introduction, and methods; expanded figures; clarified intervention-site definitions, baseline variation, and RF results

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 77/100

The gist: The gist The Amplitude Gating (AG) method improves tool-structured LLM inference by preserving pretrained FFN weight directions and modulating activation magnitudes during decoding, showing positive

Key concepts

Amplitude Gating (AG)
AG is an inference intervention that masks FFN activation coordinates by clipping or gating their magnitudes during decoding. It preserves the direction of pretrained weights while controlling how strongly those activations influence the output, aiming for a conservative and effective change.
SwiGLU Sites
These are six specific locations within the SwiGLU-style gated FFN architecture where AG is applied. These sites separate inputs, gate/value channels, and the final FFN output. The method uses these points to apply fine-grained control over different parts of the network's computation.
Per-Sample Fix-Harm Evaluation (PFHE)
PFHE is a protocol that evaluates AG by reporting four outcomes for every sample: Fix 0/1 (AG corrects an error), Harm 1/0 (AG breaks success), Preserved correct 1/1, and Preserved wrong 0/0. This measures the intervention's effect on native partial-credit magnitudes, providing a more nuanced view than simple accuracy metrics.
Tool-Structured Failures
These are errors in LLM inference that occur specifically when the model is tasked with using external tools. The paper suggests that these failures often resemble 'last-mile transformation errors,' making the FFN an ideal place to apply conservative interventions like AG.

Terminology

Summary

The gist The Amplitude Gating (AG) method improves tool-structured LLM inference by preserving pretrained FFN weight directions and modulating activation magnitudes during decoding, showing positive exploratory results on Qwen3.5-9B tool-route micro-average and Hermes function-call endpoints, while emphasizing that deployment requires a combination of candidate availability, selector validity, and acceptance policy >

How it works

The core intuition behind the method is that tool-structured failures often look like last-mile transformation errors, making the FFN a natural place to seek a conservative intervention surface > The Amplitude Gating (AG) method operates by preserving pretrained FFN weight directions and instead clips or gates activation magnitudes during inference > This approach is deliberately conservative, keeping the useful insight from earlier work while avoiding the destructive part of Orthogonal Residual Projection (ORP), which involved changing matrix or subspace directions > AG separates the process into four distinct operations: candidate generation, ranking, acceptance or fallback decision, and baseline fallback >

The method is instantiated on SwiGLU-style gated FFNs, utilizing a fine-grained intervention system that includes inherited positions P1 and P3 from ORP > It also introduces refined positions P1s, P2a, and P2b that separate gate-branch, value-branch, fused, and output-side signals specific to the SwiGLU architecture > The six-point taxonomy in this paper is the SwiGLU instantiation of that principle across FFN input, internal nonlinear activation, and output sites >

Key Components and Protocols

The AG control path is defined by four distinct operations: candidate generation, candidate selection, acceptance or abstention, and baseline fallback > The method uses an entropy-based AG family where the parameter e denotes retained mass for hard/soft/NMF-style gates > The taxonomy includes several variants such as Hard AG (L2-energy gate), Soft AG (L2-derived tau shrinkage), Top-k AG (fixed top-k magnitude), Local-norm AG (local standard deviation threshold), and NMF-style AG, which treats positive and negative activation components separately >

A critical component is the Per-Sample Fix-Harm Evaluation (PFHE) protocol, which complements classical task metrics by reporting four mutually exclusive correctness transitions: Fix 0/1 (AG corrects a baseline error), Harm 1/0 (AG breaks a baseline success), Preserved correct 1/1 (Both paths are correct), and Preserved wrong 0/0 (Neither path is correct) > PFHE ensures that paired outcomes reveal behavior that marginal averages can hide, coupling the intervention to native partial-credit magnitudes >

Empirical Results and Evidence Boundaries

The results show that AG is not uniformly strong across all text tasks; its most credible gain concentrates in the audited Qwen3.5-9B tool route, where the heterogeneous-scorer category micro-average improves from 38.66% to 42.92% (+4.27 pp) and two primary Hermes function-call endpoints improve by about +7.6 pp > The corresponding same-output PFHE-format view records 48 fixes, 26 harms, and a much larger unchanged mass on the same 2,556 category units >

The analysis of Qwen3 and Qwen2.5 shows that their train-selected fixed tool routes are non-positive in native score, establishing a measured cross-checkpoint boundary rather than a universal gate transfer > The exploratory random-forest (RF) candidate selection diagnostic on Qwen3.5 shows that the twelve-candidate matrix contains potentially learnable nonlinear ranking signal, but it lacks baseline fallback and external validation, thus it is not a deployable gate >

Limitations and Future Directions

The paper makes a deliberately bounded claim, noting that the strongest current selector evidence is exploratory offline results on Qwen3.5 tool-structured inference rather than a prospective online A/B validation > The RF diagnostic does not establish statistical superiority because it lacks held-out record provenance and paired uncertainty, and the decision-time legality of features remains unresolved > A credible deployment path requires a useful AG candidate, a selector evaluated with legal features on held-out units, and an acceptance policy that can retain the baseline > Future work should focus on consolidating protocol repair using existing measurements and validating the hypotheses regarding P1s/P2a/P2b mechanisms >

The current engineering evidence is neither a deployment estimate nor a serving benchmark, as online cost scales with the number of candidates, and single-candidate hidden-dimensional routes are the plausible first deployment target > The paper concludes that AG is not a universal quality booster but an evidence-bounded framework for FFN intervention in which classical effectiveness, PFHE safety, and selector validity are reported separately >

--- Page 1 ---

The gist The Amplitude Gating (AG) method improves tool-structured LLM inference by preserving pretrained FFN weight directions and modulating activation magnitudes during decoding, showing positive exploratory results on Qwen3.5-9B tool-route micro-average and Hermes function-call endpoints, while emphasizing that deployment requires a combination of candidate availability, selector validity, and acceptance policy >

Key Components and Protocols

The AG control path is defined by four distinct operations: candidate generation, ranking, acceptance or fallback decision, and baseline fallback > The method uses an entropy-based AG family where the parameter e denotes retained mass for hard/soft/NMF-style gates > The taxonomy includes several variants such as Hard AG (L2-energy gate), Soft AG (L2-derived tau shrinkage), Top-k AG (fixed top-k magnitude), Local-norm AG (local standard deviation threshold), and NMF-style AG, which treats positive and negative activation components separately >

Limitations and Future Directions

The paper makes a deliberately bounded claim, noting that the strongest current selector evidence is exploratory offline results on Qwen3.

Improvements for AI systems

  1. Bold AG preserves pretrained FFN weight directions and changes only activation magnitudes during inference, which can be implemented as a deliberately conservative design to avoid damaging samples while preserving useful insight from ORP. This allows the system to perform inference-time feed-forward network (FFN) intervention without retraining model weights.

  2. The system can incorporate a candidate generation, candidate selection, acceptance or abstention, and baseline fallback control path, separating the amplitude intervention from the policy that decides whether a candidate should be used and providing a common interface for linear, ridge, mixed-feature, and nonlinear selectors.

  3. Implement Per-Sample Fix-Harm Evaluation (PFHE) to complement native task metrics with four mutually exclusive correctness transitions (Fix 0 1 A AG corrects a baseline error, etc.), explicitly reporting preserved correct cases and preserved wrong cases to understand the safety of the intervention.

  4. The system can use a modular control pipeline where different intervention points (P1, P2a, P2b, etc.) are prioritized based on research goals; for deployment, prioritizing P3 first because it is described as Highest Clean output-side hidden-size signal before residual addition.

  5. The model can utilize an entropy-based AG variant (NS AG) where the ranking score is derived from a Newton-Schulz score from a recent token window, allowing the system to adapt its selection based on local token trajectory rather than just static magnitude.

  6. The system can deploy an exploratory random-forest (RF) candidate selector to rank AG candidates, allowing for testing of nonlinear interactions, although this must be treated as exploratory offline because it lacks a baseline fallback and external validation.

  7. For tool-structured inference tasks, the system can use prefix/format features and early logprob trajectory features in its online selector to make decision-time legal decisions, enabling it to perform candidate ranking based on signals that are useful but not yet sufficient for long QA.

Sources

Related papers