How Token Influence Decays with Distance: A Green-Function View of Trained Language Models
summary
The gist
We study how the next-token prediction of an autoregressive Transformer language model changes under small perturbations of earlier input token embeddings This yields a local linearization of the…
In short
The study investigates how a language model's next-token prediction changes when small changes are made to earlier input embeddings. It found that trained models exhibit slowly decaying, long-range sensitivities in these predictions, which is a learned property of the model. This suggests that trained operators possess long-range response modes that can inform better hierarchical model designs.
Key concepts
- Distance-resolved sensitivity
- This measures how much the prediction for a specific next token changes when an earlier input token's embedding is slightly perturbed. It is calculated as the derivative of a selected logit with respect to that earlier embedding, providing a quantitative view of local influence across different causal distances.
- Learned long-range response modes
- These are slow-decaying interactions within the trained Transformer operator that persist even at large causal distances. Unlike simple linguistic dependencies, these modes are not explicitly semantic relations but rather structural properties learned by the model during training, appearing in the distance-resolved sensitivity profile.
- Power-law decay profile
- The distribution of sensitivities across different distances is often well-described by a power law rather than an exponential decay. This suggests that trained models possess a specific, structured way of responding to distant inputs, which is distinct from simpler models and hints at underlying mathematical structures in the learned operator.
Terminology used across episodes
This episode discusses
- How Token Influence Decays with Distance: A Green-Function View of Trained Language Models · Paper Radio
- Overlapping Schwarz Attention: Hierarchical Attention via Domain Decomposition
- In-context Learning and Induction Heads
- Scaling Laws for Neural Language Models
- Jacobian Scopes: A Unified Geometric Framework for Token-Level LLM Attribution · Paper Radio
- A Structural Theory of Position Bias in Transformers · Paper Radio
- Lost in the Middle: How Language Models Use Long Contexts
- On the Emergence of Position Bias in Transformers
- Information Flow Routes: Automatically Interpreting Language Models at Scale
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
- Qwen2.5 Technical Report
- Pointer Sentinel Mixture Models
The paper
How Token Influence Decays with Distance: A Green-Function View of Trained Language Models · Read on arXiv
Matthias Brändel, Stephan Köhler, Oliver Rheinbach
Technische Universi Bergakademie Freiberg
We study how the next-token prediction of an autoregressive Transformer language model changes under small perturbations of earlier input token embeddings. Motivated by operator learning and iterative solvers for differential equations, we investigate how the influence of one token on another decays with distance in a trained model. In multilevel methods for differential equations, such as domain decomposition, multigrid, and multilevel preconditioning, one often exploits a separation between strong local interactions and weaker but essential global interactions. The latter correspond to the long tail of the Green's function and are typically handled by a coarse-level operator. Inspired by this perspective, we compute an empirical, distance-resolved gradient profile of token dependencies using autograd. Experiments on trained Pythia models and Qwen2.5-0.5B show that, over the measured distance range, the median Jacobian sensitivity is much better described by a power-law-type decay than by an exponential alternative: the diagonal-normalized profile is well described by G(r) about γ+β(r+1)-p with exponents p about 0.7 -- 0.9 (typically 0.8 -- 0.9). This behavior appears on coherent text from Gutenberg and WikiText-103. Token-shuffling experiments show that the power-law profile persists even when syntax and prediction quality collapse, whereas randomly initialized models do not exhibit it. The slowly decaying long-range sensitivity thus appears to be a learned property of trained autoregressive Transformer operators. These findings suggest that hierarchical or coarse-level mechanisms in language models may be able to exploit the long-tailed sensitivity profiles.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "How Token Influence Decays with Distance".
Jane: Here's the gist of the paper. We study how the next-token prediction of an autoregressive Transformer language model changes under small perturbations of earlier input token embeddings This yields a local linearization of the learned nonlinear operator Pruning redundant reasoning tokens The main empirical observation is that trained Transformer language models exhibit slowly decaying, longrange Jacobian sensitivities This behavior appears on coherent text from Gutenberg and WikiText-103 The slowly decaying longrange sensitivity thus appears to be a learned property of trained autoregressive Transformer operators.
Tom: How the training signal is built To obtain a scalar derivative, we select one component of this logit vector The distance-resolved sensitivity is defined for a target position j and causal token distance r by Gα(j, r) = ∇e j−r ϕ j (E) α ∈ ∇ej−r ϕj (E) α ∈ ∇ej−rϕ j (E), where j − r is the source position.
Jane: The absolute scale of Gα(j, r) depends on embedding norms, logit scales, model size, and the selected target position.
Tom: Where the gains come from The median diagonal-normalized profile is well described by a power-law-type decay rather than by the exponential alternative tested here This suggests that trained LLMs contain learned long-range response modes.
Title and authors: Tom: So we're looking at this paper called "How Token Influence Decays with Distance: A Green-Function View of Trained Language Models". It looks like they are using math from differential equations to figure out how much one word influences another word based on how far apart they are in the text.
Jane: That sounds complicated, but the core idea seems to be about finding a way to measure that influence in a trained model, not just looking at what attention weights say. It’s trying to get a clearer picture of the long-range structure inside these models.
Lu: Exactly. They are treating the language model like an operator and looking for those strong local interactions versus weaker global ones that need a coarse approximation, which is a concept from numerical analysis. This paper computes an empirical, distance-resolved gradient profile of token dependencies using autograd > <ref:2606.29139#pg1>.
Meng: Okay, so they're measuring this sensitivity between input embeddings and the output logit vector at position j with respect to an earlier input embedding at position j minus r > <ref:2606.29139#pg3>. That means for a target word, they measure how much its prediction score changes when you slightly nudge the embedding of an earlier word.
Jane: And they do this by selecting one component of that logit vector, either the score for what actually happened or the model's prediction > <ref:2606.29139#pg3>. The key finding is that this distance-resolved sensitivity profile, called G alpha at position j and distance r, looks like it follows a power-law type decay over a measured distance range > <ref:2606.29139#pg2>.
Tom: Power law decay is interesting because we usually expect something exponential when things die off in a system like this, so seeing that power law suggests the model has these specific learned long-range response modes > <ref:2606.29139#pg2>.
Lu: That’s what I find compelling. It implies that trained language models don't just have random connections; they actually learn these specific modes of interaction, which are what a coarse level would need to capture if we were designing a hierarchical system > <ref:2606.29139#pg2>.
Meng: But the paper mentions that this profile persists even when you shuffle the tokens, though prediction quality drops significantly, and it completely disappears for randomly initialized models > <ref:2606.29139#pg2>. That’s a big hint about what's learned versus what's just random noise.
Jane: So it suggests that these long-range interactions aren't just surface-level linguistic coherence or syntax in the text itself, but rather something structural baked into the trained operator > <ref:2606.29139#pg3>. It’s a property of the trained model architecture and training process.
Tom: The paper also points out that this structure exists even without direct semantic relations between words, which is important because we don't always have clear semantic mappings for these interactions > <ref:2606.29139#pg2>.
Lu: Right, and this observation opens up a direction for designing hierarchical models in NLP. If we can find a coarse space based on this learned Jacobian structure instead of just sentences or topics, that could lead to much better architectures > <ref:2606.29139#pg2>.
Jane: So what are the practical implications for how we actually use these models? It seems like this mathematical view is giving us tools to design things that capture those long-range modes more effectively than what we do now > <ref:2606.29139#pg2>.
Tom: The authors suggest that existing methods for efficiency, like summary tokens or memory tokens, are motivated by a desire for efficiency, but this research suggests we should be motivating our coarse level design from this measured Jacobian structure instead > <ref:2606.29139#pg4>.
Meng: From an engineering standpoint, if we can use these distance-resolved profiles to guide how we build the coarse attention mechanisms you mentioned, like replacing fully global attention with local and coarse attention > IMPROVEMENT one <ref:2606.29139#pg2>. that would mean fewer parameters while still keeping those long-range weak interactions alive > IMPROVEMENT one <ref:2606.29139#pg2>.
Jane: That sounds like a good way to balance approximation quality with parameter count, especially if we can use the power law decay to decide where the coarse level should cut off its influence > IMPROVEMENT two <ref:2606.29139#pg2>.
Lu: And they are suggesting that for building those coarse variables, instead of relying purely on human semantic units like sentences or topics, we should design them based on this learned long-range response structure itself > IMPROVEMENT two <ref:2606.29139#pg2>.
Tom: So we’re moving from designing coarse spaces based purely on linguistic meaning to designing them based on the mathematical structure the model has learned about its own operator behavior > IMPROVEMENT two <ref:2606.29139#pg2>.
Jane: It also suggests developing better diagnostic tools, not just looking at attention weights, but measuring this actual distance-resolved Jacobian sensitivity to get a quantitative view of token interactions > IMPROVEMENT three <ref:2606.29139#pg3>.
Meng: If we can do that, it helps us understand exactly where the model is paying its attention and where it's relying on those weaker long-range connections > IMPROVEMENT three <ref:2606.29139#pg3>.
Lu: And finally, they suggest training objectives should be tweaked to actually encourage the model to learn those specific long-range response modes instead of just optimizing for prediction quality on coherent text > IMPROVEMENT four <ref:2606.29139#pg2>.
Tom: It sounds like a lot of work, but the idea is that by using this Green-function view of trained language models, we can build better hierarchical architectures for NLP > Conclusion.
The paper's summary: Tom: So, this paper is essentially looking at how much one word influences another word based on how far apart they are in the text > <ref:2606.29139#pg1>. They’re treating the language model like an operator and measuring its sensitivity to input embeddings > <ref:2606.29139#pg2>.
Jane: Right, so it’s trying to figure out if these models have some kind of long-range memory or structure baked into them that we can see mathematically > <ref:2606.29139#pg3>. The main thing they found was that this influence doesn't just drop off randomly; it decays in a very specific pattern over distance > <ref:2606.29139#pg2>.
Tom: And the pattern they saw, which is the median diagonal-normalized profile, follows a power law decay instead of something exponential > <ref:2606.29139#pg2>. That’s interesting because exponentially decaying things are pretty common in physics and many systems, but a power law suggests some specific learned behavior here > <ref:2606.29139#pg4>.
Lu: It implies that trained language models have these learned long-range response modes that are what a coarse level would need to represent > <ref:2606.29139#pg2>. They're not just randomly connected; they’ve built in these structural connections over distance > <ref:2606.29139#pg4>.
Jane: But the authors stress that this structure isn't just about the text being coherent or having long-range semantic links between words > <ref:2606.29139#pg3>. It's a property of the trained operator itself, regardless of whether the text is good or bad > <ref:2606.29139#pg2>.
Tom: They showed this profile survives when they shuffle the tokens around, even though prediction quality tanks, and it completely vanishes for models that are randomly initialized > <ref:2606.29139#pg2>. That really proves it’s something learned during training > <ref:2606.29139#pg4>.
Lu: That’s why I think this matters for building new architectures, because if we can use this measured Jacobian structure to design our coarse attention mechanisms instead of just relying on semantic units like sentences or topics, we could get much better results > <ref:2606.29139#pg2>.
Jane: So they aren't suggesting we look at the words themselves to design a coarse space; they’re suggesting the model’s learned response structure is the blueprint for that space > <ref:2606.29139#pg2>.
Tom: And they also pointed out that existing efficiency methods, like summary tokens or memory tokens, are usually motivated by just making things run faster or cheaper > <ref:2606.29139#pg4>. This paper says we should be designing those coarse levels based on this distance-resolved Jacobian structure instead of just chasing efficiency gains alone > <ref:2606.29139#pg2>.
Lu: Exactly. It’s a shift in how we think about coarse representation—from linguistics to operator theory > <ref:2606.29139#pg2>.
Jane: So, they're giving us a way to design better ways for the AI to "see" and process long-range context, even if it doesn't explicitly understand the meaning of every single word > <ref:2606.29139#pg3>. This is a big step toward making AI systems more structured > <ref:2606.29139#pg2>.
Tom: It’s definitely a strong signal that we need to incorporate this kind of structural analysis into how we design these language models for production, not just focus on getting the highest prediction score on some benchmark dataset > <ref:2606.29139#pg4>.
Lu: The next step is using this mathematical insight to actually build those two-level attention mechanisms I mentioned earlier, combining local attention with a coarse one based on this decay profile > IMPROVEMENT one.
The paper's improvements: Tom: So, the authors aren't just stopping at describing the power law decay; they’re suggesting concrete ways to actually improve these language models > <ref:2606.29139#pg2>. They’ve got a few ideas on how we can make them better by using this structural knowledge > <ref:2606.29139#pg4>.
Jane: It sounds like they want us to use this math not just for understanding, but for fixing the actual model architecture itself > <ref:2606.29139#pg2>. They're talking about making two-level attention mechanisms, mixing local and coarse attention > IMPROVEMENT one.
Lu: Yeah, that makes sense because standard global attention is too expensive; if you can combine it with a coarse mechanism informed by this distance decay, you get better approximation quality while using fewer parameters > IMPROVEMENT one. It’s like using a good shortcut instead of checking every single road on the map > <ref:2606.29139#pg4>.
Meng: From my side, if we can cut down the number of parameters while keeping those weak long-range interactions alive, that’s a win for deployment cost and speed > IMPROVEMENT one. But how does this help when we try to define what that coarse level even is?
Lu: Well, the second big suggestion is about defining what those coarse variables should be > IMPROVEMENT two. Instead of picking something purely linguistic like a sentence or a topic, they suggest designing them based on this learned long-range response structure itself > <ref:2606.29139#pg2>.
Jane: So we move away from just looking at human meaning and start using the AI’s own internal structure as the guide for how we organize its information > <ref:2606.29139#pg2>. That’s pretty ambitious for representation learning, right?
Tom: It's ambitious, but if it works, it means our coarse spaces won't be based on whatever dictionary we use; they’ll be based on what the model has actually learned about its own operator behavior > <ref:2606.29139#pg2>.
Jane: And they also proposed a new way to look at things for debugging and explanation > IMPROVEMENT three. Instead of just looking at attention weights, you measure this distance-resolved Jacobian sensitivity directly > <ref:2606.29139#pg3>.
Lu: That would be much more quantitative, Jane; it gives us a real measurement of how sensitive the model is to small changes in input embeddings across different distances > <ref:2606.29139#pg3>. It’s a Green-function view that actually tells us something about the network’s internal dynamics > <ref:2606.29139#pg2>.
Meng: I see the value in diagnostics if we can get clear numbers instead of just fuzzy attention maps, but what about training itself? How do we stop the model from forgetting those long-range modes as it trains?
Lu: They suggested incorporating new things into the training process to specifically encourage the model to learn those response modes > IMPROVEMENT four. There could be auxiliary losses designed to stabilize or enhance those slowly decaying sensitivity profiles during training > <ref:2606.29139#pg4>.
Jane: So, we’re not just finding a structure in a finished model; they are suggesting how we can guide the AI to build that structure more effectively from the start > <ref:2606.29139#pg2>.
Tom: It sounds like a whole new way of thinking about training objectives, moving past just optimizing for prediction accuracy on text > <ref:2606.29139#pg4>.
Lu: This paper is really pushing for a connection between numerical analysis and NLP architecture design, using this Jacobian structure as the foundation for building better hierarchical systems > <ref:2606.29139#pg2>.
Conclusion: Tom: So we're wrapping up on "How Token Influence Decays with Distance: A Green-Function View of Trained Language Models" and I think the main thing to take away is that trained language models have these specific, learned long-range interactions that follow a power law decay > <ref:2606.29139#pg2>.
Jane: That means we don't need to assume every word is equally important just because it’s in the text; there’s a measurable structure over distance inside the AI itself > <ref:2606.29139#pg3>. This is way more concrete than just looking at attention maps > IMPROVEMENT three.
Lu: It really suggests that if we want to design better hierarchical systems for NLP, we should stop basing those levels on human concepts like paragraphs and start basing them on this learned mathematical structure of the model operator > <ref:2606.29139#pg2>.
Meng: For deployment, it means we might be able to prune or compress the model more intelligently, targeting these long-range modes instead of just doing a generic summary > <ref:2606.29139#pg4>.
Lalam: From my side, I’m pretty sure this is solid; the data from Pythia and Qwen2 point five on Gutenberg and WikiText-one hundred three consistently shows that this power law profile is stable across different input types > <ref:2606.29139#pg6> <ref:2606.29139#pg2>. I'm confident in this result > <ref:2606.29139#pg6>.
Jane: It’s a big step forward because it gives us a new mathematical language to describe what’s happening inside these huge neural networks > <ref:2606.29139#pg3>. It moves the discussion beyond just semantics > <ref:2606.29139#pg2>.
Tom: Yeah, so we're taking this Green-function view of the trained operator and using it to design better hierarchical architectures for AI > <ref:2606.29139#pg4>.
Lu: Exactly; it’s about using the model’s own learned behavior as a blueprint for representation learning > <ref:2606.29139#pg2>.
Meng: I'm still thinking about how much overhead this structural analysis adds during training, but if we can get a good trade-off between accuracy and parameter count, that would be valuable for real applications > IMPROVEMENT one.
Jane: Before we go, Tom and I think this paper sets a really strong foundation for designing systems that aren't just memorizing text but actually learning structural relationships > <ref:2606.29139#pg3>.
Tom: Yeah, so the next step is definitely moving from theory to implementation, figuring out how to use this knowledge practically in our own model designs > <ref:2606.29139#pg4>.
More episodes
- 2603.28595-Optimistic Actor-Critic with Parametric Policies for Linear Markov Decision Processes
- 2603.29765-Training-Free Dynamic Upcycling of Expert Language Models
- 2605.00778-Observable Performance Does Not Fully Reflect Adaptive System Organization: A Multi-Level Analysis of Gait Dynamics Under Occlusal Constraint
- 2605.06906-TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond
- 2605.09165-Sparse Layers are Critical to Scaling Looped Language Models
- 2605.09439-Inverse Design for Conditional Distribution Matching
- 2605.10313-Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards: Regret Bound and Statistical Inference
- 2605.10878-Minimum Neural Weight Norm = Kolmogorov Complexity
- 2605.11617-MIST: Reliable Streaming Decision Trees for Online Class-Incremental Learning via McDiarmid Bound
- 2605.15085-From Data to Action: Accelerating Refinery Optimization with AI