Architecture-Dependent Causal Transfer of Activation States Across Large Language Models

summary

The gist

Direct communication between artificial intelligence systems currently relies on natural language as an intermediate layer, incurring encoding/decoding overhead, token cost…

In short

Researchers tested if internal AI activation states can be causally transferred between different large language models using a learned projection network. While representational alignment showed promise, end-to-end causal transfer during generation succeeded for only one model pair. This suggests the transfer is architecturally dependent, not universal, proving a proof-of-concept vulnerability.

Key concepts

Representational Alignment
This measures how similar the internal 'understanding' or state of two different LLMs is when trained on specific concepts. The study used metrics like CKA and mutual k-NN alignment to quantify this similarity across various concept pairs, showing that trained models are significantly more aligned than random ones.
Causal Transfer
This refers to the ability to directly cause a change in one model's output by injecting its internal activation state into another model during generation. The paper found this transfer is highly specific and only worked for one pair of decoder-only models, indicating it is not a general property.
Representational Vehicle
This term describes the specific pattern of internal activations that acts as the 'vehicle' for transferring information between LLMs. The study argues that this vehicle transfers a representation, not necessarily the underlying meaning or intent of the original model's output.

Terminology used across episodes

This episode discusses

The paper

Architecture-Dependent Causal Transfer of Activation States Across Large Language Models · Read on arXiv

Fernando Cárdenas Piepereit, Velez-Malaga

Spain

Direct communication between AI systems relies on natural language as an intermediate layer, incurring encoding/decoding overhead, token cost, and latency. We ask whether internal activation states can instead be transferred causally between different large language model (LLM) architectures via a learned projection, evaluated at three levels: representational similarity, cross-model retrieval from projected states, and end-to-end causal transfer via activation injection during generation. Using four architecturally diverse open-weight models (Qwen2-0.5B, Phi-3-mini, Mistral-7B, FLAN-T5-base), we find that representational alignment in trained models exceeds a random-initialization null baseline and is best captured by a rank-based metric (mutual k-nearest-neighbour alignment), more robust to activation-magnitude outliers than centered kernel alignment (CKA) or Procrustes analysis. A learned projection network retrieves the correct target-model representation from a held-out set well above chance for the three causal decoder-only model pairs (45-50% top-1 accuracy vs. 5% chance) but at chance level for the encoder-based FLAN-T5. Injecting projected activations into a target model during generation produces a statistically significant, pre-registered causal effect on retrieval-based output similarity for only one of the three decoder-only pairs (Qwen2-0.5B to Phi-3-mini: 23.3% vs. 0.0% under negative control, p=0.047, FDR-corrected); the two pairs targeting Mistral-7B show no such effect despite comparable representational alignment at the hidden-state level. We interpret these results as evidence for causal transfer of the representational vehicle, not of meaning, and conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Architecture-Dependent Causal Transfer of Activation States Across Large Language Models".

Jane: Here's the gist of the paper. Direct communication between artificial intelligence systems currently relies on natural language as an intermediate layer, incurring encoding/decoding overhead, token cost, and latency We ask whether internal activation states can instead be transferred causally between different large language model (LLM) architectures via a learned projection Using four architecturally diverse open-weight models (Qwen2-0.5B, Phi-3-mini, Mistral7B, FLAN-T5-base), we find that representational alignment in trained models exceeds a random-initialization null baseline A learned projection network retrieves the correct target-model representation from a held-out set well above chance for the three causal decoder-only model pairs (45–50% top-1 accuracy vs.

Tom: 5% chance) but at chance level for the encoder-based FLAN-T5 Injecting projected activations into a target model during generation, however, produces a statistically significant, pre-registered causal effect on retrieval-based output similarity for only one of the three decoder-only model pairs (Qwen2-0.5B→Phi-3-mini: 23.3% vs.

Jane: 0.0% under negative control, p = 0.047, false-discovery-rate corrected) We interpret these results as evidence for causal transfer of the representational vehicle, not of meaning Pruning redundant reasoning tokens The paper investigates whether internal activation states can be transferred causally between different LLM architectures via a learned projection The study evaluates this question at three levels: representational similarity, cross-model retrieval from projected states, and end-to-end causal transfer via activation injection during generation This research addresses the gap between observing representational similarities in LLMs and demonstrating that these similarities can be exploited to causally change a target model's behavior The central causal claim is explicitly scoped to the transfer of a representational vehicle, an activation pattern with a demonstrable causal effect on downstream output How the training signal is built The researchers utilized four architecturally diverse open-weight LLMs spanning three independent developers and two architectural families: Qwen2-0.5B, Phi-3-mini, Mistral7B, and FLAN-T5-base This selection was made to vary both training organisation (Alibaba, Microsoft, Mistral AI, Google) and attention architecture (causal vs.

Tom: bidirectional) Representational similarity was assessed across 12 RELATED concept pairs and 12 SYNONYM concept pairs at different layers for these models The SYNONYM set served as the harder triviality control because its members were lexically dissimilar, preventing explanation by shared surface tokens The study employed three alignment metrics: CKA, orthogonal Procrustes analysis, and mutual k-nearest-neighbour (k-NN) alignment Where the gains come from The results show that representational alignment in trained models exceeds a random-initialization null baseline The rank-based metric, mutual k-NN alignment, was used as the primary alignment metric in the projection-network experiment because it was scale invariant and showed expected patterns (trained > untrained by a factor of 3–6×) Cross-model projection network accuracy reached 45–50% top1 accuracy against a 5% chance level for the three causal decoder-only model pairs However, end-to-end causal state transfer succeeded for only one of the three model pairs: Qwen2-0.5B→Phi-3-mini (23.3% vs.

Paper summary: Tom: We've seen how they tested whether you can move internal AI activation states directly between different large language models using a learned projection network, and this paper is titled "Architecture-Dependent Causal Transfer of Activation States Across Large Language Models."

Jane: That title sounds really technical, but what does it actually mean for us as regular people using these tools?

Tom: It means the success of transferring these internal states isn't universal; it only works for specific pairings, which is a big limitation. The authors found that when they tried to inject these activations into a target model during generation, they only saw success on one specific pair out of three tested decoder-only models.

Jane: So if we think about this for our daily use, does that mean we can’t just plug any two models together and expect them to communicate internally?

Tom: Exactly. The research shows that the way these models are built, their architecture, is a major factor in whether this transfer works at all. It’s not some universal property of the AI itself.

Jane: What about the part where they talk about transferring a "representational vehicle" instead of just "meaning"? Is that a big deal?

Tom: Yeah, it’s important. They are saying we can transfer the pattern of activation, like a specific way information is structured internally, but they aren't proving that the target model will suddenly understand what that pattern means in the same way.

Jane: So we’re talking about transferring a numerical fingerprint between two different AI systems, not transferring actual ideas?

Tom: Right. It’s a proof-of-concept for a technical channel, but it’s not yet ready for any kind of deployment where you rely on that transfer being reliable.

Jane: That leads us to the next thing—what happens when we look at the safety side of this?

Jane: It seems like the main point of this paper is that the physical mechanism for representation transfer is real and measurable, but its practical application depends heavily on which specific models you are using >

Conclusion: Tom: We're wrapping up our look at this paper today, "Architecture-Dependent Causal Transfer of Activation States Across Large Language Models." Basically, they’re showing that you can move internal AI activation states directly between different models without relying on natural language translation.

Jane: That sounds like a big technical thing. So, what's the main takeaway for us as listeners? What does it actually mean when they say "architecture dependent"?

Tom: It means this isn't some universal trick you can use with any two AI models you pick. The success of this transfer depends entirely on the specific way each model is built, like whether it’s a causal decoder or an encoder.

Jane: So, if I have two different AI systems, I need to make sure they are architecturally compatible before I even think about trying to use this idea?

Tom: Exactly. The numbers show that representation alignment in trained models beats random guessing significantly, but the actual end-to-end transfer only worked on one specific pairing out of three they tested.

Jane: It’s a proof of concept for a technical channel, not a magic translation of intelligence, right?

Tom: That’s right. It proves that direct activation-level state transfer is technically feasible using current components, but it isn't ready for deployment yet because the results are so specific and limited.

Jane: So we're looking at a potential backdoor for how AI systems communicate internally, even if it’s just a tiny fraction of success right now?

Tom: That's the risk they flagged. It points to a vulnerability in how we currently think about communication between different AI agents operating at the activation level instead of just the token level.

Jane: Next time we talk about this, we're going to look at what researchers are doing right now to test if this is a real risk and how they’re trying to build defenses against it.

More episodes

← Home