How Shared Representations Become Specialized: Formation, Maintenance, and Latent Susceptibility

arXiv:2610.15830 · cs.LG, cs.NE · Submitted 2026-10-11 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "How Shared Representations Become Specialized".

Jane: The gist The paper studies how shared representations become specialized by investigating formation, maintenance, and latent susceptibility in minimal artificial learners.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So let's start with "How Shared Representations Become Specialized: Formation, Maintenance, and Latent Susceptibility." This paper really looks at how AI representations become specialized when they are learning things.

Jane: That sounds like it could be huge for anyone training models because it gets into the mechanics of that bias. What’s the big idea here?

Lu: The core idea is functional bias, which is essentially which targets are easier to recover from a specific internal code, and they measure this using an allocation index s.

Tom: So if we have a positive s, that means target A is more precisely readable from that internal code >

Jane: And negative s means target B is the one that’s harder to read with the same representation >

Meng: That sounds like a specific way to quantify which direction the learned features are leaning, and it’s not just some vague correlation.

Tom: Exactly. The paper investigates three things: what selects and redirects the favored target, what presently supports that specialization, and what determines how learners respond when they look similar under different measurements >

Jane: That gives us a solid framework to unpack the whole paper because it’s not just one simple learning process.

Lu: We see formation happening when acquisition precision changes the noise on the target-specific input before it mixes in >

Tom: High- and low-precision noise standard deviations are zero point zero five and one point two zero, which is a big difference in how that input gets handled > <ref:2610.15830#pg1>

Meng: So if you feed it super precise data for target A, that's the signal we are talking about, but how does that translate into this index s >

Jane: The paper shows that when acquisition precision changes, precise acquisition of target A yields a mean s of zero point three three eight five >

Tom: And if you precisely acquire target B instead, you get a mean s of negative zero point three three three eight >

Lu: So those specific numbers are what show the initial directional pull during formation >

Meng: That’s concrete evidence for how the system decides which path to take when it’s learning from mixed signals.

Tom: Now we have to ask, does that specialization just stay there forever after training is done?

Jane: Because if it doesn't stick, then the whole process of learning might be more about management than just initial training >

Lu: Maintenance comes in here; it involves interventions that change the learning conditions to keep that specialization going >

Tom: The paper points out that state-following feedback can maintain this bias, which is a way of actively supporting the formed direction >

Meng: So it’s not passive storage; you have to keep feeding the model in a way that reinforces what it already learned >

Jane: And then they also looked at latent susceptibility to see how sensitive these learners are to changes later on >

Lu: They tested this by matching selected endpoint measurements before applying the same next intervention and exchanging candidate persistent-state components >

Tom: Exchanging either component between two route endpoints leaves the mean route difference almost unchanged, but exchanging v reverses that difference entirely >

Jane: That testing of a single component directly, rather than just inferring importance from history, is a really interesting way to look at model steering.

Meng: From an engineering standpoint, that tells us we can target specific internal parts if we know which one to swap or adjust >

Tom: So the paper concludes that formation history affects later learning through more than one time-dependent route >

Jane: It seems like the key finding is that present conditions—what you are doing right now with acquisition and updating—are what truly sustain a specialization, not just looking back at how it happened >

Lu: That means we should focus on current interventions to guide these representations effectively in the future >

Conclusion: Tom: So, we're wrapping up this look at "How Shared Representations Become Specialized: Formation, Maintenance, and Latent Susceptibility." Basically, this paper explores how shared AI representations get specialized when they are learning things.

Jane: That seems like a really big question for anyone who works on training models. I mean, what does that specialization actually look like in practice? Is it just one thing the model gets really good at?

Lu: It’s more nuanced than that. The paper focuses on functional bias—which targets are easier to recover from a specific internal code. They measure this with something called an allocation index, s. A positive s means target A is more readable from that code, and negative means target B is.

Meng: Okay, so we’re talking about different targets getting pulled in different directions by the same underlying AI structure. What are the specific mechanisms they found for how that happens during training?

Tom: They looked at formation and maintenance. Formation involves what selects a target as favored—like how precise acquisition of one target changes the noise on the input before it mixes. It showed that precise acquisition of A gives a mean s of zero point three three eight five, while B gives negative zero point three three three eight.

Jane: That number, zero point three three eight five and negative zero point three three three eight, it sounds like a real shift in how the representation is biased. But does that specialization stick around once training is over?

Lu: Not automatically. Maintenance involves interventions that change the learning conditions to keep that specialization going. They found state-following feedback can maintain it, but they also saw that just having a formation history isn't enough to keep the original magnitude under new conditions.

Meng: So, it’s not just about the initial training run; you have to actively manage those learning conditions if you want to keep a specific target specialized. That means deployment needs some kind of steering mechanism, right?

Tom: Exactly. And then they tested latent susceptibility by swapping components between different route endpoints. They found that exchanging either component leaves the mean route difference almost unchanged, but exchanging v reverses the difference entirely. It’s testing a single component directly without relying on correlation with the whole history.

Jane: That’s interesting because it moves away from just looking at the big picture history and actually tests a specific part of that internal structure. What does this mean for how we think about steering models?

Lu: The core finding is that formation history affects later learning through multiple routes, and what truly matters is the current conditions sustaining that specialization, not just the past. We need to focus on what’s happening right now with the input and updates to guide these representations effectively.

Toru Takahashi

Doshisha University · Linked Open Data Initiative · Keio Research Institute at SFC · Stroly Inc.

cs.LG, cs.NE

Submitted: 2026-10-11

Updated: 2026-10-11

Comments: 23 pages including appendices; 5 figures

License: http://creativecommons.org/licenses/by/4.0/

The gist: The gist The paper studies how shared representations become specialized by investigating formation, maintenance, and latent susceptibility in minimal artificial learners.

Key concepts

Functional Bias: This refers to which specific targets within a shared internal representation are more precisely recoverable from one internal code. It measures how much easier it is to read one target compared to others using the same underlying representation.
Allocation Index (s): This index quantifies specialization. A positive value means target A is more precisely readable, while a negative value indicates target B is more precisely readable from that internal code. It measures the degree of precision for different targets.
Formation History: This refers to the sequence of learning conditions and interventions that occurred during the initial stages of representation formation. The study shows this history can leave a residual effect, but its influence on later learning is complex and not always direct.
Latent Susceptibility: This tests whether a specialization persists by matching endpoint measurements before applying the same intervention and swapping components. This method directly checks if a specific component drives the persistence of the learned specialization.

Summary

The gist The paper studies how shared representations become specialized by investigating formation, maintenance, and latent susceptibility in minimal artificial learners.

How it works

The study explores how information about certain targets can remain easier to recover than others when several predictions use the same internal representation The core concept is functional bias, defined as which targets are more precisely recoverable from one internal code. This specialization is measured using an allocation index s, where a positive value indicates that target A is more precisely readable and negative for target B.

Formation and Selection Mechanisms

The paper investigates what selects and redirects the favored target through various learning conditions Acquisition precision changes the noise on the target-specific input before mixing. Under equal loss weights, precise acquisition of A yields mean s = 0.3385, while precise acquisition of B yields s = −0.3338.

Maintenance and Persistence

The paper examines what presently supports a formed specialization through interventions that change the learning conditions State-following feedback can maintain it. The study shows that formation history can leave a residual without maintaining its original magnitude under changed current conditions.

Latent Susceptibility and Optimizer State

The paper tests latent susceptibility by matching selected endpoint measurements before applying the same next intervention and exchanging candidate persistent-state components. Exchanging either component between two route endpoints leaves the mean route difference almost unchanged; exchanging v reverses the difference. This tests a component directly rather than inferring its importance from a correlation with formation history.

General Findings

The results connect each question to an intervention, showing that acquisition and updating are competing controls over specialization. The paper concludes that formation history affects later learning through more than one time-dependent route: inherited optimizer state strongly modifies the early transient, while model-side differences can appear later. The study establishes that present conditions that sustain a specialization are what matter, rather than assigning its persistence to history alone.

Improvements for AI systems

Bold header: Precise acquisition selection under equal loss weights. This improvement allows an AI system to select a target more accurately by varying observation noise precision while holding update weights equal, as Precise acquisition of A yields mean s = 0.3385, while precise acquisition of B yields s = −0.3338.
Bold header: Redirection of formed states via acquisition changes. The system can be engineered so that Opposite-target acquisition changes the favored target in all 40 seeds for both starting states, allowing for dynamic switching between learned representations based on observation quality rather than just update weighting.
Bold header: Sustaining specialization through state-following feedback. By implementing a rule where it preferentially updates the better-represented target, the system can maintain a learned bias, as state-following feedback amplifies incidental initial differences: terminal |s| is 0.8683, versus 0.4207 under fixed propensity.
Bold header: Causal modification of short-horizon response via optimizer state exchange. The system can test which internal component modifies a response difference by Exchanging v reverses the difference, from +0.1968 to −0.1265, allowing for targeted optimization of the model's update mechanism to mitigate or exploit specific learning trajectories.
Bold header: Adaptation of learning rate based on coordinate-wise second-moment scaling. The system can employ Coordinate-wise pattern interventions, where Equalizing a¯v between routes reduces the ten-update response difference from 0.1907 to 0.0875, enabling the network to dynamically adjust its update scaling based on the learned representation geometry rather than relying solely on aggregate second-moment statistics.
Bold header: Distinguishing formation history effects from current support effects. The system can be designed such that making both acquisition precision and update weights symmetric reduces mean |s| from about 0.41 to 0.04 over 400 updates, allowing for the removal or symmetrization of learning conditions to determine if a bias is transient or persistent.
Bold header: Enhanced three-target specialization detection via observable allocation planes. The system can utilize Singular values summarize the target-specific intervention-effect matrix, providing two observable coordinates that describe how targets are read out, such as the gain in qA and qB under specific acquisition conditions.

Abstract

When several predictions use the same internal representation, information about some targets can remain easier to recover than information about others. We study how this specialization forms, persists, and changes in minimal artificial learners. With reconstruction errors weighted equally during learning, changing which target is observed precisely selects the favored target and redirects an already specialized learner. Removing current support can reduce a formed bias, whereas state-following update feedback can maintain it. We then compare learners formed through different observation and update conditions but matched on specified reconstruction measurements. Both change under the same brief opposing intervention, yet their responses differ. Resetting Adam largely removes the ten-update difference; exchanging its second moment reverses which route is more susceptible. New interventions show that equalizing the mean second-moment scaling reduces but does not remove the difference: its coordinate-wise pattern also matters. With fresh Adam or calibrated SGD, smaller route differences emerge later, whereas 500 common opposing updates bring the selected allocation endpoints close. A further control produces response differences with identical acquisition and equal mean loss weights but jointly different weight variability and realized sequence. Acquisition-selected formation also extends to a symmetric three-target setting. These interventions distinguish specialization's formation, present support, and time-dependent latent susceptibility.