A Training-Free Socket for DNA in Frozen Language Models

arXiv:2611.05401 · q-bio.GN, cs.AI · Submitted 2026-10-16 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: I'm Ines.

Marcus: And I'm Marcus, here with Yuki.

Ines: We're looking at the paper: "A Training-Free Socket for DNA in Frozen Language Models".

Marcus: Here's the gist of the paper. Connecting a DNA foundation model to a large language model usually requires training a connector, and often the LLM too Our method, the socket, fits one ridge regression (the plug) from frozen Evo 2 features to the host LLM’s own representation of the gene’s symbol, name and description at an early layer In a single run on the held-out test set, under a frozen analysis plan, real DNA beats shuffled DNA on all four hosts: by +22.2 points

19.8, 25.0: on Qwen3.5-4B, +26.1

24.3, 27.9: on Qwen3.6-27B, +16.7

14.2, 19.7: on Mistral7B and +18.4

16.2, 21.0: on OLMo-2-7B The Socket Mechanism The socket procedure involves replacing the hidden state at placeholder positions with the map’s output, and no gradient passes through the host The plug is an affine map P(x) = µt + (x − µx)W from the encoder feature xg to tg This map is fitted by ridge regression on the 8,319 genes of usable GeneHop training items that share no gene with the development set and have a protein name or description After block l∗, we replace the hidden state at every DNA slot by hˆ(x), and the forward pass continues To feed host Y with host X’s plug, we fit a second ridge map SX→Y from X’s target space at l∗X to Y’s at l∗Y Controls and Guarantees The controls include nearest-neighbour retrieval, homology splits, and a gene swap The socket conveys identity, as the plug trained on gene symbols alone works, and swapping genes removes the effect Retrieving the nearest training gene gives most of the gain, but the plug beats this retrieval on every host by 3.0–5.2 points and keeps most of its effect without training homologs The socket cannot add knowledge the host lacks; it lets the host use the knowledge it has Layer Selection Strategy The working layer is early, and a rule that uses no task items was selected on three hosts; it predicted the working layer of the fourth before any DNA-plug scoring on it The rule measures whether the host can read a known text representation written at the slots It then predicted the layer of a held-out host, OLMo-2-7B, before any DNA-plug scoring on it Modularity and Modalities Through textonly maps, one DNA plug feeds four hosts from three families A protein plug uses the same sockets (exploratory) The socket tells the host which gene it is looking at, and the host answers from what it already knows We map a frozen biological encoder in closed form into a frozen LLM’s own text representation, and write the result at an early layer chosen without task items In a single run on a held-out test set, this lets four LLMs from three developers answer gene-level questions about sequences whose names they are not told, at no training cost: +16.7 to +26.1 points over shuffled DNA, with every host surviving Holm correction and every null control at the sham level The plug moves between hosts through a text-only linear map, within one point of the host’s own plug It transfers gene identity, beyond 1-NN lookup but not as sequence-level knowledge the host lacks The socket mechanism A Training-Free Socket for DNA in Frozen Language Models is a method that maps a frozen biological encoder in closed form into a frozen LLM’s own text representation, and writes the result at an early layer chosen without task items The socket procedure involves replacing the hidden state at placeholder positions with the map’s output, and no gradient passes through the host The plug is an affine map P(x) = µt + (x − µx)W from the encoder feature xg to tg Controls and Guarantees The controls include nearest-neighbour retrieval, homology splits, and a gene swap The socket conveys identity, as the plug trained on gene symbols alone works, and swapping genes removes the effect Retrieving the nearest training gene gives most of the gain, but the plug beats this retrieval on every host by 3.0–5.

Title and authors: Ines: We've covered the title and authors of "A Training-Free Socket for DNA in Frozen Language Models", and it seems the whole point is showing how to bypass the usual need to train a connector or fine-tune the language model itself >

Marcus: Right. The paper focuses on this specific technique, which they call the socket, that uses a closed-form map from frozen biological features directly into the host language model's representation at an early layer >

Yuki: It’s about mapping that DNA structure onto what the LLM already knows about gene symbols and descriptions so you can ask gene-level questions without any extra training cost >

Ines: And they show this works by fitting one ridge regression, or the plug, from frozen Evo two features to that host's representation at an early layer >

Marcus: The key is that the plug’s output replaces the hidden state at placeholder positions and no gradient passes through the host during training >

Yuki: That way, it sounds like you can leverage existing knowledge in the LLM without needing to modify its weights or train a whole new adapter for every biological task >

Ines: And they didn't just test it once; they also developed a layer rule that uses no task items to pick the correct embedding layer based on whether the host can read known text at those DNA positions >

Marcus: That rule let them predict the working layer of a fourth host, OLMo two seven billion parameters, before any DNA plug scoring was done on it >

Yuki: It sounds like they found a way to dynamically choose where to inject this information based on what the language model is already capable of reading >

Ines: So we're moving beyond just plugging things in blindly; they are selecting the right spot inside the language model's structure first >

The paper's summary: Marcus: Now, let's look at what the paper actually summarizes about this method. It basically says that a closed-form map is sufficient for answering gene-level questions directly >

Ines: They emphasize that the socket carries identity, and they prove this by showing that a plug trained only on gene symbols works and swapping genes removes the effect >

Yuki: So it's conveying the specific identity of the gene without needing complex sequence-level knowledge that isn't already in the host model >

Marcus: They also found that even if you don't have homologs for a gene, refitting the plug keeps most of its performance, which is a big deal because it means you don't need to train new adapters for every single gene variation >

Ines: And they explicitly state that the socket cannot add knowledge the host lacks; it just lets the host use what it already possesses >

Yuki: That’s a crucial distinction. It’s not about teaching the LLM new biology; it’s about guiding its existing knowledge to answer specific questions about DNA sequences >

Marcus: And they highlight that they have four hosts and three families of models, showing the modularity when you use text only maps to feed these different systems >

Ines: So in short, the summary is that this method fits a frozen biological encoder into the LLM's own text space at an early layer using this plug mechanism >

The paper's improvements: Yuki: I’m interested in the improvements they suggest because they tackle some of the limitations we see when connecting these two modalities, like needing a connector >

Ines: They improve it by showing that a closed-form map is enough, meaning you don't need to train a complex adapter or fine-tune the host model itself >

Marcus: They also improved the layer selection by using a rule based on slot readability that doesn't require any task items to be used when making that choice >

Yuki: That layer rule is clever because it measures whether the host can read text already written at those DNA positions before you even apply the plug scoring >

Ines: It’s not just about plugging in randomly; it’s about finding a way to choose the right layer where that information will actually be used by the model >

Marcus: And they showed that they can feed one DNA plug into four different hosts through a text only linear map, which shows how modular this whole socket concept is >

Yuki: It seems like the main improvement is making it training-free while still achieving results that are competitive against things like nearest neighbour retrieval in specific spaces >

Conclusion: Ines: So to wrap up on "A Training-Free Socket for DNA in Frozen Language Models", the main implication is that you can map a frozen biological encoder into an LLM's text representation using this socket mechanism without needing any training costs >

Marcus: Right. It lets you answer gene-level questions about sequences whose names you weren't explicitly told, and the results are impressive, showing gains of plus sixteen point seven to twenty-six point one points over shuffled DNA across four different hosts >

Yuki: It suggests that this identity transfer is more powerful than just a simple nearest neighbour lookup because the plug beats it by three point zero to five point two points on every host >

Ines: And they showed that even if you don't have homologs, the socket keeps most of its effect, which means you don't need to re-train for every new gene variant >

Marcus: Overall, this work establishes a new baseline for trained connectors by showing that a learned lookup of gene identity can achieve similar gains without any training cost >

Yuki: It’s a powerful tool because it lets us see how much knowledge we can transfer between these different biological and language systems with just the right mapping >

Arash Fatehi

q-bio.GN, cs.AI

Submitted: 2026-10-16

Updated: 2026-10-16

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 89/100

The gist: Connecting a DNA foundation model to a large language model usually requires training a connector, and often the LLM too Our method, the socket…

Key concepts

The Socket Mechanism
This procedure replaces hidden states at specific DNA positions with the output of a pre-trained map (the plug). Crucially, no gradients flow back through the host LLM. The plug is an affine transformation that maps biological features to the LLM's text space.
The Plug
The plug is an affine map, P(x) = µt + (x − µx)W, fitted via ridge regression on gene symbols and descriptions. It takes the input from the DNA encoder features and transforms them into a representation suitable for the host LLM's layer.
Layer Selection Strategy
The paper uses a rule based on whether the host can read known text representations at specific slots to select the working embedding layer. This selection is made without using any task items, successfully predicting the correct layer in various hosts like OLMo-2-7B before scoring DNA.
Modularity
The system uses text-only maps that allow a plug trained for one host to feed another host via a linear map. This demonstrates modularity, showing that the socket effectively conveys gene identity across different LLM families and modalities like protein plugs.

Terminology

Summary

Connecting a DNA foundation model to a large language model usually requires training a connector, and often the LLM too Our method, the socket, fits one ridge regression (the plug) from frozen Evo 2 features to the host LLM’s own representation of the gene’s symbol, name and description at an early layer In a single run on the held-out test set, under a frozen analysis plan, real DNA beats shuffled DNA on all four hosts: by +22. p. 0

Where the paper says it

A Training-Free Socket for DNA in Frozen Language Models Arash Fatehi Abstract Connecting a DNA foundation model to a large language model (LLM) usually requires training a connector, and often the LLM too. We show that a closed-form map is enough for genelevel questions.

2 points [19. p. 0
Where the paper says it

In a single run on the held-out test set, under a frozen analysis plan, real DNA beats shuffled DNA on all four hosts: by +22. 2 points [19.

8, 25.0] on Qwen3. p. 0
Where the paper says it

2 points [19. 0] on Qwen3.

5-4B, +26.1 [24.3, 27.9] on Qwen3. p. 0
Where the paper says it

2 points [19. 0] on Qwen3.

6-27B, +16. p. 0
Where the paper says it

9] on Qwen3. 6-27B, +16.

7 [14.2, 19.7] on Mistral7B and +18. p. 0
Where the paper says it

6-27B, +16. 7] on Mistral7B and +18.

4 [16.2, 21.0] on OLMo-2-7B

The Socket Mechanism

The socket procedure involves replacing the hidden state at placeholder positions with the map’s output, and no gradient passes through the host The plug is an affine map P(x) = µt + (x − µx)W from the encoder feature xg to tg This map is fitted by ridge regression on the 8,319 genes of usable GeneHop training items that share no gene with the development set and have a protein name or description After block l∗, we replace the hidden state at every DNA slot by hˆ(x), and the forward pass continues To feed host Y with host X’s plug, we fit a second ridge map SX→Y from X’s target space at l∗X to Y’s at l∗Y

Controls and Guarantees

The controls include nearest-neighbour retrieval, homology splits, and a gene swap The socket conveys identity, as the plug trained on gene symbols alone works, and swapping genes removes the effect Retrieving the nearest training gene gives most of the gain, but the plug beats this retrieval on every host by 3.0–5.2 points and keeps most of its effect without training homologs The socket cannot add knowledge the host lacks; it lets the host use the knowledge it has

Layer Selection Strategy

The working layer is early, and a rule that uses no task items was selected on three hosts; it predicted the working layer of the fourth before any DNA-plug scoring on it The rule measures whether the host can read a known text representation written at the slots It then predicted the layer of a held-out host, OLMo-2-7B, before any DNA-plug scoring on it

Modularity and Modalities

Through textonly maps, one DNA plug feeds four hosts from three families A protein plug uses the same sockets (exploratory) The socket tells the host which gene it is looking at, and the host answers from what it already knows

Conclusion

We map a frozen biological encoder in closed form into a frozen LLM’s own text representation, and write the result at an early layer chosen without task items In a single run on a held-out test set, this lets four LLMs from three developers answer gene-level questions about sequences whose names they are not told, at no training cost: +16.7 to +26.1 points over shuffled DNA, with every host surviving Holm correction and every null control at the sham level The plug moves between hosts through a text-only linear map, within one point of the host’s own plug It transfers gene identity, beyond 1-NN lookup but not as sequence-level knowledge the host lacks

A Training-Free Socket for DNA in Frozen Language Models is a method that maps a frozen biological encoder in closed form into a frozen LLM’s own text representation, and writes the result at an early layer chosen without task items The socket procedure involves replacing the hidden state at placeholder positions with the map’s output, and no gradient passes through the host The plug is an affine map P(x) = µt + (x − µx)W from the encoder feature xg to tg

The controls include nearest-neighbour retrieval, homology splits, and a gene swap The socket conveys identity, as the plug trained on gene symbols alone works, and swapping genes removes the effect Retrieving the nearest training gene gives most of the gain, but the plug beats this retrieval on every host by 3.0–5.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Implementation of a closed-form plug for foreign modality encoding allows connecting DNA foundation models to host LLMs without training a connector or fine-tuning the host, reducing training costs significantly. The socket fits one ridge regression (the plug) from frozen Evo 2 features to the host LLM’s own representation of the gene’s symbol, name and description at an early layer.

  2. Development of a layer rule based on slot readability allows for choosing the correct embedding layer without using any task items. This rule measures whether the host can read a known text representation written at the DNA positions and successfully predicted working layers across different hosts, such as predicting the working layer of OLMo-2-7B.

  3. Creation of a modular map that allows a plug fitted for one host to feed another host through a text only linear map between their spaces. This enables one DNA plug feeds four hosts from three families and shows that the socket conveys gene identity.

  4. Utilization of controls like nearest-neighbour retrieval and homology splits as baselines allows researchers to quantify the gain provided by the socket, showing that the plug beats this retrieval on every host by 3.0–5.2 points while keeping most of its effect without training homologs.

  5. Integration of a protein plug into the same sockets demonstrates modularity across modalities, where the protein plug is about as good as nearest-neighbour retrieval in ESMC space, and shows that Protein and DNA carry about the same usable signal.

  6. A new baseline for trained connectors is established by comparing them against the socket, showing that a trained connector's gain must be measured against this method, as a large benchmark gain is compatible with a learned lookup of gene identity.

Abstract

Connecting a DNA foundation model to a large language model (LLM) usually requires training a connector, and often the LLM too. We show that a closed-form map is enough for gene-level questions. Our method, the socket, fits one ridge regression (the plug) from frozen Evo 2 features to the host LLM's own representation of the gene's symbol, name, and description at an early layer. The plug's output replaces the hidden state at placeholder positions, and no gradient passes through the host. We test it on GeneHop, a five-way multiple-choice benchmark split by homology. In a single run on the held-out test set, under a frozen analysis plan, real DNA beats shuffled DNA on all four hosts: by +22.2 points [19.8, 25.0] on Qwen3.5-4B, +26.1 [24.3, 27.9] on Qwen3.6-27B, +16.7 [14.2, 19.7] on Mistral-7B, and +18.4 [16.2, 21.0] on OLMo-2-7B. All four survive Holm correction, every null control stays within two points of the sham level, and the development results agree within 2.3 points. A layer rule that uses no task items was selected on three hosts; it predicted the working layer of the fourth before any DNA-plug scoring on it. A text-only linear map lets one plug feed any other host, at a cost of at most 0.9 points on test. The socket conveys gene identity: nearest-neighbour retrieval recovers 72-84% of the effect, but the plug beats it on every host by 3.0-5.2 points and keeps most of its effect without training homologs.

Sources

Related papers