A Training-Free Socket for DNA in Frozen Language Models
summary
The gist
Connecting a DNA foundation model to a large language model usually requires training a connector, and often the LLM too Our method, the socket…
In short
The method maps a frozen biological DNA encoder into an LLM's text representation using a 'socket' mechanism. This involves fitting a simple affine map (the plug) to connect gene features to the host LLM at an early layer, chosen without task items. In tests, this socket significantly boosts performance on four different LLMs compared to shuffled DNA, proving it conveys gene identity without training costs.
Key concepts
- The Socket Mechanism
- This procedure replaces hidden states at specific DNA positions with the output of a pre-trained map (the plug). Crucially, no gradients flow back through the host LLM. The plug is an affine transformation that maps biological features to the LLM's text space.
- The Plug
- The plug is an affine map, P(x) = µt + (x − µx)W, fitted via ridge regression on gene symbols and descriptions. It takes the input from the DNA encoder features and transforms them into a representation suitable for the host LLM's layer.
- Layer Selection Strategy
- The paper uses a rule based on whether the host can read known text representations at specific slots to select the working embedding layer. This selection is made without using any task items, successfully predicting the correct layer in various hosts like OLMo-2-7B before scoring DNA.
- Modularity
- The system uses text-only maps that allow a plug trained for one host to feed another host via a linear map. This demonstrates modularity, showing that the socket effectively conveys gene identity across different LLM families and modalities like protein plugs.
Terminology used across episodes
This episode discusses
- A Training-Free Socket for DNA in Frozen Language Models · Paper Radio
- Prompt Compression via Activation Aggregation
- Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries
- BioReason: Incentivizing Multimodal Biological Reasoning within a DNA-LLM Model
- Can We Read the Mind of an Audio LLM? A Verbalizable, Multilingual Middle-Layer Workspace · Paper Radio
- Prot2Text-V2: Protein Function Prediction with Multimodal Contrastive Alignment
- Dissecting Recall of Factual Associations in Auto-Regressive Language Models
- Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models
- Ventriloquist LLMs: Linear Alignment of Late-Stage Representations · Paper Radio
- Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts
- The Platonic Representation Hypothesis
- Harnessing the Universal Geometry of Embeddings
- From Tokens to Words: On the Inner Lexicon of LLMs
- The Remarkable Robustness of LLMs: Stages of Inference?
- Shared Global and Local Geometry of Language Model Embeddings
- Towards 3D Molecule-Text Interpretation in Language Models
- LangBridge: Interpreting Image as a Combination of Language Embeddings
- EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations
- MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter
- Latent Space Translation via Semantic Alignment
- AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding
The paper
A Training-Free Socket for DNA in Frozen Language Models · Read on arXiv
Arash Fatehi
Connecting a DNA foundation model to a large language model (LLM) usually requires training a connector, and often the LLM too. We show that a closed-form map is enough for gene-level questions. Our method, the socket, fits one ridge regression (the plug) from frozen Evo 2 features to the host LLM's own representation of the gene's symbol, name, and description at an early layer. The plug's output replaces the hidden state at placeholder positions, and no gradient passes through the host. We test it on GeneHop, a five-way multiple-choice benchmark split by homology. In a single run on the held-out test set, under a frozen analysis plan, real DNA beats shuffled DNA on all four hosts: by +22.2 points [19.8, 25.0] on Qwen3.5-4B, +26.1 [24.3, 27.9] on Qwen3.6-27B, +16.7 [14.2, 19.7] on Mistral-7B, and +18.4 [16.2, 21.0] on OLMo-2-7B. All four survive Holm correction, every null control stays within two points of the sham level, and the development results agree within 2.3 points. A layer rule that uses no task items was selected on three hosts; it predicted the working layer of the fourth before any DNA-plug scoring on it. A text-only linear map lets one plug feed any other host, at a cost of at most 0.9 points on test. The socket conveys gene identity: nearest-neighbour retrieval recovers 72-84% of the effect, but the plug beats it on every host by 3.0-5.2 points and keeps most of its effect without training homologs.
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: I'm Ines.
Marcus: And I'm Marcus, here with Yuki.
Ines: We're looking at the paper: "A Training-Free Socket for DNA in Frozen Language Models".
Marcus: Here's the gist of the paper. Connecting a DNA foundation model to a large language model usually requires training a connector, and often the LLM too Our method, the socket, fits one ridge regression (the plug) from frozen Evo 2 features to the host LLM’s own representation of the gene’s symbol, name and description at an early layer In a single run on the held-out test set, under a frozen analysis plan, real DNA beats shuffled DNA on all four hosts: by +22.2 points
19.8, 25.0: on Qwen3.5-4B, +26.1
24.3, 27.9: on Qwen3.6-27B, +16.7
14.2, 19.7: on Mistral7B and +18.4
16.2, 21.0: on OLMo-2-7B The Socket Mechanism The socket procedure involves replacing the hidden state at placeholder positions with the map’s output, and no gradient passes through the host The plug is an affine map P(x) = µt + (x − µx)W from the encoder feature xg to tg This map is fitted by ridge regression on the 8,319 genes of usable GeneHop training items that share no gene with the development set and have a protein name or description After block l∗, we replace the hidden state at every DNA slot by hˆ(x), and the forward pass continues To feed host Y with host X’s plug, we fit a second ridge map SX→Y from X’s target space at l∗X to Y’s at l∗Y Controls and Guarantees The controls include nearest-neighbour retrieval, homology splits, and a gene swap The socket conveys identity, as the plug trained on gene symbols alone works, and swapping genes removes the effect Retrieving the nearest training gene gives most of the gain, but the plug beats this retrieval on every host by 3.0–5.2 points and keeps most of its effect without training homologs The socket cannot add knowledge the host lacks; it lets the host use the knowledge it has Layer Selection Strategy The working layer is early, and a rule that uses no task items was selected on three hosts; it predicted the working layer of the fourth before any DNA-plug scoring on it The rule measures whether the host can read a known text representation written at the slots It then predicted the layer of a held-out host, OLMo-2-7B, before any DNA-plug scoring on it Modularity and Modalities Through textonly maps, one DNA plug feeds four hosts from three families A protein plug uses the same sockets (exploratory) The socket tells the host which gene it is looking at, and the host answers from what it already knows We map a frozen biological encoder in closed form into a frozen LLM’s own text representation, and write the result at an early layer chosen without task items In a single run on a held-out test set, this lets four LLMs from three developers answer gene-level questions about sequences whose names they are not told, at no training cost: +16.7 to +26.1 points over shuffled DNA, with every host surviving Holm correction and every null control at the sham level The plug moves between hosts through a text-only linear map, within one point of the host’s own plug It transfers gene identity, beyond 1-NN lookup but not as sequence-level knowledge the host lacks The socket mechanism A Training-Free Socket for DNA in Frozen Language Models is a method that maps a frozen biological encoder in closed form into a frozen LLM’s own text representation, and writes the result at an early layer chosen without task items The socket procedure involves replacing the hidden state at placeholder positions with the map’s output, and no gradient passes through the host The plug is an affine map P(x) = µt + (x − µx)W from the encoder feature xg to tg Controls and Guarantees The controls include nearest-neighbour retrieval, homology splits, and a gene swap The socket conveys identity, as the plug trained on gene symbols alone works, and swapping genes removes the effect Retrieving the nearest training gene gives most of the gain, but the plug beats this retrieval on every host by 3.0–5.
Title and authors: Ines: We've covered the title and authors of "A Training-Free Socket for DNA in Frozen Language Models", and it seems the whole point is showing how to bypass the usual need to train a connector or fine-tune the language model itself >
Marcus: Right. The paper focuses on this specific technique, which they call the socket, that uses a closed-form map from frozen biological features directly into the host language model's representation at an early layer >
Yuki: It’s about mapping that DNA structure onto what the LLM already knows about gene symbols and descriptions so you can ask gene-level questions without any extra training cost >
Ines: And they show this works by fitting one ridge regression, or the plug, from frozen Evo two features to that host's representation at an early layer >
Marcus: The key is that the plug’s output replaces the hidden state at placeholder positions and no gradient passes through the host during training >
Yuki: That way, it sounds like you can leverage existing knowledge in the LLM without needing to modify its weights or train a whole new adapter for every biological task >
Ines: And they didn't just test it once; they also developed a layer rule that uses no task items to pick the correct embedding layer based on whether the host can read known text at those DNA positions >
Marcus: That rule let them predict the working layer of a fourth host, OLMo two seven billion parameters, before any DNA plug scoring was done on it >
Yuki: It sounds like they found a way to dynamically choose where to inject this information based on what the language model is already capable of reading >
Ines: So we're moving beyond just plugging things in blindly; they are selecting the right spot inside the language model's structure first >
The paper's summary: Marcus: Now, let's look at what the paper actually summarizes about this method. It basically says that a closed-form map is sufficient for answering gene-level questions directly >
Ines: They emphasize that the socket carries identity, and they prove this by showing that a plug trained only on gene symbols works and swapping genes removes the effect >
Yuki: So it's conveying the specific identity of the gene without needing complex sequence-level knowledge that isn't already in the host model >
Marcus: They also found that even if you don't have homologs for a gene, refitting the plug keeps most of its performance, which is a big deal because it means you don't need to train new adapters for every single gene variation >
Ines: And they explicitly state that the socket cannot add knowledge the host lacks; it just lets the host use what it already possesses >
Yuki: That’s a crucial distinction. It’s not about teaching the LLM new biology; it’s about guiding its existing knowledge to answer specific questions about DNA sequences >
Marcus: And they highlight that they have four hosts and three families of models, showing the modularity when you use text only maps to feed these different systems >
Ines: So in short, the summary is that this method fits a frozen biological encoder into the LLM's own text space at an early layer using this plug mechanism >
The paper's improvements: Yuki: I’m interested in the improvements they suggest because they tackle some of the limitations we see when connecting these two modalities, like needing a connector >
Ines: They improve it by showing that a closed-form map is enough, meaning you don't need to train a complex adapter or fine-tune the host model itself >
Marcus: They also improved the layer selection by using a rule based on slot readability that doesn't require any task items to be used when making that choice >
Yuki: That layer rule is clever because it measures whether the host can read text already written at those DNA positions before you even apply the plug scoring >
Ines: It’s not just about plugging in randomly; it’s about finding a way to choose the right layer where that information will actually be used by the model >
Marcus: And they showed that they can feed one DNA plug into four different hosts through a text only linear map, which shows how modular this whole socket concept is >
Yuki: It seems like the main improvement is making it training-free while still achieving results that are competitive against things like nearest neighbour retrieval in specific spaces >
Conclusion: Ines: So to wrap up on "A Training-Free Socket for DNA in Frozen Language Models", the main implication is that you can map a frozen biological encoder into an LLM's text representation using this socket mechanism without needing any training costs >
Marcus: Right. It lets you answer gene-level questions about sequences whose names you weren't explicitly told, and the results are impressive, showing gains of plus sixteen point seven to twenty-six point one points over shuffled DNA across four different hosts >
Yuki: It suggests that this identity transfer is more powerful than just a simple nearest neighbour lookup because the plug beats it by three point zero to five point two points on every host >
Ines: And they showed that even if you don't have homologs, the socket keeps most of its effect, which means you don't need to re-train for every new gene variant >
Marcus: Overall, this work establishes a new baseline for trained connectors by showing that a learned lookup of gene identity can achieve similar gains without any training cost >
Yuki: It’s a powerful tool because it lets us see how much knowledge we can transfer between these different biological and language systems with just the right mapping >
More episodes
- 2610.33881-Sensorimotor behavior from an untrained brain-and-nerve-cord connectome
- 2610.40344-Morphological Features of Murine Pulmonary Arterial Networks for Diagnosing Vascular Disease
- Daily Summary for 2026-11-19
- 2609.10193-Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets
- 2610.38236-Generative discovery of chemical reaction networks
- 2507.11176-An Interpretable AI framework Quantifying Traditional Chinese Medicine Principles Towards Enhancing and Integrating with Modern Biomedicine
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2504.03590-Towards a Unified Framework for Determining Conformational Ensembles of Disordered Proteins
- 2603.15217-A multiscale discrete-to-continuum framework for structured population models