Genome-Factory: A Library for Tuning, Deploying, and Interpreting Genomic Foundation Models

arXiv:2509.12266 · q-bio.GN, cs.LG · Submitted 2025-09-13 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.

Ines: I'm Ines.

Marcus: And I'm Marcus, here with Yuki.

Ines: We're looking at the paper: "Genome-Factory: A Library for Tuning, Deploying, and Interpreting Genomic Foundation Models".

Marcus: Here's the gist of the paper. GENOME-FACTORY introduces GENOME-FACTORY, the first integrated Python library for tuning, deploying, and interpreting genomic foundation models This framework simplifies and unifies the workflow for genomic model development: data collection, model tuning, inference, benchmarking, and interpretability Framework Components GENOME-FACTORY features six modular components to unify the workflow for genomic models The components include (i) Genome Collector which retrieves genomic sequences from public repositories and applies preprocessing such as GC content normalization and N nucleotide correction, (ii) Model Loader which supports diverse GFMs including HyenaDNA, DNABERT-2, Caduceus, Nucleotide Transformer, EVO and GenomeOcean as well as their tokenizers, (iii) Model Trainer which enables full-parameter fine-tuning and parameter-efficient methods such as low-rank adaptation (LoRA) and adapter tuning for both classification and regression tasks, (iv) Inference Engine which facilitates both embedding extraction and DNA sequence generation, (v) Benchmarker which provides two built-in, open-source genomic benchmarks and a plugin system for incorporating custom, domain-specific evaluation tasks, and (vi) Biological Interpreter which enhances model interpretability through a sparse autoencoder.

Ines: Fine-Tuning Efficiency The experiments demonstrate the utility of GENOME-FACTORY across three dimensions, including compatibility with diverse models and fine-tuning methods The adapter-based method consistently shows the highest efficiency in terms of trainable parameters, consuming less memory and achieving higher throughput compared to full fine-tuning for most models.

Marcus: For instance, the DNABERT-2 adapter uses only 0.2 million trainable parameters and reaches a throughput of 124,000 tokens per second.

Ines: LoRA tuning also offers strong efficiency gains as model size grows, reducing the parameter ratio and increasing memory savings for models like Nucleotide Transformer.

Title and authors: Ines: Let’s talk about the title again, "Genome-Factory: A Library for Tuning, Deploying, and Interpreting Genomic Foundation Models." It’s very descriptive of what they’re trying to achieve with this library.

Marcus: I think the authors are setting out to solve a real problem in the field. They point out that existing language model fine-tuning frameworks don't focus on biological interpretability, which is where scientists need it most for genomics.

Yuki: It’s interesting how they framed this as a unified platform because it addresses the fragmented nature of genomic AI development right now.

Ines: They are providing a way to streamline that entire lifecycle—from data collection and preprocessing to inference and finally interpretation—all in one place. It simplifies the workflow significantly, according to their abstract.

Marcus: It’s about reducing the technical barrier for researchers who aren't deep machine learning experts but still need access to these large genomic models. They emphasize making it user-friendly with a zero-code command line interface and a web based user interface.

Yuki: That accessibility is important because if you can’t easily run these complex tools, the potential impact on personalized medicine or evolutionary biology gets stuck in research labs.

Ines: And they are showing that this unification helps across three dimensions they validated: compatibility with diverse models, efficiency of fine-tuning methods, and the utility of the biological interpretation aspect.

Marcus: They’re trying to show that this isn't just a neat piece of software; it’s a framework that actually works across different genomic model types and tuning strategies.

The paper's summary: Ines: To summarize what the paper actually proposes, they are introducing GENOME-FACTORY as the first integrated Python library designed to tune, deploy, and interpret these genomic foundation models. It simplifies the entire development pipeline for genomic models.

Marcus: The core contribution is unifying data collection, model tuning—which can be full fine-tuning or parameter efficient methods—inference for both embeddings and sequence generation, benchmarking against existing tasks, and biological interpretation via a sparse autoencoder.

Yuki: It’s about providing that whole pipeline in one place. I imagine that makes it much easier to reproduce experiments because you don't have to stitch together dozens of different scripts from various repositories.

Ines: That’s what they are aiming for, making the process reproducible by standardizing the steps for every stage of model development. They also focus on how to handle diverse models like HyenaDNA or DNABERT-two through a single loader component <ref:2509.12266#pg1>.

Marcus: They are explicitly addressing the gap in current frameworks where biological interpretability isn't a main focus, and they are solving that by adding the sparse autoencoder workflow for interpretation.

Yuki: That’s a big step because it shifts the focus from just getting high accuracy scores to actually understanding what those models are learning about our biology.

The paper's improvements: Ines: The paper highlights some specific improvements they’ve built into this library, focusing heavily on how efficiently you can fine tune these models. They show that adapter-based methods, like LoRA and adapters, are often more efficient than full fine-tuning for most models.

Marcus: They give concrete numbers here. For example, they show the DNABERT-two adapter uses only zero point two million trainable parameters and hits a throughput of one hundred twenty-four thousand tokens per second. That’s a big efficiency gain over full fine-tuning.

Yuki: The fact that LoRA tuning also shows strong gains as model size increases, reducing the parameter ratio for models like Nucleotide Transformer, suggests this is a flexible approach that scales well with larger architectures.

Ines: Beyond just efficiency, they’ve added task-specific dataset builders within the Genome Collector. This means you can automatically generate data for specific genomic tasks like promoter region classification or enhancer region datasets.

Marcus: And that’s useful because it cuts down on a huge manual effort in preparing data for downstream tasks. It automates the creation of task-ready datasets, which is a big practical improvement for researchers.

Yuki: So, they are making the input side much more automated, which feeds into the training side and ultimately makes it easier to get results across different biological questions.

Conclusion: Ines: So wrapping this up, GENOME-FACTORY is presented as a unified infrastructure for tuning and deploying genomic foundation models by offering six modular components that handle everything from data collection to interpretation. It’s designed to lower the technical hurdle for using these models in research.

Marcus: The main implication is that we can get researchers using diverse genomic models, like DNABERT-two or GenomeOcean, much faster and more efficiently through parameter efficient methods like LoRA tuning <ref:2509.12266#pg1>.

Yuki: For me, it means we have a better tool to move beyond just prediction and actually start asking meaningful biological questions about the underlying sequence features because of that sparse autoencoder interpretation.

Ines: Exactly what I mean. It bridges the gap between complex machine learning architectures and practical biological discovery by making it easier to link model internals to measurable genomic properties.

Marcus: Overall, this paper provides a solid open-source infrastructure that supports broader benchmark coverage through built in tools and plugin systems, which is good for comparing different tuning strategies.

Yuki: It lays a foundation for more reproducible and responsible scientific innovation in this area by providing these standardized tools for the entire genomic model lifecycle.

Center for Foundation Models and Generative AI at Northwestern University · School of Natural Sciences at University of California at Merced, USA · Department of Energy Joint Genome Institute, Lawrence Berkeley National Laboratory, USA · Environmental Genomics and Systems Biology Division at Lawrence Berkeley National Laboratory, USA · Department of Statistics and Data Science at Northwestern University

q-bio.GN, cs.LG

Submitted: 2025-09-13

Updated: 2026-11-05

Code: https://github.com/WeiminWu2000/Genome_Factory

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 98/100

The gist: GENOME-FACTORY introduces GENOME-FACTORY, the first integrated Python library for tuning, deploying, and interpreting genomic foundation models This framework simplifies and unifies the workflow for…

Key concepts

Genome Collector
This component retrieves genomic sequences from public repositories. It performs necessary preprocessing steps, such as normalizing GC content and correcting 'N' nucleotides, ensuring the input data is clean and ready for model training.
Model Trainer
This module allows users to fine-tune various foundation models using different strategies. It supports full parameter fine-tuning as well as parameter-efficient methods like LoRA or adapter tuning, which significantly reduces computational needs and memory usage.
Biological Interpreter
This feature enhances model understanding by using a sparse autoencoder. It maps the complex internal model embeddings to specific, biologically meaningful latent dimensions, allowing researchers to see which parts of the model relate to known genomic features.
Benchmarker
The Benchmarker module is used for rigorous evaluation. It provides two built-in open-source genomic benchmarks and allows users to add custom tasks via a plugin system, enabling standardized comparison of different models and tuning strategies.

Terminology

Summary

GENOME-FACTORY introduces GENOME-FACTORY, the first integrated Python library for tuning, deploying, and interpreting genomic foundation models This framework simplifies and unifies the workflow for genomic model development: data collection, model tuning, inference, benchmarking, and interpretability

Framework Components

GENOME-FACTORY features six modular components to unify the workflow for genomic models The components include (i) Genome Collector which retrieves genomic sequences from public repositories and applies preprocessing such as GC content normalization and N nucleotide correction, (ii) Model Loader which supports diverse GFMs including HyenaDNA, DNABERT-2, Caduceus, Nucleotide Transformer, EVO and GenomeOcean as well as their tokenizers, (iii) Model Trainer which enables full-parameter fine-tuning and parameter-efficient methods such as low-rank adaptation (LoRA) and adapter tuning for both classification and regression tasks, (iv) Inference Engine which facilitates both embedding extraction and DNA sequence generation, (v) Benchmarker which provides two built-in, open-source genomic benchmarks and a plugin system for incorporating custom, domain-specific evaluation tasks, and (vi) Biological Interpreter which enhances model interpretability through a sparse autoencoder.

Fine-Tuning Efficiency

The experiments demonstrate the utility of GENOME-FACTORY across three dimensions, including compatibility with diverse models and fine-tuning methods The adapter-based method consistently shows the highest efficiency in terms of trainable parameters, consuming less memory and achieving higher throughput compared to full fine-tuning for most models. For instance, the DNABERT-2 adapter uses only 0. p. 5

Where the paper says it

Among all available methods, adapter tuning consistently shows the highest efficiency. For example, the DNABERT-2 adapter uses only 0.

2 million trainable parameters and reaches a throughput of 124,000 tokens per second. p. 5
Where the paper says it

2 million trainable parameters, consumes 2. 09 gigabytes of memory, and reaches a throughput of 124,000 tokens per second.

LoRA tuning also offers strong efficiency gains as model size grows, reducing the parameter ratio and increasing memory savings for models like Nucleotide Transformer. p. 5
Where the paper says it

7%, increases memory savings from 5% to 21%, and raises throughput from 1. These results show that LoRA becomes effective as model size grows, making it a strong choice for adapting large genomic models within GENOME-FACTORY.

Benchmarking Performance

The Benchmarker module provides essential tools for evaluating genomic foundation models on both classification and regression tasks The study compares six genomic foundation models with three different fine-tuning strategies across two benchmark suites, highlighting the role of the Benchmarker module in comparing model performance and tuning methods on standardized downstream tasks. Results indicate that GenomeOcean-100M achieves the strongest performance under full fine-tuning on the GUE benchmark, while DNABERT-2 performs best on Genomic Benchmarks. p. 7

Where the paper says it

The overall GUE score is computed by averaging this F1 value with the correlation coefficients from the remaining tasks. We observe that GenomeOcean-100M achieves the strongest performance under full fine-tuning on the GUE, while DNABERT-2 performs best on the Genomic Benchmarks.

Biological Interpretation Utility

The Biological Interpreter module enables meaningful interpretation of genomic foundation models by explicitly linking internal model representations to biological features This is achieved through a sparse auto-encoder workflow involving sequence embedding extraction, sparse auto-encoder training, and regression to external biological readouts. Analysis of the sparse auto-encoder latent dimensions reveals that specific latent dimensions capture biologically meaningful, motifcentric regulatory features. p. 8

Where the paper says it

0196, respectively. These results show that specific SAE latent dimensions capture biologically meaningful, motifcentric regulatory features.

Practical Usability and Future Directions

GENOME-FACTORY offers user-friendly interfaces through a zero-code command-line interface (CLI) and an intuitive Gradio-based web-based user interface (WebUI). The framework lowers the technical barrier to using large-scale models in genomics research Future work includes incorporating more memory and compute-efficient training strategies like quantization and optimized kernels, as well as integrating classical computational biology methods into the evaluation framework. p. 8

Where the paper says it

These results highlight its end-to-end usability for real-world analysis. GENOME-FACTORY lowers the technical barrier to using large-scale models in genomics research.

Responsible deployment also requires safeguards beyond user-level ethical guidelines, such as sequence screening API integration for high-risk generation tasks. p. 9
Where the paper says it

Responsible deployment therefore requires safeguards beyond user-level ethical guidelines. Future versions of GENOME-FACTORY can incorporate software-level guardrails, including sequence screening API integration, pathogen- or virulence-associated motif blocklists, riskaware generation filters, request logging and auditing, and configurable access controls for high-risk generation tasks.

Usage Examples

Users can access all GENOME-FACTORY functionalities through the command-line interface (CLI) or the web-based user interface (WebUI) Usage examples are provided for data downloading, processing, model training, inference, and biological interpretation via YAML configuration files. The system supports task-specific dataset generation tools for common genomic tasks such as promoter region datasets and enhancer region datasets. p. 1

Where the paper says it

It also includes task-specific dataset builders for histone modification, enhancer, and promoter classification, with automated region extraction and labeling. (ii) Model Loader: Supports diverse GFMs, including discriminative models—HyenaDNA (Nguyen et al.

The framework is designed to support both non-expert users and advanced developers with minimal computational and engineering effort. p. 1
Where the paper says it

These support both non-expert users and advanced developers with minimal computational and engineering effort. In summary, we have the following three main contributions: • We introduce GENOME-FACTORY, the first integrated Python framework to streamline the complete genomic model workflow.

Limitations Identified

Limitations include the high computational cost of full fine-tuning for very large models like EVO-7B, leading the framework to emphasize parameter-efficient adaptation methods. The context truncation mechanism may remove biologically meaningful elements in long-range benchmarks, necessitating future support for sliding-window evaluation. Furthermore, the integration of classical computational biology methods like PWM scanning and HMM-based motif finding is not yet systematic within the framework. p. 8

Where the paper says it

We include traditional baselines to establish a performance floor for foundation-model evaluation: a Convolutional Neural Network (CNN), Random Forest, and Support Vector Machine (SVM) classifiers trained on 4-mer frequency features, Position Weight Matrix (PWM)-based motif-scanning pipelines paired with Random Forest or linear SVM classifiers, and Hidden Markov Model (HMM)-based motif-finding methods. “—” marks settings we did not evaluate due to computational constraints.

The validation of multi-task learning across diverse task combinations remains an area for future study. p. 2
Where the paper says it

Standardizing dataset construction across diverse biological tasks, it ensures high-quality, task-ready inputs for the model. Furthermore, the framework supports custom multi-stage bioinformatics pipelines.

Model Support Details

The supported models vary by either parameter size or input sequence length, with options such as HyenaDNA supporting sequence lengths from 1K to 1M and Nucleotide Transformer supporting parameter sizes up to 2.5B. The framework includes specific variants for each model, such as DNABERT-2's parameter size of 117M and Caduceus's sequence length options. This detailed support helps users select the most suitable model for a customized task

Experimental Results Summary

The results show that GenomeOcean-4B achieves an average MCC of 60.43 on the GUE benchmark when fine-tuned with LoRA, while DNABERT-2 performs best on Genomic Benchmarks. p. 7

Where the paper says it

The overall GUE score is computed by averaging this F1 value with the correlation coefficients from the remaining tasks. We observe that GenomeOcean-100M achieves the strongest performance under full fine-tuning on the GUE, while DNABERT-2 performs best on the Genomic Benchmarks.

The analysis of the sparse auto-encoder latent dimensions confirms that latent 382, latent 519, and latent 3519 exhibit strong correlations with sequence length. p. 7
Where the paper says it

We train a logistic regression probe on SAE latent activations to classify CTCFbound sequences versus background sequences. In the sequence-length analysis, we find that latent 382, latent 519, and latent 3519 exhibit strong correlations with sequence length, suggesting that individual SAE latent dimensions encode interpretable sequence-level properties.

This demonstrates that the Biological Interpreter can move beyond black-box prediction by linking individual latent features to measurable genomic properties. p. 8
Where the paper says it

These results show that specific SAE latent dimensions capture biologically meaningful, motifcentric regulatory features. Together, these analyses demonstrate that the Biological Interpreter can move beyond black-box prediction by linking individual latent features to measurable genomic properties and regulatory signals.

The comparison between full fine-tuning and parameter-efficient methods shows a flexible trade-off space between predictive performance and computational efficiency. p. 7
Where the paper says it

This shows that a task-aware adapter improves the task performance. We visualize the trade-offs between predictive performance and computational efficiency for DNABERT-2, HyenaDNA, and Nucleotide Transformer in Figure 2 under three finetuning strategies: full fine-tuning, low-rank adaptation, and adapter-based methods.

Additional Benchmark Coverage

The framework supports broader benchmark coverage through additional experiments on epigenetic mark prediction with larger models, unsupervised multi-species representation evaluation, and genomic sequence-to-function benchmarks like GROVER and LRB. GenomeOcean-100M obtains the best performance on GUANinE when using LoRA fine-tuning, suggesting strong sequence-to-function adaptation under this protocol. p. 7

Where the paper says it

The overall GUE score is computed by averaging this F1 value with the correlation coefficients from the remaining tasks. We observe that GenomeOcean-100M achieves the strongest performance under full fine-tuning on the GUE, while DNABERT-2 performs best on the Genomic Benchmarks.

The results confirm that GENOME-FACTORY can support broader benchmark coverage and more diverse evaluation settings beyond the original GUE and Genomic Benchmarks experiments. p. 1
Where the paper says it

This provides deeper biological insights into models. • We validate the utility of GENOME-FACTORY across three dimensions: (i) Compatibility with diverse genomic foundation models and three widely used finetuning methods; (ii) Benchmarking downstream performance using two open-source benchmarks: Genome Understanding Evaluation (GUE) Benchmark (Zhou et al.

Conclusion

GENOME-FACTORY provides open-source infrastructure that bridges complex machine learning architectures and practical biological discovery by unifying the genomic model lifecycle By offering zero-code command-line and web interfaces, GENOMEFACTORY lowers the technical barrier to advanced genomic modeling. This can accelerate research in personalized medicine, evolutionary biology, and therapeutic design while supporting open, reproducible, and responsible scientific innovation. p. 9

Where the paper says it

By offering zero-code command-line and web interfaces, GENOMEFACTORY lowers the technical barrier to advanced genomic modeling and enables researchers with varying computational expertise to fine-tune, benchmark, and interpret genomic foundation models. This can accelerate research in personalized medicine, evolutionary biology, and therapeutic design while supporting open, reproducible, and responsible scientific innovation.

Responsible deployment requires safeguards beyond user-level ethical guidelines to mitigate dual-use biosecurity risks from generative sequence models. p. 9
Where the paper says it

At the same time, democratized access to generative sequence models introduces potential dual-use biosecurity risks, including the in silico generation of pathogenic, virulence-associated, or heavily modified viral sequences. Responsible deployment therefore requires safeguards beyond user-level ethical guidelines.

Acknowledgements

Weimin Wu would like to thank the anonymous reviewers and program chairs for constructive comments JH is partially supported by the Walter P. Murphy Fellowship and the Terminal Year Fellowship (Paul K. p. 9

Where the paper says it

JH is partially supported by the Walter P. Murphy Fellowship and the Terminal Year Fellowship (Paul K.

Richter Memorial Award) of Northwestern University Han Liu is partially supported by NIH R01LM1372201, NSF AST2421845, Simons Foundation MPS-AI-00010513, AbbVie, Dolby, and Chan Zuckerberg Biohub Chicago Spoke Award. p. 9
Where the paper says it

Richter Memorial Award) of Northwestern University. Han Liu is partially supported by NIH R01LM1372201, NSF AST2421845, Simons Foundation MPS-AI-00010513, AbbVie, Dolby, and Chan Zuckerberg Biohub Chicago Spoke Award.

Zhong Wang’s work conducted at the U. p. 9
Where the paper says it

Han Liu is partially supported by NIH R01LM1372201, NSF AST2421845, Simons Foundation MPS-AI-00010513, AbbVie, Dolby, and Chan Zuckerberg Biohub Chicago Spoke Award. Zhong Wang’s work conducted at the U.

S. Department of Energy Joint Genome Institute is supported by the Office of Science of the U. p. 9
Where the paper says it

Department of Energy Joint Genome Institute (https://ror. org/ 04xm1d337), a DOE Office of Science User Facility, is supported by the Office of Science of the U.

S. Department of Energy operated under Contract No. p. 9
Where the paper says it

org/ 04xm1d337), a DOE Office of Science User Facility, is supported by the Office of Science of the U. Department of Energy operated under Contract No.

DE-AC02-05CH11231. p. 9
Where the paper says it

Department of Energy operated under Contract No. DE-AC02-05CH11231.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Unified Genomic Model Workflow Management: The system can manage an end-to-end pipeline integrating "Genome Collector for acquiring and preprocessing data; Model Loader for accessing genomic models; Model Trainer for fine-tuning models tailored to specific downstream tasks; Inference Engine for embedding extraction and sequence generation; Benchmarker for evaluating model performance; and Biological Interpreter for interpretability via sparse autoencoders."

  2. Scalable Parameter-Efficient Fine-Tuning: The system supports full and parameter-efficient fine-tuning across diverse genomic models, specifically enabling users to choose between full, LoRA, and adapter tuning with configurable parameters like LoRA rank r = 8 and the scaling factor α = 32.

  3. Task-Specific Data Generation: The framework allows for automated creation of task-ready datasets through specialized tools such as the ability to generate a Promoter region dataset: Generate promoter vs. non-promoter classification data for human, mouse, and zebrafish genomes.

  4. Enhanced Model Interpretability: The system provides Biological interpretation of learned representations with DNABERT-2 by using a sparse autoencoder to disentangle embeddings into sparse latent units and aligns them with genomic features via regression against external biological readouts.

  5. Robust Performance Benchmarking: The Benchmarker module enables rigorous evaluation by supporting two built-in, open-source genomic benchmarks and allowing the integration of custom tasks via a plugin system.

Abstract

We introduce Genome-Factory, the first integrated Python library for tuning, deploying, and interpreting genomic foundation models. Our core contribution is to simplify and unify the workflow for genomic model development: data collection, model tuning, inference, benchmarking, and interpretability. For data collection, Genome-Factory offers an automated pipeline to download genomic sequences and preprocess them. For model tuning, Genome-Factory supports both full and parameter-efficient fine-tuning across diverse genomic models. For inference, Genome-Factory enables both embedding extraction and DNA sequence generation. For benchmarking, we include two existing benchmarks and provide a flexible interface to incorporate additional benchmarks. For interpretability, Genome-Factory introduces an open-source biological interpreter based on a sparse auto-encoder. We validate the utility of Genome-Factory across three dimensions: (i) Compatibility with diverse models and fine-tuning methods; (ii) Benchmarking downstream performance using two open-source benchmarks; (iii) Biological interpretation of learned representations with DNABERT-2. These results highlight its practical value for real-world genomic analysis. GitHub: https://github.com/WeiminWu2000/Genome Factory.

Sources

Related papers