MMAI Gym for Science: Training Liquid Foundation Models for Drug Discovery
cs.LG, cs.AI, cs.CL
Submitted: 2026-03-03
Updated: 2026-08-31
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks.
Terminology
Abstract
General-purpose large language models (LLMs) that rely on in-context learning do not reliably deliver the scientific understanding and performance required for drug discovery tasks. Simply increasing model size or introducing reasoning tokens does not yield significant performance gains. To address this gap, we introduce the MMAI Gym for Science, a one-stop shop molecular data formats and modalities as well as task-specific reasoning, training, and benchmarking recipes designed to teach foundation models the 'language of molecules' in order to solve practical drug discovery problems. We use MMAI Gym to train an efficient Liquid Foundation Model (LFM) for these applications, demonstrating that smaller, purpose-trained foundation models can outperform substantially larger general-purpose or specialist models on molecular benchmarks. Across essential drug discovery tasks - including molecular optimization, ADMET property prediction, retrosynthesis, drug-target activity prediction, and functional group reasoning - the resulting model achieves near specialist-level performance and, in the majority of settings, surpasses larger models, while remaining more efficient and broadly applicable in the domain.
Sources
- Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs
- LFM2 Technical Report
- Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models
- Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models
- GLU Variants Improve Transformer
- Galactica: A Large Language Model for Science
- Kimi K2: Open Agentic Intelligence
- TxGemma: Efficient and Agentic LLMs for Therapeutics
- LlaSMol: Advancing Large Language Models for Chemistry with a Large-Scale, Comprehensive, High-Quality Instruction Tuning Dataset
- When Single Answer Is Not Enough: Rethinking Single-Step Retrosynthesis Benchmarks for LLMs
- MolTextNet: A Two-Million Molecule-Text Dataset for Multimodal Molecular Learning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks