Latent JEPA: Abstract Future Prediction for Latent Reasoning in Chemistry
cs.LG, cs.CL
Submitted: 2026-10-01
Updated: 2026-10-01
License: http://creativecommons.org/licenses/by/4.0/
The gist: Large language models offer a promising foundation for chemical reasoning, bringing together chemical knowledge and multistep problem solving.
Terminology
Abstract
Large language models offer a promising foundation for chemical reasoning, bringing together chemical knowledge and multistep problem solving. Chemical intuition can provide an initial sense of plausible outcomes before the details of a solution are fully worked out. Inspired by how such expectations complement explicit analysis, we study how continuous latent thoughts can be trained to anticipate informative aspects of future solutions without verbalizing every intermediate step. We introduce Latent JEPA, a framework that combines autoregressive learning with joint-embedding prediction of one or more future views. For chemical reasoning, we develop textual and molecular prediction objectives that connect latent thoughts to both subsequent reasoning and molecular outcomes. Experiments on ChemCoTBench show gains in molecular optimization and on several editing and reaction metrics. Representation analyses show that future prediction makes latent thoughts more informative about molecular outcomes and strengthens their correspondence with chemical structure. These findings support abstract future prediction as a learning principle for connecting continuous latent reasoning with scientific outcomes.
Sources
- VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
- Revisiting Feature Prediction for Learning Visual Representations from Video
- LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
- ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
- Intuitive physics understanding emerges from self-supervised pretraining on natural videos
- Training Large Language Models to Reason in a Continuous Latent Space
- Qwen3 Technical Report
- LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning
- MolEmb: Multimodal Large Language Models Can Be Strong Molecular Embedding Models
- RetroAgent: Harnessing LLMs to Search Over Structured Memory for Agentic Retrosynthesis Planning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks