Next-Latent Prediction Transformers Learn Compact World Models
cs.LG
Submitted: 2025-11-08
Updated: 2026-11-18
Code: https://github.com/JaydenTeoh/NextLat
Terminology
Sources
- GPT-4 Technical Report
- Efficient Joint Prediction of Multiple Future Tokens
- SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
- A Cookbook of Self-Supervised Learning
- Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
- The Free Transformer
- Language models scale reliably with over-training and on downstream tasks
- Stream of Search (SoS): Learning to Search in Language
- Think before you speak: Training Language Models With Pause Tokens
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Efficiently Modeling Long Sequences with Structured State Spaces
- World Models
- Dream to Control: Learning Behaviors by Latent Imagination
- Temporal Difference Learning for Model Predictive Control
- Gaussian Error Linear Units (GELUs)
- Distilling the Knowledge in a Neural Network
- The Belief State Transformer
- MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
- LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architectures
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis