Unsupervised Post-Training of Foundation Models: A Survey
cs.CL, cs.AI, cs.CV, cs.LG, cs.MM
Submitted: 2026-08-25
Updated: 2026-08-27
Comments: Accepted to Findings of EMNLP 2026. 20 pages, 3 figures, 8 tables
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Foundation-model post-training usually relies on human labels, preference data, stronger teachers, or executable verifiers.
Terminology
Abstract
Foundation-model post-training usually relies on human labels, preference data, stronger teachers, or executable verifiers. We study Unsupervised Post-Training (UPT): update-bearing adaptation on unlabeled inputs whose learning signal is derived from same-lineage model artifacts rather than an external oracle. We catalog 80 strict UPT methods and organize them by the object that supplies the update signal: a prediction statistic, a sample relation, a self-generated target, or an internal evaluator. Beyond inventory, we show how the choice of internal signal and task structure determines whether post-training improves the model or recursively amplifies error. An orthogonal Input Visibility times Update Persistence view maps deployment regimes and defines a unified framework for UPT selection and evaluation.
Sources
- Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
- Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
- Self-Improvement in Multimodal Large Language Models: A Survey
- Distribution-Aware Reward Estimation for Test-Time Reinforcement Learning
- Dual Consensus: Escaping from Spurious Majority in Unsupervised RLVR via Two-Stage Vote Mechanism
- Data Engineering for Scaling Language Models to 128K Context
- LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
- AERO: Autonomous Evolutionary Reasoning Optimization via Endogenous Dual-Loop Feedback
- One-shot Entropy Minimization
- G-Zero: Self-Play for Open-Ended Generation from Zero Data
- R-Zero: Self-Evolving Reasoning LLM from Zero Data
- Simple and Scalable Strategies to Continually Pre-train Large Language Models
- Model Whisper: Steering Vectors Unlock Large Language Models' Potential in Test-time
- TTSR: Test-Time Self-Evolving via Reflection
- Continual Pre-training of Language Models
- Self-Calibrating Language Models via Test-Time Discriminative Distillation
- LLM Post-Training: A Deep Dive into Reasoning Large Language Models
- SUDER: Self-Improving Unified Large Multimodal Models for Understanding and Generation with Dual Self-Rewards
- SLOT: Sample-specific Language Model Optimization at Test-time
- Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering