Context--Answer Metamodels: Forecasting LLM Activations Before the First Answer Token
cs.LG, cs.AI
Submitted: 2026-11-04
Updated: 2026-11-19
Code: https://github.com/Tufalabs/context-answer-metamodels
Terminology
Sources
- Eliciting Latent Predictions from Transformers with the Tuned Lens
- Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Language
- Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution
- Predicting Future Behaviors in Reasoning Models Enables Better Steering
- Learning a Generative Meta-Model of LLM Activations
- No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probes
- LatentQA: Teaching LLMs to Decode Activations Into Natural Language
- Learning to Learn with Generative Models of Neural Network Checkpoints
- UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering
- Neural Network Diffusion
- The Truth Lies Somewhere in the Middle (of the Generated Tokens)
- Qwen2.5 Technical Report
- Instruction-Following Evaluation for Large Language Models
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis