Compile as Hypothesis: A Headroom Typology for Prompt Optimization on Real Reasoning Models
cs.AI
Submitted: 2026-10-31
Updated: 2026-10-31
Terminology
Sources
- GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
- Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation
- SEPO: Evidence-Grounded Prompt Optimization via Structural Editing
- FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets
- Training Verifiers to Solve Math Word Problems
- Measuring Mathematical Problem Solving With the MATH Dataset
- DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
- Holistic Evaluation of Language Models
- Humanity's Last Exam
- Automatic Prompt Optimization with "Gradient Descent" and Beam Search
- GPQA: A Graduate-Level Google-Proof Q&A Benchmark
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
- Safe RLHF: Safe Reinforcement Learning from Human Feedback
- Large Language Models as Optimizers
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
- Large Language Models Are Human-Level Prompt Engineers
Related papers
- Multi-Objective Structured Pruning of LLMs for Latency and Model Size Optimization
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Embeddings for Preferences, Not Semantics
- Can We Trust Item Response Theory for AI Evaluation?
- Diagnosing Task Insensitivity in Language Agents