Vector Policy Optimization: Training for Diversity Improves Test-Time Search
cs.LG, cs.AI, cs.CL, cs.NE
Submitted: 2026-05-21
Updated: 2026-11-05
Code: https://github.com/algorithmicsuperintelligence/openevolve
Terminology
Sources
- The Best of N Worlds: Aligning Reinforcement Learning with Best-of-N Sampling via max@k Optimisation
- InfAlign: Inference-aware language model alignment
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
- Evaluating Large Language Models Trained on Code
- Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models
- Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models
- Training Verifiers to Solve Math Word Problems
- UltraFeedback: Boosting Language Models with Scaled AI Feedback
- KL-Regularized Reinforcement Learning is Designed to Mode Collapse
- Where does output diversity collapse in post-training?
- ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
- GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
- Random Latent Exploration for Deep Reinforcement Learning
- AlphaEvolve: A coding agent for scientific and algorithmic discovery
- Curiosity-driven Exploration by Self-supervised Prediction
- Rollout Roulette: A Probabilistic Inference Approach to Inference-Time Scaling of LLMs using Particle-Based Monte Carlo Methods
- Reaching Beyond the Mode: RL for Distributional Reasoning in Language Models
- ToolRL: Reward is All Tool Learning Needs
- POPE: Learning to Reason on Hard Problems via Privileged On-Policy Exploration
- e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis