Trust the Critic More
cs.LG, cs.AI
Submitted: 2026-09-30
Updated: 2026-10-01
Code: https://github.com/WhenWen/AC2
Sources
- MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
- Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal States
- DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Reinforcement Learning with Action Chunking
- Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding
- QED-Nano: Teaching a Tiny Model to Prove Hard Theorems
- EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
- Best Practice Critic Optimization
- Proximal Policy Optimization Algorithms
- High-Dimensional Continuous Control Using Generalized Advantage Estimation
- Reuse your FLOPs: Scaling RL on Hard Problems by Conditioning on Very Off-Policy Prefixes
- Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Qwen3 Technical Report
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks
- $V_{0.5}$: Generalist Value Model as a Prior for Sparse RL Rollouts
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks