When Does External Guidance Help LLM Reasoning? A Bias-Variance Theory of Guidance-Augmented GRPO
cs.LG, cs.CL
Submitted: 2026-07-08
Updated: 2026-07-08
Terminology
Sources
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Proximal Policy Optimization Algorithms
- Perception-Aware Policy Optimization for Multimodal Reasoning
- TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning
- Global Convergence of Policy Gradient Methods to (Almost) Locally Optimal Policies
- Trust Region Policy Optimization
- Policy Gradient with Active Importance Sampling
- Reusing Historical Trajectories in Natural Policy Gradient via Importance Sampling: Convergence and Convergence Rate
- Loaded DiCE: Trading off Bias and Variance in Any-Order Score Function Estimators for Reinforcement Learning
- Off Policy Lyapunov Stability in Reinforcement Learning
- Trajectory-Aware Eligibility Traces for Off-Policy Reinforcement Learning
- Reward Shaping via Diffusion Process in Reinforcement Learning
- From Reward Shaping to Q-Shaping: Achieving Unbiased Learning with LLM-Guided Knowledge
- Vanishing Bias Heuristic-guided Reinforcement Learning Algorithm
- Internally Rewarded Reinforcement Learning
- Causal-Paced Deep Reinforcement Learning
- Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction
- Reconciling modern machine learning practice and the bias-variance trade-off
- On lower bounds for the bias-variance trade-off
- Training language models to follow instructions with human feedback
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks