Diffusion Guidance Is a Controllable Policy Improvement Operator
cs.LG
Submitted: 2025-05-29
Updated: 2026-11-11
Code: https://github.com/kvfrans/cfgrl
Project page: https://diffusionflow.github.io
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control
- IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policies
- DiffCPS: Diffusion Model based Constrained Policy Search for Offline Reinforcement Learning
- AlignIQL: Policy Alignment in Implicit Q-Learning through Constrained Optimization
- Gaussian Error Linear Units (GELUs)
- Classifier-Free Diffusion Guidance
- Reward-Conditioned Policies
- Flow Matching Guide and Code
- Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbone
- AWAC: Accelerating Online Reinforcement Learning with Offline Datasets
- Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning
- Proximal Policy Optimization Algorithms
- Behavior Regularized Offline Reinforcement Learning
- Policy Representation via Diffusion Probability Model for Reinforcement Learning
- Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning
Related papers
- A Physics-Regulated Neural Framework for Learning 3D Grain Growth Dynamics
- FlexMoE: Learning One-for-All Multi-Budget Expert Pruning for Flexible MoE Deployment
- OptiMind: Improving Small Language Models for Optimization Modeling through Formulation Guidance and Multi-Turn Learning
- Staying in the Sweet Spot: Boost Reasoning Evolution via Responsive Rollout Difficulty Manipulation
- Examining the Impact of Data and Training Methods on Emergent and Subliminal Misalignment
- Rethinking Black-Box On-Policy Distillation: A Rubric-based Analysis