Thinking Seeds: Restarting On-Policy RL from the Model's Own Past Reasoning States

Name: Thinking Seeds: Restarting On-Policy RL from the Model's Own Past Reasoning States
Uploaded: 2026-10-22T19:24:37.923920+00:00
Duration: 1524 s
Description: Thinking Seeds introduces a token-level mix-policy framework for reinforcement learning in language models. It solves the problem of policy entropy collapsing during training by using historical model checkpoints as off-policy prefixes and on-policy continuations for gradient quality. This position-aware approach maintains diversity and improves performance across benchmarks.

Thinking Seeds introduces a token-level mix-policy framework for reinforcement learning in language models. It solves the problem of policy entropy collapsing during training by using historical model checkpoints as off-policy prefixes and on-policy continuations for gradient quality. This position-aware approach maintains diversity and improves performance across benchmarks.

Open the full episode