LoopRPT: Reinforcement Pre-Training for Looped Language Models
LoopRPT introduces a reinforcement pre-training framework for Looped Language Models (LoopLMs) by treating next-token prediction as a reasoning task. It assigns reinforcement signals directly to internal latent steps using an EMA teacher and noisy rollouts. This method shapes intermediate representations, leading to better per-step quality and improved accuracy across various tasks while reducing total inference steps.