Next Forcing: World Action Modeling with Multi-Chunk Prediction

summary

The gist

Autoregressive video generation has emerged as a powerful paradigm for World Action Models (WAMs) but existing approaches suffer from slow training convergence and limited converged accuracy…

In short

Next Forcing introduces a multi-chunk prediction (MCP) framework for autoregressive video generation to overcome slow training and limited accuracy at high frame rates. By forcing the model to predict multiple future video chunks simultaneously, it learns deeper temporal dynamics instead of relying on simple appearance shortcuts, leading to faster convergence and better generalization.

Key concepts

Multi-Chunk Prediction (MCP)
This framework augments the standard training by having auxiliary modules predict several future video chunks at once. This shifts the learning goal from predicting just the next frame to understanding long-range temporal evolution across multiple steps, which improves training efficiency.
Pruning Redundant Reasoning Tokens
The core idea is that predicting multiple chunks compels the model to learn underlying physical dynamics instead of memorizing visual patterns. This is validated by showing it trains auxiliary modules that predict several future tokens, boosting sample efficiency and inference speed.
Multi-Layer Feature Fusion
Instead of only using the final output layer for prediction, Next Forcing collects hidden states from four intermediate layers and combines them. These fused features are used by the MCP modules to supervise both early and late stages of the main model during training, providing temporally-aware feedback.

Terminology used across episodes

This episode discusses

The paper

Next Forcing: World Action Modeling with Multi-Chunk Prediction · Read on arXiv

Gangwei Xu, Qihang Zhang, †Jiaming Zhou, Xing Zhu, ℘Yujun Shen, Xin Yang, ℙYinghao Xu

Robbyant University of Science and Technology Shanghai Institute of Technology Hefei National University Institute of Technology Hefei National University

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Next Forcing: World Action Modeling with Multi-Chunk Prediction".

Jane: Here's the gist of the paper. Autoregressive video generation has emerged as a powerful paradigm for World Action Models (WAMs) but existing approaches suffer from slow training convergence and limited converged accuracy, particularly at high frame rates Next Forcing, a multi-chunk prediction (MCP) framework for causal world modeling that enables faster training, higher accuracy, and accelerated inference Pruning redundant reasoning tokens The central insight is that turning the local single-chunk objective into a long-range multi-chunk objective forces the model to learn the latent dynamics governing temporal evolution rather than relying on appearance shortcuts This idea has been validated in language modeling where multitoken prediction (MTP) trains auxiliary modules to predict multiple future tokens, improving sample efficiency and enabling inference acceleration Next Forcing augments the main model with a small set of auxiliary MCP modules that simultaneously predict video chunks at multiple future horizons (next1, next2, next3) and form a causal chain across prediction depths How the training signal is built The framework augments the standard teacher-forcing training objective with multi-chunk prediction (MCP), where auxiliary modules predict video chunks at multiple next temporal horizons in addition to the current chunk The MCP modules form a causal chain across prediction depths, where intermediate features fused from multiple layers of the main model are leveraged to predict future dynamics, allowing near-future predictions to inform farther-future ones and providing dense multi-scale temporal supervision back to the main model The MCP modules are trained with a higher timestep shift than the main model, forcing them to rely more heavily on the main model’s representations and tightening the coupling between the two Where the gains come from The dramatic improvement at 50 fps can be understood through supervision signal density, as adjacent chunks are nearly identical, making next-chunk denoising trivially solvable via appearance copying Multi-chunk prediction breaks this shortcut by exhibiting substantial visual differences that can only be predicted by understanding the underlying physical dynamics The MCP modules fuse intermediate features from multiple layers of the main model rather than only the final output, allowing the MCP supervision signal to influence the main model’s representations at various depths through gradient backpropagation Inference and Generalization The MCP modules can be retained at inference in two modes: Zero-Overhead Mode, where all modules are discarded for zero-overhead deployment, or Parallel Chunk Generation Mode, where they predict the next video chunk in parallel with the current one Next Forcing establishes new state-of-the-art results on RoboTwin (94.1/93.5% on Clean/Random) and achieves 2× inference acceleration Furthermore, Next Forcing demonstrates over 50% FVD reduction on general video pretraining with 3.5M in-house video clips, confirming its effectiveness beyond robot-specific data Next Forcing establishes new state-of-the-art results on RoboTwin (94.1/93.5% on Clean/Random), achieves 2.3× faster convergence at 50 fps, enables inference acceleration, and demonstrates over 50% FVD reduction on general video pretraining, validating its generality beyond robot-specific data The main limitation is that the MCP modules introduce extra training cost Next Forcing motivates further investigation of training objectives, beyond context construction and noise scheduling, as a key axis for improving autoregressive video generation Page 9 --- Table 5 provides the full training convergence data at 12 fps and 50 fps, complementing the convergence curves in Figure 1 Next Forcing shows the largest absolute improvement in the early training stages (5k–10k steps), indicating that multi-chunk prediction provides a stronger learning signal from the start by preventing appearance shortcuts At 50 fps, however, adjacent chunks are nearly identical, making it easy for the model to shortcut by simply copying the previous chunk Page 4 --- Figure 3 shows qualitative comparison on PhyWorld where Next Forcing generates more physically consistent dynamics Next Forcing improves both FVD and Abnormal Ratio over LingBot-VA, with larger gains on the out-of-template setting, suggesting that multi-chunk prediction encourages generalizable physical dynamics rather than template-specific memorization Page 5 --- Table 4 shows that MCP-accelerated inference predicts the next video chunk in parallel via the MCP module, yielding 2× inference acceleration The depth-2 and depth-3 MCP modules are not used in this mode, as their predictions are superseded by the main model in the next step Page 5 --- Table 5 provides the full training convergence data at 12 fps and 50 fps, complementing the convergence curves in Figure 1 Next Forcing shows the largest absolute improvement in the early training stages (5k–10k steps), indicating that multi-chunk prediction provides a stronger learning signal from the start by preventing appearance shortcuts At 50 fps, however, adjacent chunks are nearly identical, making it easy for the model to shortcut by simply copying the previous chunk Page 3 --- Figure 2 illustrates the overall framework where the main model denoises the current chunk while chained MCP modules predict future chunks using features from the main model The MCP modules share the same attention mask as the main model, requiring only a single mask construction per training step for efficient training Page 4 --- The key idea of Next Forcing is to extend the prediction target from the current chunk to multiple next chunks, providing explicit supervision about future dynamics This encourages trajectory-level temporal reasoning during training and directly addresses the myopic supervision problem Page 5 --- Temporal Chunk Shifting involves constructing a temporally shifted target x

k: 0 by advancing the video latent by k chunks Each shifted target is independently noised using the flow matching formulation with its own timestep and noise sample To inform the MCP modules of their temporal offset, the chunk shift is incorporated into the rotary position embeddings (RoPE)

53: so that each MCP module receives positional information for the future chunk it predicts Page 4 --- Multi-Layer Feature Fusion involves collecting hidden states at 4 intermediate layers and concatenating them along the feature dimension and compressing through a two-layer MLP During backpropagation, the MCP loss gradients flow through hfuse back into these intermediate layers, providing temporally-aware supervision to both the early and late stages of the main model Page 5 --- The MCP modules share the same attention mask as the main model, requiring only a single mask construction per training step for efficient training This allows depth-2 predictions to build on depth-1 features, and depth-3 on depth-2 Page 5 --- The total loss combines the main losses and the MCP losses, with L = Lvideo + Laction + X3k=1 wk · LMCPk The complete training objective combines the main losses and the MCP losses Page 4 --- The total loss is L = Lvideo + Laction + X3k=1 wk · LMCPk The complete training objective combines the main losses and the MCP losses

Paper summary: Tom: So looking at the whole "Next Forcing: World Action Modeling with Multi-Chunk Prediction" paper, it’s about using a multi-chunk prediction objective to train world action models, which results in faster training and higher accuracy compared to previous methods.

Jane: Essentially, the authors claim this framework enables faster convergence and better accuracy by forcing the model to learn long-range temporal dynamics rather than just short-term appearance copying.

Lu: The implication is that this approach directly addresses the problem of myopic supervision in autoregressive video generation by providing explicit supervision about future dynamics across multiple chunks.

Meng: From a deployment standpoint, they offer a choice between zero-overhead inference or parallel chunk generation, which helps with operational speed while still leveraging the model's predictive power.

Lalam: The paper establishes new state of the art results on RoboTwin with accuracy numbers around ninety-four point one percent on clean data and ninety-three point five percent on random data when using this method at both twelve frames per second and fifty frames per second <ref:2606.11187#pg2>.

Tom: And they also show it can reduce frame-video difference by over fifty percent in general video pretraining, validating its effectiveness beyond just robot-specific data.

Jane: The main limitation they state is that the MCP modules introduce extra training cost, which motivates future work to investigate other training objectives besides context construction and noise scheduling for improving autoregressive video generation.

Conclusion: Tom: So we've been looking at Next Forcing, and now it’s time to talk about what this paper is actually trying to achieve with its title and who wrote it.

Jane: The title is "Next Forcing: World Action Modeling with Multi-Chunk Prediction," and the authors are trying to fix a problem where models learn things too quickly.

Tom: Right, they’re tackling slow training convergence and limited accuracy in autoregressive video generation by changing how the model looks at time.

Lu: The core idea is pruning redundant reasoning tokens by turning that local prediction into a long-range multi-chunk objective. They argue this forces the model to learn the underlying dynamics of how things move over time, not just short appearance shortcuts.

Jane: So, instead of just looking at one frame next, the model has to predict several frames ahead in a chain. That’s what they call Multi-Chunk Prediction or MCP.

Meng: From my side, I’m curious about the cost of this setup—does it make deployment more expensive? Because if we're shipping models to production, overhead matters way more than training speed sometimes.

Tom: The paper says the main limitation is that these MCP modules do introduce extra training cost, but they show massive gains in early training stages and inference speed.

Jane: They show two times faster convergence at fifty frames per second, and they even manage to reduce frame-video difference by over fifty percent when tested on general video data.

Lu: The implication is that this method isn't just about making a specific robot model better; it shows generalizable physical dynamics when trained on diverse footage.

Tom: It seems like the authors are suggesting that for next generation models, we need to look beyond noise scheduling and context construction for training objectives.

Jane: So, if this works well on general video, what does that mean for the kind of complex action modeling we hope to see in real-world applications? (Music swells slightly)

More episodes

← Home