Reconsidering Positional Supervision in Masked Diffusion Language Model Training
cs.CL, cs.LG
Submitted: 2026-01-30
Updated: 2026-11-05
Code: https://github.com/open-compass/opencompass
Terminology
Sources
- Program Synthesis with Large Language Models
- LLaDA2.0: Scaling Up Diffusion Language Models to 100B
- MaskGIT: Masked Generative Image Transformer
- Training Verifiers to Solve Math Word Problems
- LV-CTC: Non-autoregressive ASR with CTC and latent variable models
- The Llama 3 Herd of Models
- Sequence Transduction with Recurrent Neural Networks
- Speech Recognition with Deep Recurrent Neural Networks
- Edit Flows: Flow Matching with Edit Operations
- Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predict
- Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models
- Learning Unmasking Policies for Diffusion Language Models
- Mercury: Ultra-Fast Language Models Based on Diffusion
- Any-Order Flexible Length Masked Diffusion
- Fine-Tuning Masked Diffusion for Provable Self-Correction
- Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models
- Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models
- Large Language Diffusion Models
- Path Planning for Masked Diffusion Model Sampling
- Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference
Related papers
- ALARM: Audio-Language Alignment for Reasoning Models
- ThinkSum: Probabilistic reasoning over sets using large language models
- Coherence boosting: When your pretrained language model is not paying enough attention
- From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
- Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data