Exploring the Design Space of Permutation Diffusion LLMs
The study investigates three ways to constrain language models for permutation generation: hard assignment, soft assignment, and direct ordering score prediction. These methods place constraints at different stages of training and decoding, leading to different objectives. Results show that while some methods achieve good ranking quality, others guarantee valid permutations without complex steps like Hungarian assignment. The key takeaway is that designing these components jointly is crucial.