A Theoretical Analysis of Why Masked Diffusion Models Mitigate the Reversal Curse
Masked Diffusion Models mitigate the reversal curse by coupling forward and reverse positional conditionals through parameter-level storage and attention routing. Forward training strengthens position-invariant evidence, which is preserved by relative positional encodings, allowing the model to reuse this stored information for accurate reverse prediction.