Self-Augmenting Retrieval for Diffusion Language Models

Name: Self-Augmenting Retrieval for Diffusion Language Models
Uploaded: 2026-11-01T16:36:21.321434+00:00
Description: Discrete diffusion models generate text by iteratively denoising a response in parallel. This process discards low-confidence tokens, but these discarded tokens actually reveal useful lookahead signals for retrieval. SARDI is a dynamic RAG framework that uses these intermediate predictions to guide retrieval during denoising, significantly boosting performance over existing methods.

Discrete diffusion models generate text by iteratively denoising a response in parallel. This process discards low-confidence tokens, but these discarded tokens actually reveal useful lookahead signals for retrieval. SARDI is a dynamic RAG framework that uses these intermediate predictions to guide retrieval during denoising, significantly boosting performance over existing methods.

Open the full episode