The paradox of diffusion distillation (2024)

Diffusion models generate data by iteratively denoising small steps, because large steps produce blurry averages that lose high-frequency detail. Recent work uses distillation—training a student model with a teacher’s predictions—to reduce the required sampling steps, even achieving single-step generation while preserving quality.