Continuous Diffusion Language Models (CDLM's)

Continuous diffusion models for language are resurging after a period of dormancy, challenging the autoregressive dominance that has driven large language models. Early discrete diffusion attempts in 2021, such as multinomial diffusion and SUNDAE, gave way to 2022’s continuous approaches like Diffusion-LM, which embed tokens as vectors to permit Gaussian noise corruption and enable controllable text generation. Recent papers—including DiffuSeq, SSD-LM, Difformer, SeqDiffuSeq, GENIE, and LD4LG—extend this paradigm, reviving interest in diffusion-based language modeling.