How to build a diffusion language model

Diffusion language models generate entire token sequences from an initial noisy guess, iteratively refining them through denoising steps that incorporate bidirectional context and allow speed-quality trade-offs. By 2026, models such as Mercury 2 from Inception Labs have matched autoregressive quality, using forward noise addition and reverse denoising training adapted from image diffusion techniques.