dLLM Notes

Paper notes · arXiv 2025

Fast-dLLM v2: Efficient Block-Diffusion LLM

Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao, Yonggan Fu, Zhijian Liu, Pavlo Molchanov, Ping Luo, Song Han, Enze Xie

Fast-dLLM (v1) sped up LLaDA and Dream without training, but its cache was an approximation. v2 changes the model instead: fine-tune an AR model (Qwen2.5-1.5B/7B) into a block diffusion model, causal across blocks of 32 and bidirectional inside each block. Then:

Result: on par with Qwen2.5-7B fine-tuned on the same data with next-token loss (avg 60.3 vs 59.6), and 2.5× its throughput at threshold 0.9.

Loading PDF… open it directly

PDF reproduced unmodified under CC BY-NC-ND 4.0. Highlights and notes are mine.