dLLM Notes

Paper notes · arXiv 2026

Multi-Block Diffusion Language Models (MBD-LMs)

Yijie Jin, Jiajun Xu, Yuxuan Liu, Chenkai Xu, Yi Tu, Jiajun Li, Dandan Tu, Xiaohui Yan, Kai Yu, Pengfei Liu, Zhijie Deng

A block diffusion model (M08) decodes one block at a time. The next block waits until this one is finished and its K/V are written to the cache. D2F (my notes), from the same group, lets several blocks run at once. This paper asks why that works less well than it should, and fixes two things:

Decoding: 5 blocks of 4 tokens, all three lanes in step. Orange outlines mark the blocks in this forward pass. A finished block is stored in the next pass. Single-block decoding spends that pass on storing alone; the multi-block lanes overlap it with decoding. The toy rates are illustrative: the untrained model commits in its second slot every other step, the MultiTF model every step. Training states: what each recipe shows the model around block 4.

Result on LLaDA2-Mini: 3.47 → 6.19 tokens per forward, accuracy 79.95 → 81.03, and 517 → 746 tokens/s (Tables 1 and 3).

Loading PDF… open it directly

PDF reproduced unmodified under CC BY 4.0. Highlights and notes are mine.