dLLM Notes

Reading list · 13 papers

Paper notes

The original PDFs, with my highlights and notes pinned in the margin.

  1. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory DistillationICMLLearn the decoding order from the teacher, decode several blocks at once, and score speed and accuracy as one curve.Yu-Yang Qian, Junda Su, Lanxiang Hu, Peiyuan Zhang, Zhijie Deng, Peng Zhao, Hao Zhang
  2. 2026DFlash: Block Diffusion for Flash Speculative DecodingICMLA five-layer block diffusion drafter, fed with the target model's hidden features, guesses 16 tokens in one pass.Jian Chen, Yesheng Liang, Zhijian Liu
  3. 2026Large Language Continuous Diffusion Models (Nemotron-Labs-Sigma)arXivGaussian noise on 16-dimensional token embeddings, at 3B and 8B, against a masked model trained the same way.Zhihan Yang, Wei Guo, Jean-Marie Lemercier, Simon Welker, Mohammad Mahdi Kamani, Yonggan Fu, Sajad Norouzi, Julius Berner, Tomas Geffner, Karsten Kreis, Yongxin Chen, Molei Tao, John Thickstun, Pavlo Molchanov, Ante Jukić, Arash Vahdat, Morteza Mardani
  4. 2026Multi-Block Diffusion Language Models (MBD-LMs)arXivDecode two or three blocks at once, and train for what that actually looks like.Yijie Jin, Jiajun Xu, Yuxuan Liu, Chenkai Xu, Yi Tu, Jiajun Li, Dandan Tu, Xiaohui Yan, Kai Yu, Pengfei Liu, Zhijie Deng
  5. 2025DPad: Efficient Diffusion Language Models with Suffix DropoutarXivMost of the masked suffix is never needed. Keep a window of nearby masks, thin it with a Gaussian, and drop the rest before the forward pass.Xinhua Chen, Sitao Huang, Cong Guo, Chiyue Wei, Yintao He, Jianyi Zhang, Hai "Helen" Li, Yiran Chen
  6. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing (D2F)arXivDistill a full-attention dLLM into a block-causal one that can start the next block before the last one is done.Xu Wang, Chenkai Xu, Yijie Jin, Jiachun Jin, Hao Zhang, Zhijie Deng
  7. 2025Attention Is All You Need for KV Cache in Diffusion LLMs (Elastic-Cache)arXivRefresh the cache only when the most-attended token moves, and only from that layer down. Masks far to the right stay cached.Quan Nguyen-Tri, Mukul Ranjan, Zhiqiang Shen
  8. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMarXivTurn Qwen2.5 into a block diffusion model with ~1B tokens of fine-tuning, then decode with a block-level cache plus a sub-block DualCache.Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao, Yonggan Fu, Zhijian Liu, Pavlo Molchanov, Ping Luo, Song Han, Enze Xie
  9. 2025Taming the Memory Footprint Crisis: System Design for Production Diffusion LLM ServingarXivA serving system for dLLMs: cap the logit spike, schedule Refresh and Reuse phases separately, store per-head sparse KV densely.Jiakun Fan, Yanglin Zhang, Xiangchen Li, Dimitrios S. Nikolopoulos
  10. 2026TEAM: Temporal–Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model AccelerationarXivMoE dLLMs load far more experts per step than they accept tokens. Cache decoded tokens, fence in cold masks, speculate on hot ones.Linye Wei, Zixiang Luo, Pingzhi Tang, Meng Li
  11. 2025dKV-Cache: The Cache for Diffusion Language ModelsNeurIPSCache a token only after it is decoded, and one step late. No blocks, no retraining.Xinyin Ma, Runpeng Yu, Gongfan Fang, Xinchao Wang
  12. 2026dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive CachingICMLRefresh the prompt rarely, the response often, and only the response tokens that actually changed.Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyan Wei, Shaobo Wang, Yichen Zhu, Linfeng Zhang
  13. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel DecodingarXivAn approximate KV cache for bidirectional attention, plus confidence-thresholded parallel decoding. No retraining.Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, Enze Xie