dLLM Notes

Paper notes · NeurIPS 2025

dKV-Cache: The Cache for Diffusion Language Models

Xinyin Ma, Runpeng Yu, Gongfan Fang, Xinchao Wang

A KV cache for masked diffusion that does not need semi-autoregressive blocks. The observation: a token’s keys and values jump when it goes from [MASK] to decoded, then barely move. So cache decoded tokens only, one step after they are decoded, keep recomputing the masked ones, and refresh everything every few steps.

§3.3 step by step: one answer token is decoded per step, in a non-sequential order. Switch the options to compare caching at once against one step late, and Decode against PD.

cache a decoded token
refresh
prompt at refresh

Toy example: 4 prompt tokens, 8 answer positions, one token decoded per step. Rows are steps, columns are positions. A token decoded at step t appears in the input at step t+1, where its K/V jump. dKV-Cache-Decode computes it once more at step t+2 and caches that settled value. Refresh rows (⟳) rebuild the cache.

Loading PDF… open it directly

PDF reproduced unmodified under CC BY 4.0. Highlights and notes are mine.