Paper notes · arXiv 2025
Attention Is All You Need for KV Cache in Diffusion LLMs (Elastic-Cache)
Earlier caches refresh on a fixed schedule: Fast-dLLM at each block, dKV-Cache every steps. Elastic-Cache decides adaptively:
- Where to predict: a sliding window of the leftmost masks. Masks further right keep their cached K/V.
- When to refresh: when the attention of the most-attended token changes (cosine similarity ).
- Which layers: only from the layer where that change shows up, down to the last. Shallow layers keep their cache.
Algorithm 1, step by step. Drag γ to see refreshes come and go, and switch the far masks to recomputed for the ablation of Table 15.
far masks
σ vs γL1L2L3L4L5L6L7L8
promptanswer →shallow layers on top, deep layers below
Loading PDF… open it directly
PDF reproduced unmodified under CC BY 4.0. Highlights and notes are mine.