dLLM Notes

Paper notes · arXiv 2025

Attention Is All You Need for KV Cache in Diffusion LLMs (Elastic-Cache)

Quan Nguyen-Tri, Mukul Ranjan, Zhiqiang Shen

Earlier caches refresh on a fixed schedule: Fast-dLLM at each block, dKV-Cache every NN steps. Elastic-Cache decides adaptively:

Algorithm 1, step by step. Drag γ to see refreshes come and go, and switch the far masks to recomputed for the ablation of Table 15.

far masks
σ vs γL1L2L3L4L5L6L7L8

promptanswer →shallow layers on top, deep layers below

Toy run: 4 prompt tokens, 16 answer positions, 8 layers, window β = 4. Each row is a layer, each column a position. Only the window, the tokens decoded last step and the watched ★ token run as queries. The σ column is the cosine similarity of ★'s attention row to last step's, per layer. The first layer below γ (dashed) triggers a recompute of everything below it. Drift values are made up; their shape follows Fig. 1: larger in deeper layers, and spiking after a content word such as Paris is decoded.

Loading PDF… open it directly

PDF reproduced unmodified under CC BY 4.0. Highlights and notes are mine.