Paper notes · arXiv 2025
DPad: Efficient Diffusion Language Models with Suffix Dropout
A training-free method for full-attention dLLMs (LLaDA, Dream). Every step of vanilla decoding runs the whole masked suffix, though only the current block is decoded. DPad cuts the suffix down before the forward pass:
- Sliding window: keep only the masks right after the current block. The window moves with the block, so the suffix cost no longer grows with the generation length.
- Distance-decay dropout: inside the window, keep each mask with a probability that falls off like a Gaussian with distance.
The kept set is drawn once per block, the RoPE positions of kept tokens stay the original ones, and DPad stacks with parallel decoding and the prefix cache. It also stops at the first block that contains eos.
Loading PDF… open it directly
PDF reproduced unmodified under CC BY 4.0. Highlights and notes are mine.