Into Production · P05
Parallel decoding
How many tokens can we commit in one step?
Tokens committed in the same step are sampled independently (M05). A fixed number per step is a blunt tool: too many on a hard stretch, too few on an easy one. Fast-dLLM lets the model’s confidence decide.
Commit what the model is sure of
Each step, every masked token gets a confidence , its top-1 probability. Two rules:
- Threshold: commit every token with , and always at least the most confident one.
- Factor: sort confidences and dynamically commit the largest with .
Lower finishes in fewer passes but commits tokens before their context is settled, and some come out wrong. The paper uses .
Why high confidence makes it safe
Theorem 1 (Fast-dLLM). Suppose tokens each have marginal confidence above . If , then greedy parallel decoding (argmax of each marginal) picks exactly what greedy sequential decoding (argmax of the joint) would.
So the more tokens you commit at once, the surer each must be: one token needs no bound, two need , four need . The factor rule is this bound turned into a decision, and the log in the demo checks it for every step.
References
- Wu et al. Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding. 2025.