dLLM Notes

Into Production · P05

Parallel decoding

How many tokens can we commit in one step?

1 min read

Tokens committed in the same step are sampled independently (M05). A fixed number per step is a blunt tool: too many on a hard stretch, too few on an easy one. Fast-dLLM lets the model’s confidence decide.

Commit what the model is sure of

Each step, every masked token gets a confidence cic_i, its top-1 probability. Two rules:

  • Threshold: commit every token with ci≥τc_i \ge \tau, and always at least the most confident one.
  • Factor: sort confidences c(1)≥c(2)≥…c_{(1)} \ge c_{(2)} \ge \dots and dynamically commit the largest nn with (n+1)(1−c(n))<f(n+1)(1 - c_{(n)}) < f.
    Toy model, illustrative numbers. Bars are the confidences of the still-masked tokens for the next step; the dashed line is τ, and orange bars clear it. Each committed token makes its neighbors more confident. Drag the slider and run all steps.

    Lower τ\tau finishes in fewer passes but commits tokens before their context is settled, and some come out wrong. The paper uses τ=0.9\tau = 0.9.

    Why high confidence makes it safe

    Theorem 1 (Fast-dLLM). Suppose nn tokens each have marginal confidence above 1−ϵ1 - \epsilon. If (n+1) ϵ≤1(n+1)\,\epsilon \le 1, then greedy parallel decoding (argmax of each marginal) picks exactly what greedy sequential decoding (argmax of the joint) would.

    So the more tokens you commit at once, the surer each must be: one token needs no bound, two need ϵ≤1/3\epsilon \le 1/3, four need ϵ≤1/5\epsilon \le 1/5. The factor rule is this bound turned into a decision, and the log in the demo checks it for every step.

    References

    1. Wu et al. Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding. 2025.